Tracking blinking LEDs

Hello evvvverybody,

I’m trying to track different people on stage, and somebody suggested me to do this by tracking IR-LEDs blinking with different frequency. Do you know any ressources how this can be done? Any hint on nodes?

Greetings,

Alex

Hey Alex,

Sounds like a cool idea, but could be hard to do.

How many people do you need to track? For higher prices you might get (a little) different IR-LED-Colors - would solve some problems - I don’t know much about this.
Or we could have different Shapes with LEDs on the persons - e.g. a triangle, a square, maybe a pentagon, hexagon.

As Hardware a Wii-remote or a Kinect Camera would be the cheapest solutions. Kinect seems be used often with vvvv, but has of course much more possibilities.

Identifying something that is turned off regularly while moving will be hard - simplest solution to use two LEDs and only one blinking.

Stage - sounds light a very bright place - a LED emits as much IR-Light as two candles and much less than a 60W light bulb.

Kinect offers face-Tracking
Face-Tracking: http://www.codeproject.com/Articles/394975/How-To-Use-Kinect-Face-Tracking-SDK
Samples: https://kinectforwindows.codeplex.com/#list_of_samples

Cheers
Paul

Hi yau,

Thanks for the reply. Well most of the times it will be about 3 to 4 people. That would be nice. More people to track would be even more nice. Till now I tried to light them either with ordinary stage light, or the projection folia from behind with ir light (and absolute IR-Darkness in front of the screen). That captured with an IR-Cam and connectted to a contour node.

The problem is that I don’t know how to seperate the different actors from each other, for example if one leaves the stage, or when they cross ways.

Maybe your Idea with the IR-geometric figures would work, but the computer would have to recognize them, even though they get perpectively distored by the actor.

I’m not worried about about the IR emmiting lights on the stage. We could use LED-Lights that hardly emmit Near-IR-Light.

Anyway. No matter if we would use blinking LEDs or geometric figures. I don’t know how I could analyze the image captured by the cam. I would need something that counts the times a part of the image gets bright.

Is it worth to have a look at kinect? It seems not to cover enough distance, even fpor smaller stages, lets say 8 by 8 metres.

Greetings,

Alex

Hi yau,

Thanks for the reply. Well most of the times it will be about 3 to 4 people. That would be nice. More people to track would be even more nice. Till now I tried to light them either with ordinary stage light, or the projection folia from behind with ir light (and absolute IR-Darkness in front of the screen). That captured with an IR-Cam and connectted to a contour node.

The problem is that I don’t know how to seperate the different actors from each other, for example if one leaves the stage, or when they cross ways.

Maybe your Idea with the IR-geometric figures would work, but the computer would have to recognize them, even though they get perpectively distored by the actor.

I’m not worried about about the IR emmiting lights on the stage. We could use LED-Lights that hardly emmit Near-IR-Light.

Anyway. No matter if we would use blinking LEDs or geometric figures. I don’t know how I could analyze the image captured by the cam. I would need something that counts the times a part of the image gets bright.

Is it worth to have a look at kinect? It seems not to cover enough distance, even fpor smaller stages, lets say 8 by 8 metres.

Greetings,

Alex

coolux some weeks ago presented “Real-Time Tracking ID-Tags” for their PandorasBox media server system. These tags and the associated system sound exactly as what you are looking for …
http://www.coolux.de/cooluxsupport/tech-blog/blog/detail/real-time-tracking-id-tags/
Unfortunately i am almost 100% sure that they are not available seperate from their mediaservers and that they are not willing to share the code for the tracking … so this post is more or less just fyi.
As i have not seen or used these tags yet, i can not say how good they are working.
If you have a chance to get your hands on them it would be great if you could share your experience.

Markus

@MSBERGER
Wow… that sounds like a cool system. I’m sure it’s way beyond what a german theatre can efford. Do you have any experiences with that?

Actually I’m working in a show in prague, tracking 4 people on a stage 8x8mt using 2 kinects. As long as they:
don’t move too fast
when go out of the stage then enter from the same point

I’m able to separate each dancers. I’m pretty sure that with actors (not performing strange movement) is possible to have even more people.


kinect1+kinect2 —> position on stage (vertical lines are just for reference)


tracking of each different dancers

The idea is, I have X and Z position for each dancers, and every frame I assume that each the identity of each one is the nearest one in the frame before. I guess is not so clear but if you’re interested I can try to explain better!

Hey Luper,

That’s great! I’m very interested in your solution. I didn’t really understand from your graphics where you placed the devices. Downstage at the front? Did you put the on the floor? How close and how far can the dancers be?

Greetings,

Alex

Okay,

I finally had some time to try something. That’s not related to that kinect-solution though. I’m sure there are a lot of vvvvers out who can fill the gap in this attempt.

I’m using contour to get the position of the blinking IR-LEDS. Together with Pipet I can see now if the Pixel is white or black. I count the times the pixel turns black at every second. (This works fine with lower frequencies).

For now that’s everything I wanted… BUT…

Since the LEDs are blinking Countour is sending me a lot of those nasty NILs. Countour constantly looses it’s tracked markers. I would need a way to sample the position of each blob only when its visible… As I’m writing this I am not really satisfied with my description of the problem. Please look at the patch. It needs a procedure to hit that S+H at the right moment (when both Lights are turned on)

Greetings,

Alex

frequency_measure.v4p (37.3 kB)

here, the stage as seen from above, kinects are front stage on the ground (15 cm above the ground).

In yellow the area covered by kinects, as you can see there are 3 little hole in the front, and on the end of the stage is kind of discontinuous. Lets say you have 8x7mt perfectly covered

Luper,

This is really great. Can you only track 2 dancers per device, so that not more than 2 can be within the view of one kinect, or is it possible to have lat’s say 3 dancers right and 1 left?

I just started checking out the Kinect Nodes. I guess you are using the User (Kinect OpenNi). When you use 2 Kinects, how do you calculate their XYZ-Information to match the real world space. Do you simply offset the X Coordinates?

Greetings,

Alex

I use the DEPTH texture, in order to track more then 2 dancers for each kinect… theoretically I should track 8 or 10 ppl, but depends on how fast they moves, and if they stay nicely separated, and they don’t cover each other and so on.

So I have X,Y,Z of every points with pipet (EX9.Texture) and I draw only X,Z for each kinect (as you can see in the first image)
Then with Contour (FreeFrame DShow9) I obtain the position of each group of point, I join the 2 images and I have the result.

when I have to map this position in my virtual space to the real stage, usually I need couple of dancers move around and a couple of Map (Value) :)

I’m trying to recreate what you’ve done, but I’m lost. I need some more hints, I guess. I cannot even reproduce your first image. How exactly do you draw the X and Z values. Are you spreading Quads with Transform (Transform 2d) or 3d, render that to get it as a texture again for Contour (FreeFrame DShow9). That doesn’t even work for me. I’d need to blur that texture really bad to get out one blop per performer.

KinectTracking.v4p (12.6 kB)

Blinking Leds: Seems to me it would be better to not blink them fully from on to off but instead pulse from medium to high intensity at different frequencies so you never have them disappear unless you really lose them because they’re covered or some other reason.

Hey, you did everything right, only you should use Transform (Transform 2d) because you wanna see the stage from “above”
btw you can also use the 3D and just rotate the camera, but its simpler to use the 2D

unlucky I cannot try your patch 'cause I have all the gear in theatre, but here you have how I set the Contour, hope it can help you

EDIT
also, give here a look: kinect-virtual-hit-boxes

contourPARAMETER.v4p (11.6 kB)

@ beyon

Thanks for the tip. That helped a lot. Still that thing is quiet buggy. I don’t see why I can’t keep a good track of the frequency. And I don’t understand why the colors of the spheres are changeing sometimes.

@ Luper

Thanks for the help, mate. I will try your settings right on stage, and see how it works.

frequency_measure3.v4p (57.3 kB)