Video tracking - gestures

Hi All,

Wondering if someone could give me some tips on how to approach a design problem. I know there is nothing groundbreaking about my request, but I’m teaching myself 4V and would like to know how an experienced user would plan or construct the patch.

goal:
I want to have a grid of pixels (say 20x20) where I can “push” colours arounds by using gestures captures by video camera.

proposed setup:
My plan so far is to have a video camera (watching from above) a table onto which the grid is being projected (also from above). I will be using infrared filters/lamps so that the video projection is not seen by the camera (don’t want feedback!). I’m kind of picturing that someone might be able to wipe their arm or hand across the table and “push” the projected colours to one side. Almost like playing with a pile of M&Ms.

I’ve been reading through the past forum posts and come across some very cool (and distracting) examples. Please give me some pointers on how to start building!!!

Thanks, D.