Color tracking in 64bit v4 and optimizing cpu load

Hi,

I made up a patch in 32bit v4 45beta34.2 where I use the ColorTracker freeframe plugin to follow a color in a video input of a usb camera. I want to track a white ball moving around. The tracking itself works well and is very simple. The object basically is moving from left to right. If the delta of x is above some value I toggle a tracking detected value.
But I now want to optimize this patch because it uses 40% of my core i5-3570 3.4GHz CPU.
I can not change the resolution and fps of my camera. It’s a Microsoft LifeCam Cinema. It has great delay of 2 seconds all the time. I turned of every automatic image processing in the camera properties. I changed the reference clock of VideoIn. I changed Texture Size Mode to NonPow2 of VideoTexture and Wait for every nth frame to 2, 0, 900. I changed presentational interval of the Renderer to immediate. Nothing is changing. I tried this to get better performance.

So I have some questions:

  • Is it normal that VideoIn with a USB camera uses so much CPU?
  • Is it possible to lower the resolution to gain performance? I tried to lower backbuffer width and height of the connected renderer but it seems there is no difference in performance.
  • I want to use it in 64bit v4. Is this possible? Can I replace the ColorTracker with something like Pipet or Levels or Keying? I tried but don’t know how. Does this help on performance?

I made a patch with simulation to show the actual tracked object.

32bit v4 colortracking with videoin and simulation (52.7 kB)

For the first question. If I have Force Preview Output switched on in Inspektor, the load is much better.

I came up with something for the color tracking. I don’t know if this is good but it almost does what I want. I now get a spread of colors of a region within the camera. If the colors match, it toggles a value to 1.

My problem is now that there is a spread of values which are either 1 or 0. Is it possible to toggle some other single value to 1, if half of the slices are 1?

I also try to move the tracked area horizontally. Therefore I need to change the LinearSpread Input and Phase. But I could not come up with something that would kind of follow the quads x position.

with pipet (27.7 kB)

I solved the problem with the spread of values by using Add (Value Spectral) and doing a Greater Than comparison. Now I can select the amaount of hits I need for a successful detection.

Now I only need to be able to change the position of the tracked region, or even better get the position of the color found like in the freeform ColorTracker node. My try is of course not really good because I don’t get the position of a tracked color. I just managed to use it for my purpose. I’m therefore also still interested in better ways of doing this. Is this an OK way of doing a simple kind of color tracking in 64bit v4?

you can try dx11 pipet it much faster…

hi stephan,

for best results, i’d go for the imagepack. while there’s no dedicated node for colortracking (like the freeframe one), i can think of a procedure like this guy seems to use https://www.youtube.com/watch?v=c4Dfvn6VQMo. basically, the thing could be done in 2 steps:

  • extract a desired color-range from the image like explained here for example. that could ideally be a new filter node living in the imagepack. if you look at similar, already existing nodes like this one, you can see how to adapt the stuff from the link above into a new node.
  • run blobdetection over the result (with Contour (CV.Image) )

pros

  • fast (imagepack handles multithreading for you and also videoin is much faster)
  • no pipet homebrew
  • tweakable in many ways (like filtering the image with other filter nodes)

cons

  • color-extractor node must be coded (but this should not be too hard. and there should be many examples out there)
  • for now, the imagepack only supports x86 (but 2 vvvv instances can solve this)

He sebl,

I’m so happy to read something from you here :)
Thank you for the detailed answer.
This seems like a new journey for me I have to take. The next level for my poor v4 skills. I’ll do my best to frickl something together and post it here.

Is it right tough, that there is no ready to use color tracking in 64bit?

yes, i think that’s right. anyway, the freeframe stuff isn’t state of the art nowadays :)