OpenNiHands - trying to manage Hands and gesture event

Hi VVVVolks!

I’m trying to integrate Hands and Gesture recognition to OpenNISkeleton plug-in. (Or just create another plug-in that can handle this)

I started with Sample-PointViewer.exe in the NITE\samples folder, but is written in cpp and it’s hard for me to port it to VVVV cs.

Googling on the OpeniNi Group I found this thread:

http://groups.google.com/group/openni-dev/browse_thread/thread/9ce6c8a2e27ff22f

You can download the lastest version of this cs application from here:

http://dl.dropbox.com/u/1966569/kinect/kinect.rar

It’s the 5th day I’m struggling with the code to understand what to put where, trying to translate it to match the VVVV OpenNiSkeleton Node structure, but no luck.

Update:

The cs example linked above uses a public class called “Hand” along side the main class “Main Windows” (that in VVVV I think could be the main class “public class OpenNIHandsNode” under the namespace VVVV.Nodes)

That class refers to “: ViewModelBase” and I can’t find which namespace to use lo let the code find it.

If anyone can help me, thanks

Cheers
Robe

PS: Sorry for my poor english

OpenNIHands.zip (156.3 kB)

hi robe, just had a quick look at your kinect.rar and i’m not sure that’s a good point to get started. it uses wpf for the gui (so you’ll have to learn that before going any further). this ViewModelBase class you’re refering to is in Dlls/GalaSoft.MvvmLight.WPF4.dll. probably contains some custom classes for wpf, i don’t think it has anything todo with kinect. this hand class probably inherits from it in order to serve as the model in wpf’s model-view-presenter pattern. this just means it will fire events everytime a hand object changes so the user interface gets notfified and can update itself. but like i said, you’ll need to learn wpf a little in order to understand it fully (i’m not an expert in this either).

now to your plugin:
i see you copied the relevant code parts from the kinect sample but forgot to declare some members, like AvailableHands. i didn’t try to compile you’re code, but there’re probably more such errors. how did you write this code? with our code editor? maybe that’s not the best idea, because we lack a proper debugger. use sharp develop 3.2.1 for example and open up the project file with it. but before you open it, make sure you set the environment variable VVVV45 to point to your vvvv installation, so references can be found. then try to compile it. you’ll propbably get errors, if you can’t solve them on your own report back here. i can help you to at least get started, no clue about the kinect stuff though.

Thanks a lot Elias. I’m doing everything you suggested.

I was using Microsoft Visual c# and if I set VVVV45 as environment variable and went crazy in the prefs I couldn’t find VVVV.Core, VVVV.Utils etc. So I switched to the VVVV internal code editor and ““debugging”” pressing CTRL+S… :-C

Now I’m going to install SharpDevelop, I see the newer version 4.0 in the download section.

I’m not a # programmer, really I’m not a programmer (except some stupid maya mel things), but I’m really motivated to start learning.

Probably this isn’t the simpler project to start, but I want to try.

Thanks again
I’ll keep you update on my improvement

Cheers

visual studio should work too, but never tried it. so to keep the errors you can make to a minimum try sd 3.2.1 (not 4, it works with 4 too, but there are some additional steps necessary).
the environment variable VVVV45 pointing to your beta25 installation is set via windows, not in the preferences of sharp develop for example.
if the environment variable is set, open up the csproj file, press F4 (properties pad) and if you click on the referenced assemblies (like VVVV.PluginInterfaces) it should show the correct path. see attached screenshot.

after this is set, switch to project properties -> debugger and set vvvv.exe as startup program (see 2nd screenshot).

set a breakpoint in the constructor of your plugin, and start debugging (by pressing F5). vvvv should start up and you can drag the compiled dll (located under <project_dir>/bin/Debug) on your patch. the application should stop at your breakpoint.

as an alternative it is also possible to use our code editor together with a debugger. start vvvv, start SD, open up the project file, set the breakpoint, go to vvvv, open up project explorer, open the project file in vvvv (either by opening up the code editor or right clicking in project explorer), switch to debug in project explorer (so your dll is compiled with symbols by vvvv) and again the application should stop at the breakpoint set in SD. i’m not sure if SD or the code editor will refresh the code if you edit the code in the other application, but can try.

well but i recommend the first option to get you started.

Thanks a lot Elias
Very accurate!

Using the first option you describe with SharpDevelop

Debugging OpenNiskeleton (by hierro vux screamer jonferran)
I got this error:

and 18 warnings related to “pins” like:

[Input("RenderLabels")](Input("RenderLabels"))
        IDiffSpread<bool> FRenderLabelsIn;

and logging:

[Import()](Import())
        ILogger FLogger;

that will always have it bla bla bla null

I also notice there’s a variable “private bool reading” that is never used!

I’m sure the warnings are negligible but what about the error and how to solve this?

Thanks again

I tried with /o arg in the Command line arguments field of #develop and it works…

the magic of /o
:o)

But I’ve always to build a patch to test it?

Or at the most using a prepared patch I’ve always to reconnect the pins?

Thanks

well we fixed this error for the upcoming release. to get rid of it you need to set some arguments.
you can safely ignore those warnings. if you don’t want them to appear in the compile output either disable them in the project settings or in the code (see this).

see nodes-and-paths to set up vvvv in a way that some test patch + your plugin is loaded at startup if ctrl+o is too annoying for you ;)

wow!!

this is a new 45-25 feature! () I think

sorry I didn’t read the Change Log!
I’m a nevvvvbie and I really trust all you, devvvvs!
;-D

nodes-and-path open a universe of possibilities for me to make order in my messy VVVV installation path, separate the contribution and the effects from that folder etc…

I always wondered how I would do when a new release of VVVV comes out.
I’ll make some order from now…

Sorry for my english and my newbieness

And Thanks to Elias

PS: ctrl+o is not a problem for me:)

got hand tracking work on this plugin:
https://kimchiandchips.googlecode.com/svn/trunk/vvvv%20plugins/VVVV.Nodes.OpenNI

it’s specificallly made for Kinect Hadouken project:
http://www.kimchiandchips.com/blog/?p=544

At some point i think it’d be good to collaborate (hierro + others) on a modular version which exposes all the different OpenNI capabilities within the vvvv graph.

definitely!

Elliot:
does the Pade part could be achieve inside vvvv
to create the projector XY / WorldXYZ thing ?

oh yes, a modular version would be fantastic :DDDDDDDDD

im actually busy for a deadline, but defiteley i wanna work on it, was waiting for vux planning :D everybody have seen him ? :P

@sugokuGENKI:
Thanks for sharing… I’ll take a look

Thumbs UP for:

Bye

@sugokuGENKI:
I’m compiled SVN version of your kinect plugin, but can’t run it. I’m debugged it and looks like something wrong with HandsGenerator. Debugger stops at line 499:

this.FGeneratorHands = FContext.FindExistingNode(NodeType.Hands) as HandsGenerator;

And looks like it’s can’t found any Hands node, so it’s produced unhandled exeption. What’s wrong?

UPD: The problem was in my config file. Now all ok. I think including sample config in sources will be good idea.