pack dx11 kinect2 kinect
Credits: Microsoft of course, and Flat Eric doing all the job while I was just drinking beers watching him coding ;)
Pre node15 build, new features and some bug fixes, still badly lacks documentation, I know ;)
Disclaimer (from Microsoft):
“This is preliminary software and/or hardware and APIs are preliminary and subject to change.”
Same disclaimer for me, don't complain if a pin gets broken in a future release ;)
simply put the \kinect2 directory that is in the .zip in your \packs directory.
Thanks vux, this is great for those enrolled in Kinect 2 dev. program. the rest of use have to wait till 2014...or does it work with current Kinect? haven't checked.
No you need a Kinect2 , it's totally different api.
just to let you know: there was an arrayoutofbounds from the player node for some reason.
Thank you very much vux!
why the depth is red?
Wow thats great...anyone got a CS or Gsfx to reconstruct the world space coordinates...concerning bugs: i got array out of bounds errors on skeletons too and on the player textures...
About out of bounds it will be fixed in next release.
I'll also add world as per old kinect.
Depth is red because L16 format don't exist in dx11, it's R16 for single channel (I could have done a shader to change display to white of course).
hey that sounds great...if you add the world functionality...does that mean you know the instrinsics of the camera?...i couldnt find this information. Or do u use native calls of the lib to get this information. Thank you very much
Any trick on getting this to work?
I've got a dev release Kinect2 and am using vvvv_45beta31.2_x64 with the DX11 nodes installed. I copied the kinectv2 folder next to my girlpower folder but whenever I open KinectTest.v4p I just see a lot of red nodes.
Hi - after getting a new USB 3.0 card (the onboard one crashed everytime after seconds) i finally got the kinect2 working - with the windows program 'kinect status'.
The patch seems to load fine, no red boxes - but the output is kinda dissapointing . Did anybody run into the same situation/has an idea how to deal with this?
did you start Kinect Service?
right, thank you!
Can't get it to work...
I can get data from Kinectstatus but not from vvvv.
I supose we have to wait until April SDK before a node update, right? :P
@Jose the Kinect service is running yeah? (sorry to repeat, it's easy to forgot though)
yes running... :(
Just set up Kinect 2. Installed new sdk (March update). All samples and kinect studio are working.
vvvv plugin nodes are red though
00:00:09 ERR : Exception occured during creation of plugin: Could not load type 'Microsoft.Kinect.KinectStatus' from assembly 'Microsoft.Kinect, Version=188.8.131.52, Culture=neutral, PublicKeyToken=31bf3856ad364e35'
any ideas? maybe the sdk update has broken the plugins?
Hello all, i am can not found how connect kinect 2 to pc? Is it special kinect2 for win developers or specific adapter or what?
@fedrbodr: yes it is the Kinect 2 Dev kit, not the Kinect 2 that came with Xbox one (I think). The dev kit has a special USB 3.0 jack.
@metrowave kinect2 came with Xbox one works like kinect 2 dev kit. It's the same device perhaps the difference would be only in adhesives on the dev kit kinect...I have try in v4 both devices without notice differences.
(but u need to have the adpter included in the dev kit)
I can report next week if I found differences
I've just been trying to get the Kinect2 working with vvvv. I've downloaded the VVVV.Nodes_.Kinect2.zip file from this post and the latest (April) SDK from Microsoft.
I've tried putting vux's files in a contributions folder (although I am unsure where each file should fall within that) and also the
folder as a guess.
I can get something from the sample applications in the Microsoft SDK pack so I know the camera works, but I can't see anything in vvvv - only red nodes!
Any help would be great, thank you!
Make sure you download:
64 bit vvvv
64 bit dx11 pack
dx11 pack goes: \vvvv_45beta31.2_x64\packs\DX11
so you seem to be good with the placing at least.
The kinect patch and its dll can be anywhere.
Do not forget to run the kinect service.
Make sure to use the right vvvv if you happen to have
also other versions installed.
that should do
Whoever replied to my post, thank you for the help!
I've got 64 bit vvvv and DX11 pack in the right place.
I've got the Kinect patch and its dll in: vvvv_45beta31.2_x64\contributions\plugins
Just checked the services - Kinect Monitor is running in Automatic start-up (I assume that's OK).
I ran the patch and only some of the Kinect2 Microsoft nodes are red now (see attached screenshot).
Ok you seem to have dx11 working.
Now it seems only the kinect stuff is not present.
So vvvv can not see the "VVVV.DX11.Nodes.kinect2.dll"
Is it in the same folder with the "KinectTest.v4p" you click?
I think it should be. Else i am not sure what is the problem.
Yep, I have kept the KinectTest patch and dll file together and tried opening them from the root of the vvvv folder, from a folder within the root and from the desktop as well.
I will keep trying, but thank you for all your help!
@ CSmith16, did you get it running in the end? Your image aboveshows you're running the x86 vvvv again.
For me it works with the 1311 SDK..
Finally getting around to trying this, but getting all red Kinect nodes. The SDK Kinect Studio program works fine. Getting these messages on the console:
vvvv beta32.1 x64 w/addons, DX11 pack 32, and the "public preview" 1407 release of the Kinect v2 SDK. Has anyone tried this version of the SDK? What's the latest known to work? Thanks folks!
This hasn't worked for a while. Check the DX11 contribution.
I'm pretty sure Vux will doing an update soon as Kinect2 is about to be officially released!
It is released! Anyone can download the SDK now, and two units arrived here today. Through contacts at MS they are saying these are the "RTM" (release to manufacturing) units.
What DX11 contribution are you referring to? The only ones I can find other than this one are for the K1. I'll try one of the older SDKs. Thanks Gareth!
Bad news, the older SDKs (before June) do not recognize the new Kinects - they are listed in the device manager as "XBox NUI sensor". These are the SDKs that do not throw any errors with these nodes.
So if you have a new (pre-order) sensor, no joy. Looks like the SDKs that work with these nodes will only work with the developer preview Kinect2s.
BUT WAIT! Here's the really weird part: The XBox One Kinect I hacked with the USB3/12V cables, works just fine under any of the SDKs, including the new one! So it WILL work with these nodes, and the April SDK. But the ones you can buy right now won't. Go figure!
Could you expain how kinect one for xbox was hacked (some photos or diagrams maybe?.
is there possibility to make that depth texture work more like in older kinect? this red thing is killing me :3 doesnt work with stuff like dx11 hitbox detection from andrej boleslavsky etc.
ooor any advice how to make it work
and any news on development of this? i have here 4 new kinects waiting to get working :/
OK, found it in kinect2pc thread
When is a new Kinect2 node expected? The device arrived today, would love to use it with vvvv!
checkout the dx11-vvvv pacck and you're done. if the nodes don't work, compile it yourself (instructions are also on github alongside the sourcecode)
a node that support gba/gbd format could be really cool !
Will this work only in Windows 8 and 8.1 ?
Latest update now (public sdk), please test (I can't find the example patch again, but this is rather straightforward anyway ;)
mm dlls open as strings when dragged on a patch ..
@io : Ok some of this version weirdness, download is now updated.
@vux - Works like a charm! 32.1 x64, latest DX11, and latest K4W v2 SDK (1407) - with both K4W v2 and hacked XBox One Kinect. You da man!
Now if the K4W v2 SDK would support more than one device... sigh.
@mediadog does the fusion & face nodes works for you?
the skeleton works here except for the glasses/neutral/happy behaviors
@guest - I didn't try those, just the demo patch (all renders there had sensical images), and my depth-to-pointcloud code.
HSCB and turn up the brightness.
The Skeleton and Hand nodes turn red. I installed Kinect SDK 1409 (Sept 2014). Could someone pls share Kinect SDK 1407 with me? I can't find it via google nor on the microsoft page (they should provide older versions of the K4W2 SDK).
Using Win 8.1 x64
if i remember well enough, in kinect 1 player node was capable of producing separate texture for every player, so you could for example have 4 silhouette with different colors for every player
in kinect 2 thou you have 1 texture for all players
does kinect 2 support this feature?
@StiX: +1, would beinteresting for me, too! for the moment you can use a color-spread as input for the player-node to differentiate between the different users.
I have the Kinect v2 running here, work's well. But sometimes there is a strange behaviour. I'm not sure when it apprears, but mostly when multiple users pop in randomly or when it's difficult for the kinect to detect my skeleton (eg sitting on a chair).
The Kinect stream then seems to be freezed for many seconds (not sending any new data at all / holding the last data), while all the other animations in vvvv are running at 60+ fps.
Any ideas? How can i debug that?
@katzenfresser: +1 my cam also crashed with many users
kinect2 Xbox One sensor adapter for windows
there's also a nice paper on how to use visual gesture builder
same here, did work before.
latest vvvv64, sdk, dx11pack
have you tried the IR node? i had today a problem just with this node (depth and rgb worked) and found the answer in installing the latest nvidia driver and the cuda 6.5 toolkit. not sure if this was the same issue like yours...
If some one want kinect2 nodes on latest kinect drivers and on 8.1 x64
I have problems to use vux nodes so i rebuild them.
Thanks vux for your work!
i installed the latest drivers & cuda toolkit - tried noobusdeer (thanks you!) and vux Microsoft.Kinect.dll, still the same: no skeleton, no hands.
the kinect studio works fine, vvvv drops that:
Noobusdeer’s rebuild works fine for me, only downside is that it stops working with multiple Kinect nodes. For example, when open a (sub)patch with another Kinect node. It’s a pity, using multiple Kinect nodes can be quite handy sometimes.
Noobusdeer's rebuilded nodes works perfectly, maybe should be tell in the main post?
hey guys have anoyone worked with joint orientation? they seems to be all over the place, is it a bug or is it a feature?
did anyone get face tracking up and running in the meantime?
yes, i did :)
and the solution is quite simple, i've found it followinf the error i always had. And it's about some missing files needed for facetracking.
All you have to do is to take the nuidatabase folder (that should be something like "C:\Program Files\Microsoft SDKs\Kinect\v2.0_1409\bin\NuiDatabase" and you have to copy the entire folder to your kinect2 pack folder.
And that's it, it works, and also very well ;)
@screamer: yep, that's it! thx!
Just got the face tracking working with copying the folder, thanks Screamer, any idea how to use the vertices from HD face, do I need to use the vertex data to morph the candide model, and is there a obj of that model?
thanks screamer it works also on my side
Quick recap for the next guy looking for infos ,
you have to install the content of your zip in a folder named "packs" at the root of your vvvv folder
you will also need the dx11 pack to be installed in the same place
i still have a problem tho , i cant get the kinect2 node to appear by starting it from vvvv i have to start the help file from the kinect2 folder :(
@LineKernel: it worked for me when I was copying the Kinect2 folder into the packs\dx11\nodes\plugins folder as we did it in the Kinect-Pointcloud workshop. Then the nodes appeared without using the helppatch "workaround".
I'm very new to VVVV, so this might be a very basic question. How to install the those Nodes? I was able to test the patches from the kinec2 folder, but can't add the kinect2 nodes to new patches. Also some nodes don't have any pins
(..appear red first then turn grey).
Would be glad if somebody could explain what I have to do. Thx!
@StiX regarding joint orientations please see: skinned-skeleton-and-kinect
@panta_rhei, @LineKernel, @couchmonkey, @esnho regarding installation of this pack: i just fixed the pack structure and you can now simply: put the \kinect2 directory that is in the .zip in your \packs directory. that should do.
rgb and depth nodes are still red/pinless-grey here, with the 33.7 version
is there still a need for older sdks ?
_edit solved, was using 32bit ver
hi guys, totally new to working with kinect2 here. this thread seems really helpful! only thing is, i cant seem to find 'kinecttest.v4p' anywhere! i just want to get up and running really..!
@subplexus: check \packs\kinect2\nodes for a series of helppatches
Thanks Joreg! I'm up and running now. Incidentally, I notice that in KinectTest.v4p the tracking is not quite as smooth as in Microsoft kinect studio, also the open/closed hand readings are less reliable... Is that unavoidable in vvvv?
Just updated from a version I compiled myself a while back, and find that the Skeleton node User Index pin, which used to return a number, now returns a lengthy string of dubious utility - a long string of numbers. Looks more like a string representation of a resource handle or something.
Is this a bug? Why did this change?
Hi everyone, a strange thing happened when I'm trying to use the Kinect node. I'm currently using Win 10 and the vvvv version is beta 33.2 x64 with the 64bit version of dx11 pack. Initially, everything worked fine so I proceeded to save the patch. But when I opened the patch again, all of the patches turned RED. I tried to place the Kinect2 node but this time it's red, same thing with the Preview node. So I closed vvvv, reopen it and recreate the patch, suddenly it worked again.
So I experimented further: I closed the patch, but this time I opened vvvv first, then I open the saved file, viola! everything worked fine.
But I'm just curious; despite the workaround, what might be causing the nodes to turn red when I opened the patch first instead of opening vvvv first?
sounds like you have multiple instances of vvvv on your PC. by doubleclicking a .v4p in explorer you start the vvvv version that is registered to open .v4ps. run setup.exe of the version you want to start and look for the file association setting there..
Ahh so that's why. Thanks joreg!
"Kinect One" don't work with Windows 10, right?
I have to install windows 8..
i am positive this is just a myth
I have both versions of the kinect working on win10
I have both versions too
everything ok on my side on win10
My problem is:
I have the nodes of Kinect2 in red without pins!
What I have:
I have windows 10, the kinect One is conneted to usb 3.0 of the pc, I use Vvvv 34.1x86, addonpack 34.1x86, I have put the packs folder in the some way of the addonpack (is close to licenses folder) with inside kinect2 folder (version b33.7 pack) and dx11 folder (version dx11 0.6.1.x86).
Dx11 work on vvvv so it is ok.
in the menu appear all kinect2 nodes but when I choose one for example the first one (kinect: device microsoft) it is in red and without pins! with renderer TTY is write: Exception occured during creation of plugin "..\vvvv_45beta34.1x86\packs\kinect2\nodes\vvvv.dx11.nodes.kinect2.dll"
I tried with root to put the directory of the file, but don't work.
Someone can help me, please?
as the download page of the kinect2 nodes say, they only work in x64 builds. so you need to install vvvv x64 and the x64 packs.
also you said "kinect One" is connected to the usb 3 port, is that correct? if so, you need to use the kinect 1 nodes...
tonfilm: Kinect one is the newest, in vvvv it's called kinect2
ah, microsoft naming decisions deserve medals...
Ok! I have use the x64 version for Vvvv beta34 and packs: dx11 0.6.1 and kinect2 b33.7 pack.
Now, some nodes from kinect2 are ok. but the nodes: Kinect2, Hand, Gesture are in red and without pins, instead Skeleton, Face nodes are in red with pins.
What can I do? :((
Thanks a lot.
@T.Kit Did you install the latest Kinect SDK from Microsoft, and it works OK?
Also, you should NOT be using this Kinect2 b33.7 pack, as Kinect support is now in the DX11 pack - this contribution should be labeled as obsolete.
Thank you for your sharing.
I tried your x86 ver follow the instructions on github.
But something wrong with open _root_server (x64).v4p.
plz help if you have time.
plz help if you have time.
I was using 33.7. Right now is not the best timing, but i will try to take a look later.
Thanks a lot!
I made small modifications to the current Kinect2 DX11 nodes so they work with unplugged device and playback from Kinect Studio. I know some of you guys faced this issue before (@everyoneishappy, @princemio, @timpernagel)
Switching between live stream and injected stream got more robust as a side-effect. Now there is no need to reset the Kinect2 node when connecting/disconnecting Kinect Studio.
I've made a pull request toward DX11 master from Mr. Vux. Meanwhile download to nodes
(replace the .dll in your dx11 pack)
or compile yourself from my forked repo
anonymous user login