This site relies heavily on Javascript. You should enable it if you want the full experience. Learn more.

Kinect v2 HDFACE

demo
Credits: for work this patch needs dx11 a kinect2 packs installed, and to make it work, take the nuidatabase folder (that should be something like "C:\Program Files\Microsoft SDKs\Kinect\v2.0_1409\bin\NuiDatabase" and you have to copy the entire folder to your kinect2 pack folder

about

Oh, and if you press spacebar you can save the actual face, and use it as a texture for the facemodel (so for example you can give your face to someone)

download

HDFaceBufferDemo.zip
22.09.15 [23:48 UTC] by joreg | 1747 downloads
demo for new HDFaceBuffer node

Show 1 older revisions

Older Revisions

KINECT_FACEHD_EXAMPLE.zip
12.02.15 [21:27 UTC] by screamer | 1030 downloads
kinect v2 face tracking example

screamer 13/02/2015 - 00:06

also, you will need (DX11.Texture2dArray) contribution:
setslice-(dx11.texturearray)-(dx11)

Takuma 09/04/2015 - 14:10

Hey, I guess HDface node is missing (inside Face patch). or might be my problem.

kbln 25/05/2015 - 17:21

HDface is missing =(

esnho 04/07/2015 - 23:30

HDface missing!!

screamer 06/07/2015 - 14:00

unfortunately i cannot try it now, but it worked for me.
Have you copied the NuiDatabase folder from the sdk path to the kinect 2 pack, as written in the credits of the contribution?

....for work this patch needs dx11 a kinect2 packs installed, and to make it work, take the nuidatabase folder (that should be something like "C:\Program Files\Microsoft SDKs\Kinect\v2.0_1409\bin\NuiDatabase" and you have to copy the entire folder to your kinect2 pack folder

let me know, anyway i'm planning to try it out with the new version of vvvv and kinect2 pack as soon as i can

chriss0212 07/07/2015 - 17:08

in my case i have a red split node?!

thx and greetz

christian

esnho 08/07/2015 - 13:22

I understand, wich vvvv and kinect2 nodes are you using? I'm up to date :)

The split node should be in DX11.Texture2dArray contribution, is linked in the first comment.

chriss0212 08/07/2015 - 14:22

hi esnho,

the split node seems to be a complete (split.v4p) patch what ist not in the link.

i am using a new kinect2 and last vvvv version.

greetz

christian

screamer 09/07/2015 - 04:08

yep, sorry, my mistake.
split is in the mremod2 contributuion (but not really needed for the face to work)

esnho 22/07/2015 - 11:58

Thanks @screamer, and the HDFace node? Where we can find it? Is the HDFaceBuffer node? How it should be patched?

soriak 11/09/2015 - 19:24

having the same problem as @esnho , did you get it to work?

screamer 14/09/2015 - 12:16

i'm sorry, but something is changed from when i've done it, and now i'm in your same situation, it dosen't seems to work. if i'll find a solution i'll let you know

screamer 14/09/2015 - 13:13

So, i've understood how the hdfacebuffer works. To work it needs that you connect the "user index" pin from skeleton node, to "Tracking id" pin on hdfacebuffer node.
In that way, you can see the values of hdfacebuffer moving, so the head is readed.
But here is where i'm stopped. Because don't know how to use the buffers sent out from hdfacebuffer,
that are:

face vertices
face uv
geometry

the values from here are all what we need to create the mesh of the head, but who know how? hope that vux, or someone else with this knowledge can help us understand ;)

microdee 16/09/2015 - 16:42

Ooh nice vux might have released the multi version of hdface he made for us. In that case in your shader you have to read vertex position and uv's from those buffers via SV_VertexID coming from the geometry.

screamer 18/09/2015 - 12:32

hey microdee, do you have an example? i'm really stoppe here, and my knowledge about shaders is not enough :)

microdee 21/09/2015 - 23:26

Unfortunately i might only have an older version of that. I don't really have time to make an example though I'm flooded with work :(

joreg 22/09/2015 - 23:52

added a basic helppatch for HDFaceBuffer. still the tracked face is somewhat off which seems to be related to some missing coordinatemapping. it seems though that BMinXYZ and BMaxXYZ can probably be used to achieve that. any clues on how to use those @microdee @vux?

screamer 23/09/2015 - 13:04

wow, thank you joreg, that's a perfect starting point, thank you so much

screamer 23/09/2015 - 13:39

don't know why it's shifted, but it happens only with rgb.
Trying with ir or depth instead and it is perfectly aligned.
A walkaround that works perfectly for me is to add a translate after the perspective node, and translate the camera view of x: 0.053 and y: 0.004
in that way the face is perfectly aligned with the rgb image. I've tried to understand why that numbers, and tried to compute all the numbers i can together to achieve that numbers, but no luck, anyway it works perfectly ;) thanks again

joreg 23/09/2015 - 14:13

here is an explanation for the offset: https://social.msdn.microsoft.com/Forums/en-US/14b9bc56-e890-4652-ae65-6c5dea80a432/doubt-with-hd-face-basics?forum=kinectv2sdk

mrboni 12/04/2016 - 17:43

I'm trying to make a geomFX to output a dx11 gemometry from the buffer data from the HDFaceBuffer node, but it doesn't work - a shader will complain there is no POSIITON and TEXCOORD in the geometry, which doesn't make sense.

Any idea?

Patch here https://www.dropbox.com/s/s28arp8i88b9agv/HDFaceBuffer%20geomFX.zip?dl=1

everyoneishappy 14/04/2016 - 10:10

sounds like something wrong with the streamout setup. There is already a gsfx for this in Instance Noodles if you want to check there.

https://vvvv.org/contribution/instance-noodles

mrboni 14/04/2016 - 23:56

well so there is. thank you indeed

mrboni 15/04/2016 - 00:15

one wee question - the PN tesselator doensn't seem to work on the resulting face mesh for some reason. Any idea why?

(It works fine on a sphere geometry)

edit - also the tesselator displace makes all the polys separate - common vertices don't stay together

everyoneishappy 15/04/2016 - 04:44

Not sure. Sounds topology related

Noir 10/06/2016 - 12:00

Same situation here...it doesn't seems work gsfx and the geometry from the buffer data from the HDFaceBuffer.
Any tips for this?

avaos 10/02/2017 - 16:19

I also tried to read the vertices of HDFaceBuffer, but the problem is that i can't even readback the buffer. I tried multiple settings with different ReadBacks.
all I can get is 506 X-Coordinates. Not a single Y or Z!
I don't know the right setup :(

https://www.dropbox.com/s/qbhg881ykjpj2qm/Kinect2_HDFace_readback.zip?dl=0

avaos 13/02/2017 - 17:16

@Noir: I Fixed it: Try everyoneishappys InstanceNoodle Pack, there is a Kinect2Face Shader, with this you can use getVertexData. This works for me.

Noir 13/02/2017 - 18:01

@avaos thanks man
I did the same 'thing' a month ago ehhehe

Kinect v2 HDFACE

about

download

Shoutbox