» Kinect Depth Fill
This site relies heavily on Javascript. You should enable it if you want the full experience. Learn more.

Kinect Depth Fill

effect
Credits: vux (dx11, kinect2) tmp (dx11.pointcloud)

about

Fill up those black spots in the depth image of the Kinect.
Useful for filtering before using the Kinect depth as a displacement texture.

This is still a bit rough around the edges, and I'll most likely update it in time, but it might be useful to people as it is already. :)

/edit:
Thanks to tmp and their massive dx11.pointcloud contribution, this version should be just plain better, enjoy :)

download

KinectDepthFill_0.2.zip
19.06.15 [02:58 UTC] by Grandchild | 1420 downloads
v0.2 for dx11
Show 1 older revisions

Older Revisions

KinectDepthFill_0.1.zip
17.06.15 [03:52 UTC] by Grandchild | 444 downloads
v0.1 for dx11

mrboni 17/06/2015 - 12:16

This looks great!

But it's crashing my (Nvidia) display driver....

Grandchild 17/06/2015 - 15:58

@mrboni ah that's a pity... I have an AMD card, so I can't really test.
I'm not that experienced with shaders... It might be a bit many texture sample lookups (although 9 per pixel doesn't seem that overwhelming). And for loops and ifs don't seem that cool in shaders either. If anyone has pointers, I'd be glad to improve on the shader :)

tmp 18/06/2015 - 11:52

Unfortunately crashes on my Nvidia GTX 680 too.

In my dx11.pointcloud pack there are btw implementations of depth repair and depth smoothing algorithms ( see http://www.codeproject.com/Articles/317974/KinectDepthSmoothing ).

The nodes are called DepthRepair and DepthSmooth.

mrboni 18/06/2015 - 11:56

Thanks Grandchild. I'll have a look and see if I can figure it out

Tmp - I've tried yours and they work fairly well. I'm not sure they can fill in all the holes as it appears this one does though right?

Grandchild 18/06/2015 - 13:19

@tmp: yes i checked out your kinect pointcloud contribution after i put this up. amazing stuff, and it will definitely help me with a lot of things in the coming days. but yes your depth repair and smooth nodes work, but not quite in the same way. this effect here keeps old sample points around as long as needed, in order to try as hard as possible to fill the whole texture with non-zeros.

after this you would still need a smoothing filter like yours... i'll see what i can come up with. i'll probably modify your approach (i.e. nodes) to have an effect more like mine.

mrboni 18/06/2015 - 18:41

On this topic, do either of you think your techniques could work with the world position texture from the kinect?

antokhio 18/06/2015 - 19:17

suspect a framedelay... not crashing but also not working on 850m
code looks ok, neet to try the texture array framedelay instead

tmp 18/06/2015 - 20:50

@Grandchild:
Sounds like a very interesting approach and I am looking forward to a working version of your contribution!

Grandchild 19/06/2015 - 03:24

re world-position:

that would be truly cool. and i see two ways. one, recreate the world from depth with the help of RayTable, and two, create a mesh from the "raw" depth and fill in the holes there, with a shader, and then creating the world from that (writing that shader is certainly over my head atm).

the first method should be doable, actually i'll try it quickly...

... okay too tired. next week.

anonymous user login

Shoutbox

~14d ago

~18d ago

joreg: The Winter Season of vvvv workshops is now over but all recordings are still available for purchase: https://thenodeinstitute.org/ws23-vvvv-intermediates/

~24d ago

schlonzo: Love the new drag and drop functionality for links in latest previews!

~1mth ago

joreg: Workshop on 29 02: Create Sequencers and Precise Clock Based Tools. Signup here: https://thenodeinstitute.org/courses/ws23-vvvv-08-create-sequencers-and-precise-clock-based-tools-in-vvvv-gamma/

~1mth ago

joreg: Workshop on 22 02: Unlocking Shader Artistry: A Journey through ‘The Book of Shaders’ with FUSE. Signup here: https://thenodeinstitute.org/courses/ws23-vvvv-12-book-of-shaders/

~2mth ago

joreg: Talk and Workshop on February 15 & 16 in Frankfurt: https://visualprogramming.net/blog/vvvv-at-node-code-frankfurt/

~2mth ago

woei: @Joanie_AntiVJ: think so, looks doable