This site relies heavily on Javascript. You should enable it if you want the full experience. Learn more.

Revvvvamped support for Depth Cameras

Still life with cactus

In a quest to get more basic things working out of the box with VL (ie. using vvvv beta>=40 or the all-new vvvv gamma), we took on to support your favorite depth cameras. Most of the cameras and their APIs share basically the same features as a baseline and then some of them have a few extra features. This means that using them in vvvv works mostly the same for all of them.

You have the main device node that you connect ColorImage, DepthImage, PointCloud, Skeleton,... nodes to, to get the desired info out of them. See the help patches coming with the packs for details.

Here is a list and comparison of all available depth cameras with links to the respective packs on nuget.org. To learn how to use nuget packs with vvvv please watch HowTo use Nugets.

Kinect

The original Microsoft Kinect or the XBOX 360 that was released a bit later.
Get the VL pack on nuget.org.
Created with support by chaupow.

Pros

everyone has one at home
good enough for quick prototyping
can connect multiple to one PC

Cons

no longer produced
complicated cabling
skeleton tracking requires T-Pose to initialize

Kinect v2

The second version of the Microsoft Kinect.
Get the VL pack on nuget.org.
Created with support by ravazquez.

Pros

everyone has one at home
quite robust and performant skeleton tracking

Cons

no longer produced
complicated cabling
can only connect one per PC

Azure Kinect

The third version. AzureKinect.
Get the VL pack on nuget.org.
Get the VL pack for skeleton tracking on nuget.org.

Pros

easy cabling
does not require extra drivers to be installed

Cons

to this point rather slow skeleton tracking as compared to Kinect v2

Orbbec Astra

Orbbec Astra.
Get the VL pack on nuget.org.

Pros

easy cabling
small form factor

Cons

requires extra license for skeleton tracking

Intel RealSense

Intel RealSense.
Get the VL pack on nuget.org.

Pros

easy cabling
smallest form factor
highly configurable
built-in depth image filters

Cons

Skeleton tracking for now only in connection with Nuitrack (see below)

Nuitrack API

Nuitrack is a piece software that works with all of the above cameras and provides skeleton, hand and face tracking.
Get the VL pack on nuget.org.
Created with support by ravazquez.

Pros

a unified API for all of the above cameras

Cons

requires extra license for skeleton tracking
can only handle one camera at a time
weird setup and configuration

Leap Motion Controller

The Leap Motion Controller device provides hand and finger tracking.
Get the VL pack on nuget.org.

Pros

for precise and close-up hand and finger tracking

Cons

Please help us improve this list of pros and cons. Know any other or disagree with some mentioned, please add them in the comments! This could eventually grow into a page of the gray book.

joreg, Monday, Jul 27th 2020 Digg | Tweet | Delicious 3 comments

knoeterich 09/11/2020 - 20:21

Thanks a lot for implementing those cameras!!!
For my course of studies I am currently exploring the use of depthcameras in a creative sense. I also got access to a zed. I found that the zed has far more reach then orbec ( like up to 20m!!!) which would be a great plus in terms of beeing able to cover greater distances. Is there a possibility that the zed can be implemented in here too?

joreg 09/11/2020 - 22:37

reminds me, there is actually: https://github.com/m-box-de/VL.ZEDCamera
@rayment can you tell us about the status of this?

Rayment 11/11/2020 - 16:08

Hi! With this one you can do the basics only - retrieve an image and/or depth in any provided format. It returns OpenCV image.
Unfortunately I haven't test with latest gamma versions. You can give it a try - if something is not working, please open an issue.
I'm also planning to update it to Stride and latest SDK with full functionality since we are getting a ZED 2, but it will take some time.

Revvvvamped support for Depth Cameras

Kinect

Kinect v2

Azure Kinect

Orbbec Astra

Intel RealSense

Nuitrack API

Leap Motion Controller

Shoutbox