» Revvvvamped support for Depth Cameras
This site relies heavily on Javascript. You should enable it if you want the full experience. Learn more.

Revvvvamped support for Depth Cameras

Still life with cactus

In a quest to get more basic things working out of the box with VL (ie. using vvvv beta>=40 or the all-new vvvv gamma), we took on to support your favorite depth cameras. Most of the cameras and their APIs share basically the same features as a baseline and then some of them have a few extra features. This means that using them in vvvv works mostly the same for all of them.

You have the main device node that you connect ColorImage, DepthImage, PointCloud, Skeleton,... nodes to, to get the desired info out of them. See the help patches coming with the packs for details.

Here is a list and comparison of all available depth cameras with links to the respective packs on nuget.org. To learn how to use nuget packs with vvvv please watch HowTo use Nugets.

Kinect

The original Microsoft Kinect or the XBOX 360 that was released a bit later.
Get the VL pack on nuget.org.
Created with support by chaupow.

Pros

  • everyone has one at home
  • good enough for quick prototyping
  • can connect multiple to one PC
Cons

  • no longer produced
  • complicated cabling
  • skeleton tracking requires T-Pose to initialize

Kinect v2

The second version of the Microsoft Kinect.
Get the VL pack on nuget.org.
Created with support by ravazquez.

Pros

  • everyone has one at home
  • quite robust and performant skeleton tracking
Cons

  • no longer produced
  • complicated cabling
  • can only connect one per PC

Azure Kinect

The third version. AzureKinect.
Get the VL pack on nuget.org.
Get the VL pack for skeleton tracking on nuget.org.

Pros

  • easy cabling
  • does not require extra drivers to be installed
Cons

  • to this point rather slow skeleton tracking as compared to Kinect v2

Orbbec Astra

Orbbec Astra.
Get the VL pack on nuget.org.

Pros

  • easy cabling
  • small form factor
Cons

  • requires extra license for skeleton tracking

Intel RealSense

Intel RealSense.
Get the VL pack on nuget.org.

Pros

  • easy cabling
  • smallest form factor
  • highly configurable
  • built-in depth image filters
Cons

  • Skeleton tracking for now only in connection with Nuitrack (see below)

Nuitrack API

Nuitrack is a piece software that works with all of the above cameras and provides skeleton, hand and face tracking.
Get the VL pack on nuget.org.
Created with support by ravazquez.

Pros

  • a unified API for all of the above cameras
Cons

  • requires extra license for skeleton tracking
  • can only handle one camera at a time
  • weird setup and configuration

Leap Motion Controller

The Leap Motion Controller device provides hand and finger tracking.
Get the VL pack on nuget.org.

Pros

  • for precise and close-up hand and finger tracking
Cons


Please help us improve this list of pros and cons. Know any other or disagree with some mentioned, please add them in the comments! This could eventually grow into a page of the gray book.

joreg, Monday, Jul 27th 2020 Digg | Tweet | Delicious 3 comments  
knoeterich 09/11/2020 - 20:21

Thanks a lot for implementing those cameras!!!
For my course of studies I am currently exploring the use of depthcameras in a creative sense. I also got access to a zed. I found that the zed has far more reach then orbec ( like up to 20m!!!) which would be a great plus in terms of beeing able to cover greater distances. Is there a possibility that the zed can be implemented in here too?

joreg 09/11/2020 - 22:37

reminds me, there is actually: https://github.com/m-box-de/VL.ZEDCamera
@rayment can you tell us about the status of this?

Rayment 11/11/2020 - 16:08

Hi! With this one you can do the basics only - retrieve an image and/or depth in any provided format. It returns OpenCV image.
Unfortunately I haven't test with latest gamma versions. You can give it a try - if something is not working, please open an issue.
I'm also planning to update it to Stride and latest SDK with full functionality since we are getting a ZED 2, but it will take some time.

  • 1

anonymous user login

Shoutbox

~4d ago

~7d ago

joreg: The Winter Season of vvvv workshops is now over but all recordings are still available for purchase: https://thenodeinstitute.org/ws23-vvvv-intermediates/

~14d ago

schlonzo: Love the new drag and drop functionality for links in latest previews!

~22d ago

joreg: Workshop on 29 02: Create Sequencers and Precise Clock Based Tools. Signup here: https://thenodeinstitute.org/courses/ws23-vvvv-08-create-sequencers-and-precise-clock-based-tools-in-vvvv-gamma/

~29d ago

joreg: Workshop on 22 02: Unlocking Shader Artistry: A Journey through ‘The Book of Shaders’ with FUSE. Signup here: https://thenodeinstitute.org/courses/ws23-vvvv-12-book-of-shaders/

~1mth ago

joreg: Talk and Workshop on February 15 & 16 in Frankfurt: https://visualprogramming.net/blog/vvvv-at-node-code-frankfurt/

~1mth ago

woei: @Joanie_AntiVJ: think so, looks doable

~1mth ago

xd_nitro: Anyone remember who increased projector brightness by removing some components that product the color?

~1mth ago

Joanie_AntiVJ: This looks super interesting (vectors over network) would anyone here know how to implement this in beta? https://github.com/madmappersoftware/Ponk