» VL.OpenCV Release Candidate
This site relies heavily on Javascript. You should enable it if you want the full experience. Learn more.

VL.OpenCV Release Candidate

General Status

As you know, efforts have been going for the last year and a half into bringing a computer vision nodeset to VL.

The goal was to incorporate as many of the features covered by the world renowned ImagePack; contributed by Elliot Woods some years ago; while bringing in as many new features and improvements as we could.

In the winter of 2018, after setting a roadmap and having patched a decent initial nodeset, we happily announced the pre-release version of VL.OpenCV.

Since then, listening to your needs and constant feedback, we have tried to polish every corner, fix every bug, document every odd scenario, add plenty of demos and specially we tried to give you a clean, consistent and easy to use nodeset.

At this point in time, we are happy to announce that the goal has been nearly met. Most of the features available in the ImagePack made it into VL.OpenCV with the exception of Structured Light, Feature Detection, StereoCalibrate and some of the Contour detection functionality. At the same time, newer features such as YOLO, Aruco marker detection and others have been brought in for you to play with.

So what's next? Even better documentation and loads of examples!

In the mean time, here is a summary of the new things that have been brought into the package in the last couple of months:

CvImage

The new CvImage wrapper around OpenCV's Mat type allows for some optimizations, specially when dealing with non-changing images.

CvImage allows nodes to run only once when the image has changed, significantly reducing CPU usage

Cache Region lovvvves CvImage!

Since it is now possible to detect if an image has changed, CvImage is a perfect candidate to benefit from Cache regions.

Cache Region

Cache regions can now make proper usage of image inputs and outputs

Keep in mind that Cache regions have already been added to most VL.OpenCV nodes. When in doubt just open the node and have a look inside.

Renderer

The Renderer was re-built from the ground up to improve usability and to fix bugs and issues. Title, Render Mode and Show Info features were added. Renderer also remembers its bounds on document reload.

Renderer

New Renderer implementation introduces Title, Renderer Mode and Show Info pins

Histograms

Histogram analysis has been added to VL.OpenCV. A useful tool in many scenarios.

Hitograms

Histograms allow you to analyze pixel value tendencies per channel

Homography

Homography and reverse homography are now available in VL.OpenCV.

Homography

Homography (vvvv used only for point IOBox)

Stereo Matchers

Two new Stereo Matchers were added, these allow you to create a depth map from a set of stereo images. For more see the StereoDepth demo patch in VL.OpenCV.

Stereo Matcher

Depth map obtained from a pair of stereo images

Serialization

Serialization support was added for CvImage and Mat types, allowing you to use CvImage as part of more complex data structures which get serialized with no effort. This can be a heavy operation so make sure to trigger it when needed only.

For a hands on demonstration check out the Serialization VL demo that ships with VL.OpenCV.

Breaking Changes

As part of this final effort to clean everything even further and make the nodeset consistent and properly organized, we needed to rename and move a few things around which as you can imagine means the introduction of breaking changes. We understand this is an annoying thing to cope with, but this was basically the reason why we chose to keep this pack in a pre-release state until we felt confident with its structure and approach.

In summary yes, you will get red nodes when you upgrade your VL.OpenCV projects to the latest version, but in most cases it should be as easy as to double-click and find the node in its new category.

An exception to this are the nodes that got renamed, which we list below:

  • Reader (Intrinsics) -> IntrinsicsReader
  • Reader (View Projection) -> ViewProjectionReader
  • Writer (Intrinsics) -> IntrinsicsWriter
  • Writer (View Projection) -> ViewProjectionWriter
  • GetByteArray -> ToByteArray
  • GetValues -> ToValues
  • GetValues (Custom) -> ToValues (Custom)
  • FiducialTracker -> FiducialDetector
  • Update (FaceRecognizer) -> FaceRecognizerUpdate
  • VideoFile (Append) -> VideoWriter (Append)
  • VideoFile - VideoPlayer
  • Merge -> Join

Summarized Changelog

General

  • Introduced a new wrapper for Mat called CvImage
  • Cache region additions to improve performance on non-changing Images
  • ImageReader re-implementation
  • Renderer re-implementation
  • VideoIn re-implementation
  • Reintroduced "Supported Formats" output pin on VideoIn node
  • Upgraded to VL.Core 0.95.37
  • DirectX/OpenCV transformation documentation and cleanup
  • Added Blocking, Reactive and HoldLatestCopy versions of VideoIn node
  • Added Blocking and Reactive versions of ImageReader node
  • Reimplemented Extrinsics as a data type
  • Stabilized VideoFile Source node and moved from Experimental into Sources
  • Added Serialization support for Mat type
  • Added Serialization support for CvImage type

New in VL

Nodes

  • StereoBMMatcher
  • StereoSGBMMatcher
  • ToValues (Custom)
  • ToValues
  • GetChannel
  • Histogram
  • DrawHistogram
  • ToByteArray
  • Transform (Perspective)
Demos

  • DrawNodes
  • Homography
  • Histogram
  • Serialization
  • StereoSGBMMatcher

New in vvvv

Nodes

  • IntrinsicsToProjectionMatrix
  • ExtrinsicsToViewMatrix
Demos

  • Calibrate and Reproject
  • SolvePnP
  • StereoDepth

Remember that in order to use VL.OpenCV you first need to manually install it as explained here. Also, until we move away from pre-release you need to use the latest alpha builds.

We hope you find good use for this library in your computer vision projects and as always, please test and report.

ravazquez, Sunday, Mar 17th 2019 Digg | Tweet | Delicious 3 comments  
isdzaurov 19/03/2019 - 09:23

Hello guys!
Can Stereo Matchers work with real time video and create depth map?

m9dfukc 19/03/2019 - 14:58

Looking forward to try this out!

ImagePack was nice, unfortunately also quite unstable sometimes. Really curious about the performance and stability characteristics of this little gem :)

cheers
m9d

p.s.: Finally a really good reason to jump onto the VL bandwagon!!!

ravazquez 19/03/2019 - 16:10

@isdzaurov, yes, it is possible to do just that, we tested it and it worked. Be advised that this is an expensive operation so you might want to optimize performance. A couple of simple ways to do this would be to downscale the image before processing it and to make use of the AsyncLoop region to run this on its own thread.

Also, there are quite a lot of parameters to tweak and it can take some time before you get proper results based on your data. Let me know if you need any help.

@m9dfukc glad to hear you will be moving over to VL, you will not look back I am sure. As for stability and performance, stability seems to be quite good from our tests, as for performance, the underlying wrapper we are using (OpenCVSharp) does not have GPU support, so you do need to keep that in mind, but so far for us and other users trying the library out it has been good. If you are handling large images on high framerates you will very likely need to get creative.

  • 1

anonymous user login

Shoutbox

~1h ago

bo27: @sinus hi! Thanks for using the template. Run is for endusers and it kills explorer. Try developer.bat to access patches

~5h ago

sinus: @bo24: sorry for da hassle. everything´s fine again. several restarts did the job.

~5h ago

sinus: @bo24: my mail db@servus.at

~5h ago

sinus: @IvanRastr aka bo24: found your project template, tested run bat.file, now my screen is black. only cursor. what to do?

~11h ago

~5d ago

cznickesz: @tonfilm thanks for the info. Can´t wait to give it a try!

~6d ago

tonfilm: @cznickesz #vvvv #vl #xenko integration is not ready for the public yet, it's still under heavy development... more updates soon.

~6d ago

cznickesz: Possibly dumb question: is there any release containing xenko-integration right now?

~7d ago