» vvvv-ZeroMQ
This site relies heavily on Javascript. You should enable it if you want the full experience. Learn more.

vvvv-ZeroMQ

pack zmq network multithreading async 0mq
Credits: velcrome

about

What started as a easy drop-in replacement for UDP (Network Server) and TCP (Network Server turned into a full fledged toolkit for the 99% of common asynchronous communication styles.

In its most basic use case, you can use two Dealer (Network.ZeroMQ) socket plugins to then Send (Network.ZeroMQ) and Receive (Network.ZeroMQ) bundled spreads of Raw.

Bind one to an endpoint (pretty much like the old Server), and Connect any count of others to the same address (pretty much like the old Client species). From then on, you will get reliable, and ordered output for every Send you bang (pretty much like you expect, but which the naive TCP and UDP could never deliver, for protocol specification reasons). For most patterns this is even full-duplex.

As for a cooler use case, you can set up a Pub-Sub network just as easily. Bind the Publisher (Network.ZeroMQ), but connect all Subscriber (Network.ZeroMQ)s and listen to the topics of interest. ZeroMQ is smart enough to only wire the stuff subscribers are actually interesed in. With just 3 more nodes (as in xpub, xsub, proxy) you can give your clustered vvvv setup similar characteristics as when using MQTT (or mostly any other software with MQ in its name) without using a third-party broker with suspicious patents.

In brief: this toolkit gives you the means to handle advanced networking science with just a few nodes. Be aware, it does not do any security for you, this is something you'd have to stack on top of it.

Installation

Use this vpm link to install it with microdees great vpm tool.

You can also create a vvvv-ZeroMQ folder in your packs folder, and unzip a release or prerelease from the GitHub release page.

Technical details

Most of the technical advance of this pack is directly coming from the NetMQ library, which is an open c# reimplementation of the original c++ libs.

Read the manual!

The vvvv socket nodes provided take full advantage of the simplistic modularity of the original design, but take away alot of functional choices to give you instead an easy yet sufficiently complete node interface: Pick two sockets and transmit binary payloads between them. Each socket has it's own subtleties, which can help you, depending on your requirements and design. And hey, there are only so many patterns you can do ;-)

Supported base protocols are InProc (similar to shared mem, but more like a postal service), TCP (the common default for its reliability across networks even under high load) and PGM (some industry thing for massive pubsub). Sadly IPC is not yet supported.

Each Socket runs in its own thread, independent of the vvvv Mainloop. The similarity of sockets allows easy exchange of patterns during runtime. Have a look into the help patches (one for every socket) and spot the subtile differences, so you can master inter-instance and network communication.

Talking ZeroMQ is supported and understood in more than 40+ languages, including python, java, php, erlang, and c. It allows future-proof IoT for various CPU architectures, and it allows you designing your network according to the means of your hardware, instead of designing around bloated industry-commissioned specifications.
Currently it might be a smart idea to rapidly prototype your network configuration with vvvv before moving on to more low-level devices.

vvvv-Message has been interfaced as well, because it does away with a lot of hassle of converting to and binning correctly any Raw data, because that is what's going to be weird once you exchange actual data within different vvvv instances across cores or even the net.

This pack needed quite some boiler plating to get there, but I am proud all nodes are up to woei gold standard of bin-sizeing now.

License

Because this is just a smart shim between NetMQ and the vvvv interface (both LGPL), this pack is LGPL too: any use is free of charge, changing source code is liberal. That said, if you use the actual VVVV, licensing fees do apply for commercial work. Same (plus attribution) if you use additional vvvv-Message nodes to your advantage.

TL;DR

ZeroMQ it is crazy fast, supposedly sometimes even faster than naive TCP.
Code is at github.

sebl 02/12/2015 - 17:28

this is so great! thanks

everyoneishappy 04/12/2015 - 03:20

Velcrome you are a superhero \o/

btw for anyone interested I found a (quite old) blog post that gives a great executive summary on some of the main patterns:
http://nichol.as/zeromq-an-introduction

eps 07/12/2015 - 02:06

thanks for this
didn't manage to make message interface work though
raw data are coming, but message receiver (connected to dealer) doesn't see anything

any hint what to check?

velcrome 13/12/2015 - 23:40

finally found some time to fine tune this. Please test, help patches are provided :)

idab 28/02/2016 - 18:23

Hey velcrome,
look at this thread.
https://vvvv.org/forum/streaming-texture-over-network#comment-217378
Strage behaviour here. b34.2 x64 Win10 x64.

velcrome 29/02/2016 - 11:59

never forget to set the Bind toggle in the sockets, so each endpoint is bound to exactly once!

yar 02/11/2016 - 23:02

Hi! How it can be used to send/receive messages via UDP (for example OSC)?

velcrome 03/11/2016 - 22:52

it cannot. zeromq at its core can only utilize tcp, ipc and pgm as transport protocol, to make it act reliable (which udp never is, but which you'll only see when your network is under pressure).

for udp based osc, use the standard vvvv issued UDP (Network Client) and have a look into osc.

velcrome 20/06/2017 - 17:19

This pack has been successfully updated to beta35.7

io 16/10/2017 - 14:30

How can I use multiple Receive (clients) from the same Send ?

catweasel 17/10/2017 - 12:58

@io make sure the bind pin isn't set, I think.

anonymous user login

Shoutbox

~17h ago

vnm: Does Intensity Pro 4K work with VideoIn(dx11)?

~5d ago

joreg: siggraph asia 2017 technical papers trailer: https://www.youtube.com/watch?v=3OGKh_9Rj_8

~6d ago

drupal_admin: we're back

~6d ago

drupal_admin: hello. maintenance reboot at 2:06am. save your work! should be back in a few minutes

~9d ago

tekcor: Following the steps of the current vvvv chat, we put up a vvvv.js dev support. https://riot.im/app/#/room/#vvvv.js:matrix.org