Plugin development guidelines

as you all know one of our main focus for vvvv in the last couple of months was to make it easier to write plugins for it. we introduced the new plugin interface, which is mainly one interface called ISpread and a feature called “dynamic plugins”, which allows one to write and edit a plugin while vvvv is running. the idea is to allow the patcher to switch between visual or textual code as she/he sees fit in order to accomplish the task at hand. we could also say that it is now possible to “patch” a node (visual programming) or “write” a node (textual programming).

since more and more users show interest in “writing” their nodes and therefor we see and will see more and more contributions in this area i think it’s very important to start a discussion about various questions which might arise when “writing” a node. so feel free to ask here.

we’re also thinking about replacing some native nodes with nodes written with the new plugin interface because they’re less error-prone, dealing with two dimensional spreads is trivial and since they basically only rely on one interface (ISpread) they’re more suitable for future developments of vvvv.
so what i’d also like to achieve here is to come up with some kind of guidelines on how to write a node in a way so it has a chance to make it into the core of vvvv.

ok, now enough introduction talk, i’ll simply start with some guidelines i see important:

use generics
use generics when ever possible. this will ensure that your node is future proof (imagine a vvvv with a real generic type system). for now this looks like this:

public abstract class AbstractNode<T> : IPluginEvaluate
{
  [Input("Foo")](Input("Foo"))
  protected ISpread<T> FFooIn;
}

// boilerplate code. ugly, but for now there's no better solution.
[PluginInfo(Name = "MyNode", Category = "Value", ...)](PluginInfo(Name = "MyNode", Category = "Value", ...))
public class MyValueNode : AbstractNode<double>
...

[PluginInfo(Name = "MyNode", Category = "String", ...)](PluginInfo(Name = "MyNode", Category = "String", ...))
public class MyStringNode : AbstractNode<string>
...

put all your logic to the abstract class. hopefully at some point in the future it won’t be necessary anymore to define an abstract class and give concrete implementations of it, but for now it is as it is.

avoid the new operator in the Evaluate method
well this might sound strange, but let me try to explain:
the Evaluate method is called every frame. if there’s a new operator in there, memory will be allocated every frame. the allocation of memory in the managed world is super cheap, so no harm done here, but at some point in the future the memory must be freed, which is not that cheap. this is done by the garbage collector (GC). a garbage collector is a very sophisticated piece of technology and there’re tons of material to be found in the web about it, but i think the most important thing to understand about it in the context of writing a node for vvvv is this:
you don’t want the garbage collection to kick in. if you’re lucky, it’s very fast and you won’t notice it, but you’re probably not lucky and it’s not that fast and so you’ll see glitches; your rendering will stop for a few milliseconds every few seconds and that’s probably the worst thing to happen in a real time application. so to avoid the the necessity for the GC to kick in, simply don’t produce much garbage -> avoid calling new every frame.
if you find yourself in such a situation, think about reusing already allocated memory, for example if you need a dictionary in your evaluation see if you can create it once in your constructor and reuse it. or see next point.

make use ISpread or Spread
i often see code like this:

List<T> tmpList = new List<T>();
// do something with the inputs and the list
// and at the end
output.AssignFrom(tmpList);

why not using the output in the first place? missing the Add operation in ISpread? well i miss it too, and will certainly make sure that it’ll be available in the next release, but an add is simply a output.SliceCount++ followed by a output[output.SliceCount - 1](output.SliceCount - 1) = value. this will be done as an extension method.
all pin implementations of ISpread (except ISpread<ISpread>) inherit from Spread and they’ll only allocate new memory when needed. for example if the SliceCount increases, they’ll internally allocate the double amount of memory necessary to avoid allocating new memory every time the slice count changes.
what i’m trying to say here is to simply use the memory already allocated by the in- and output pins. only introduce temporary storages if really necessary and if so, try to create those storages only once (see point above).

buffering ISpread
sometimes you want to store an ISpread in a buffer to reuse it in the next frame. you’ll be tempted to write someting like:

myBuffer[i](i) = myInput;

as i outlined above, our implementations of ISpread try to reuse memory. so if the SliceCount doesn’t change much, they’ll work internally on the same array. now if you write myBufferi = myInput you’ll only store a reference to the input and in the next frame the array where the reference points to will already be overwritten with new data.
you’ll need to copy the data, so to do it right, write this:

myBuffer[i](i) = myInput.Clone();

i must admint here, that a myInput.Clone() is nothing more than a “new Spread(myInput)” call and therefor is not really the best approach (remember, avoid new in Evaluate). better would be to pre-allocate the buffer and do a

myBuffer[i](i).AssignFrom(myInput);

ok, that’s it for now, feel free to ask or add other tips, like i said, my intention is to collect as much information about writing nodes as possible and write a nice wiki article about it in the future.

is this slicecount-memory behaviour new?
i remember joreg not liking increasing slicecount when needed rather than always setting it to the right size once. that was with plugininterface.v1 however

How do you create an “Enabled” pin, such that when set to off the output pins don’t go to zero?

@woei:
and that’s still true for v1. to avoid unnecessary marshalling costs in v2, the data is copied from unmanaged to managed memory (input) and from managed to unmanaged memory (output) each frame. therefor working with ISpread (like setting the slice count) is very cheap. the expensive v1 call is done only once when the managed data is copied back to vvvv.

talking about copying data back and forth each frame between vvvv and a plugin:
if you deal with large input spreads and you’re only interested in a small subset of it, you can use the Lazy property of InputAttribute. for now it’s only implemented for all value input pins (double, int, etc.). if set to true, data is not copied from unmanaged to managed memory, instead you have to copy it yourself by calling Load(int slice, int length). to call Load you’d have to cast ISpread to Pin or simply do a Import Pin.
Pin also gives you direct access to the internally used data array through the Buffer property.
in case of ISpread<ISpread> it’s enough to simply set the Lazy property to true and the implementation will deal with the Load call itself.
to see the difference regarding performance write a GetSlice for example and increase the size of the input pin to something > 10000 and select < 100 slices.

[Input("Lazy Input", Lazy = true)](Input("Lazy Input", Lazy = true))
protected Pin<T> FLazyInputPin;
...
FLazyInputPin.Load(slice, length);
// FLazyInputPin[slice](slice) to FLazyInputPin[slice + length](slice + length) now contains valid data


// much simpler in case of ISpread<ISpread<T>>
[Input("Lazy Input", Lazy = true)](Input("Lazy Input", Lazy = true))
protected ISpread<ISpread<T>> FLazyIn;
...
var mySpread = FLazyIn[i](i);
// the implementation will make sure to call Load on the internal Pin<T>
// do whatever you want with mySpread, data is valid at this point.

@gilbi:
i’m not exactly sure what you’re asking. but i’ll try with code:

[Input("Enabled")](Input("Enabled")) ISpread<bool> FEnabledIn;
[Output("Output")](Output("Output")) ISpread<double> FOutput;
...
if (FEnabledIn[0](0))
{
  // Write to the output
}
else
{
  // Don't write to the output -> output will stay with the last written values.
}

Thanks that’s what I’m doing now but it seems the node is still consuming a ton of resources even though it’s not actually doing anything.

if you press ctrl+f9 a node not evaluating anything should be at around 0.1. what do you see? how large are the spreads on the input pins?

10,000 when enabled and 3,200 when disabled. The size of the input spread is 6,286,608.

haha, thats way to much… you cant move such data amounts thru vvvv pins with a good performance. you have to think of a clever way to handle all in a custom plugin and output only the data really neccessary. but even 10.000 is a lot…

@gilbi:
well, like tebjan said, 6.000.000 is a little heavy ;) but i assume you are only interested in a small subset of it, so that’s probably why you need the enabled pin and if disabled let the output stay on the old values. i tried it yesterday and you’re right, the node is doing a little too much even if disabled.

i found two issues:

  • a lazy ISpread<ISpread> used internally an array to check which data it had already loaded. clearing that array (well, 6.000.000 entries) took some time of course. even by using the Array.Clear(). so i replaced that array with a Dictionary and the timer dropped down from 6.0 to ~ 0.2 which is fine.
  • output pins wrote their data to vvvv every frame, even if nothing changed. that one ate a lot of performance of course.

i’ll not commit these changes as long as we don’t release beta25.1, because i hadn’t much time to test them yet. but thank you for pointing out these issues, so if it doesn’t break anything else you should be able to work with spreads of size 6.000.000 in future releases of vvvv :)

sounds great!
i vote for a custom username for gilbi: (6 million spreadcount man) :D

Thanks guys. Actually I’m not working with a subset. I’m generating an image one time and then using that image over and over. Right now I have the plug-in in a separate patch which writes the result to a dds file. It would just be nice to re-generate the image on the fly whenever I need to and not worry about the plug-in bogging everything down when I’m not using it…

̶ 6 million spreadcount man

It would be really useful to have a non-generic base interface behind ISpread, for scenarios where you’re dealing with pins whose types are determined dynamically, interfacing with data-binding systems and markup representations (like XAML) that don’t support generics easily.
It could be something like this:

public interface ISpread : IEnumerable
{
 int SliceCount{get;set;}
 object this[int index](int index){get;set;}
}

public interface IDiffSpread : ISpread
{
 SpreadChangedEventHandler Changed; // a non-generic version of SpreadChangedEventHandler<T>
 bool IsChanged{get;}
}

and then have ISpread derive from ISpread, and have IDiffSpread derive from IDiffSpread.

There could be some minor performance costs related to boxing/unboxing of value types, but it would also improve the ability to interact with them in various other contexts.
When writing plugins that deal with dynamically created pins, i often end up writing my own non-generic wrappers around the interfaces (with an abstract base class, with a generic implementation and a static method that picks the implementation based on the requested type). It would be nice to not have to do that.

@thezer0ist:
yes, that was an easy one. just added and committed it. thx.

@elias:
wow. that was really quick. thanks.