Spreading-Strategy for non-spreadable Nodes?

Dear vvvvamily,
I’m working on a threefold thesis, using vvvv for the “Knowledge Discovery in Databases” part of gathering data to be visualized and processed musically later on. Precisely speaking, this involves sanch’s JSON Parser nodes (thx for sharing these!); JsonParser and JsonArray, both of them working great but, alas, they’re not spreadable.

Ideally, JsonParser would be fed with a spread of JSON data, generating a spread to be fed to JsonArray. JsonArray itself spits out a spread of keys already (which get cleaned, sorted, concatenated further down the line), thus I’m lacking an idea what the output of a JsonArray (Spreads) would have to look like.

Unfortunately, spread-isation of the JSON nodes is not expectable, I can’t do it anyway, so my first idea was to provide as many JsonParser-JsonArray interpreting modules as needed for completing this task–looking at several hundred of initial keywords to begin with (times three), this seems theoretically feasible but is practically mad and quite error prone. There must be a more elegant, dynamic (spreadable) method to implement.

It’d be great to read your thoughts on this matter.

  • Patrick

guess you stuck… and no there is no way currently as i know… im not in to know what it does so… can only say had same with collada and had to do separate modules… unless you want to make one file with everything and do binsizes

As for your original problem:
I stumpled upon this post right before uploading my JSON-to-XElement node, which is spreadable. See((contribution:json-aselement

Indeed, spot on, herbst! Just had a quite promising go at JSON AsElement, and I’ll move on from there right after some hours sleep.

I REALLY do have to thank you!

  • Patrick

I should really be asleep but, one more question (not directly JSON related, though). GetElements (XML ByXPath) only returns values/slices for matching elements. It’s problematic when the amount of returned slices (and their order, respectively) are not equal to those on the input spread. How do you keep empty spreads in place on the output?

youd better provide small patch for that one

Please find attached demo patch, illustrating the above mentioned issue. Quote “This query returns no value for slice #3, thus slice #4 is shown as slice #3. This is misleading. Slice #3 should be “” (empty, no space, no NIL) and slice #4 should remain in position #4.”
What else remains is to separate the entries in any slice, I see no option for a separator char.

XML slice demo.v4p (12.4 kB)

hey patrick!

using XPath feels a bit like cheating… while cheating is totally ok, still, everytime a small textual domain specific language pops up and sneaks into the visual code, for me it feels like a stranger…

like RegExpr, Expr, Automata, …
It mostly means that you can express the logic of a whole patch within one programmable node and therefore their usage makes sense.

However if expressable with patching i prefer that.
You should be able to express pretty much every search path by manually using GetElements (XML ByName), Element (XML Split), Attribute (XML Split), Select (Node), GetSlice (Node), Prune (XML) as explained here:

xml-nodes-added-to-alpha-builds

Hey Gregsn,

what I came up with, is a result of extensive trialing and erroring :) where I also came around the nodes you mention (but not the blog entry, thx for that). Right now I’m stuck, as seen above, within a situation, where GetElements (XML ByXPath) doesn’t return the same amount of slices as it is being fed with–quite logically because in this test case, one of the input slices misses an XML element (). The output spread has to correspond with other output spreads, generated by other queries, by the same amount of slices as the input spread.
I see no way of dynamically inserting an empty slice in their respective positions. Maybe my approach is wonky anyway, and there’s some better way to do it. If so, please shed light on this :)

The issue, that Element (XML Split) returns slices with concatenated element data is also quite problematic. I can’t come up with a method to seperate them, as the data length is variable.

patrick, GetElements has a Elements BinSize Output which will help you identify which queries didn’t have any result.

But that’s only a statement on the amount, not the/their slice index. Still, thx for the hint!

Well, if you have 5 queries which result in something like:

Wurst, Käse, Brot, Wasser

while one query didn’ match anything, you can insert an empty slice if needed at the query position which as a BinSize of 0.

Or maybe I didn’t get something right …

Using spreads + binsizes is the vvvv way, maybe you should tell us a bit more how you want to handle the data after this step (seems to be not very spread-friendly?).

i was just hoping that throwing out the XPath and doing everything with a patch would somehow help.

So for each location step of your query “resp”, “artist”, “id” you would a small set of nodes that select the childs elements by hand. Those child elements then can be processed further. I was jsut hoping that this could help, since you can access the childs & attributes binsizes … By that it could be that you can somehow reflect upon where your results are located within the original document.

Not sure though…

Doch, ja! In order to insert the missing slice in the right slice index, the index needs to be known. Like: slice index 3 is empty, so slice 4 becomes the new slice 3. We’d need to know to insert a slice in index 3, in order to retain the right order of slices.

War das verständlich?

as an example this patch should concentrate on artists that are part of some groups.

XML elements.v4p (19.4 kB)

so you are searching for the opposite operation? putting defaults when the info is missing?

Danke gregsn, will open your patch when returning to the PC. As to your last question: Yes, putting defaults when the info is missing; the default should be something that will not interpreted as either starting point or endpoint by the Force Directed Graph (as per tonfilm’s contribution).

hi patrick, for the motionbank workshop at node13 i tweaked sanchs JSON Parser a bit. it is now limited* spreadable and you can download it on github until i finish it and update the contributions page.

limits so far:

  • JsonParser got a “go” Input, besause it consumed too much cpu when parsing a big spread
  • SelectToken is not touched at all, because i didn’t need it
  • in JsonArray, the path and key are a Single Inputs (i didn’t need this, too)

Oh bugger, now I see I missed some responses … sorry.

The last task in the vvvv part of my thesis is to display a graph of institutional relations (read institutions as a social construction) using contribution/force-directed-graph. FDGraph takes two spreads, a starting point and an endpoint, which relate to each other. Empty slices would render the two spreads, which are glued together by many other data spreads, empirically useless.
So far I yet have to grasp the concept of binsizes, which still is beyond me, sigh. Are binsizes the key to the solution?

readme: thanks for the hint, this could solve injecting the missing slice. It seems, another approach (see sebl’s contrib) might be the way to go, tho.

Gregsn, your patch clears IDs that have no corresponding Artist Group, abstrakt gesagt, when sliceN in SpreadA has no corresponding slice in SpreadB, sliceN is left out in SpreadA too.

Hey Sebl, I’m currently checking your modded nodes … thx for sharing!