All this wasn't something that you could take for granted. Up to now.
I had to tease you first, before going into detail. If you think about the statements above, or even don't think about it, all of the above should be just normal, no-brainers. Having a not-quadratic screen is the case 99% of the time. These cases occur that often, we should make them easier to work with.
So from now on we have
The main output of the mouse is the Position (Projection) XY pin, values in the case above go from (-1.78, -1) to (+1.78, +1), reflecting that the renderer is not quadratic.
The underlying technology (DirectX) comes with the following spaces and transformations, to get from one space to the other:
World T. View T. Proj. T. Object Space -> World Space -> View Space -> Proj. Space
World Transformation typically is set by the Transform pin at your "quad", that takes it from object space and places that object within the world (the 3d scene).
View Transformation is what you connect to the renderer and is about the position and orientation of your camera.
Projection Transformation is the other input on your renderer, that is for making your scene compatible to a 2d screen. It pojects that 3d stuff onto a screen.
Now, while the underlying DirectX also mixes aspect ratio into that transformation, vvvv at some point started to distinguish lense projection and aspect ratio transformation, which now feels to pay out in the end.
So here is our notion of spaces and transforms:
World T. View T. Proj. T. Aspect Ratio T. Object Space -> World Space -> View Space -> Proj. Space -> Norm. Proj. Space
Our renderer comes with this additional pin Aspect Ratio (and now also comes with that auto aspect ratio feature), treating this transformation a seperate step. Since the transformations are seperated, we got an additional space that you can think in.
And this is the space you want to be in. This at least is our theory. In our projection space the aspect ratio transformation didn't get applied yet.
Let's look at some gif before we theorize further:
Here we see how to operate in projection space when a camera is attached.
With the node WithinProjection (Transform) we tell the system that we want to operate in projection space, which is the same as saying "do not care about the camera (don't apply view and projection transformation as we already are in the right space)". So the spheres get affected by the camera, the quad does not get affected by the camera.
So what you take from the lesson should be that mouse pin Position (Projection) XY goes well together with the WithinProjection (Transform) node. The node you only need if a camera is connected to the renderer.
Now, the next step the pipeline does is applying aspect ratio, which distort everything in a way that a quadratic space matches the rectangular window or viewport. This is just technical necessity as DirectX asks for that. We are now in normalized projection space. You know, that space where left & bottom border are at -1, and right & top border are at +1. The one that you learned in your first tutorial about.
We always thought that this is the nicest space to think in, which is obviously not true. It feels nicely quadratic in size, which just doesn't align to the fact, that your renderer typically is not. So it is a distorted space.
Here is how we still give it a raison d'être:
If you have several render passes you often just want to have a fullscreen quad textured by a previous render pass. Now how would you place a quad so that it goes from left to right border and bottom to top border. Well this is obviously easy to do in a space where these borders are always at a fixed position like in the normalized projection space.
What if you want to use and render the mouse in an early render pass, maybe with many viewports, softedge and aspect ratio settings, while actually hovering with the mouse over the final renderer, that comes with different settings? Does this align?
Well, this is a rare case where you again need to use manual aspect ratio nodes. With them you can adjust how to map to meaningful mouse positions that make sense in an earlier render pass. Actually you just need to reason about the aspect ratio of your orginal scene to make this work nicely. Note however, that in this special case - especially when softedge is involved - system cursor position and rendered cursor position don't align anymore, as you were used to in earlier vvvversions. Note that the editors from the editing framework still work, you just need to use the Cursor node to visualize the cursor, since the system cursor is off.
Patches get converted so that they now work with the new mouse positions, those in projection space.
By that all patches fit well together. We are pretty sure that the benefits outweight the cons. This however still is a breaking change. If you have a patch where you don't use the mouse position for positioning elements, but map it to something else, and experience that the new value range doesn't feel right, you need to manually switch to the old behavior. Check the mouse node to access the now hidden Position (Normalized Window) XY, to access the exact old behavior. Gesture and Touch nodes come with the same pins.
Old renderers get converted in a way that the Auto Aspect Ratio is turned off - on newly created renderers it's turned on.
Patches working with touch or gesture were complicated as they just had to correct the touch position by manually transforming it in compliance to the aspect ratio. Where with mouse you got away with showing a rendered cursor that is just displaced, touch and gesture just don't let you do the same trick. You really expect the elements under your fingers to react. Those patches get converted in a way that they still work by using the Position (Normalized Window) XY, but you should consider cleaning that up by using the standard output Position (Projection) XY and throwing away all the unnecessary aspect ratio related tweaks and hacks.
Directx 11 doesn't come with the features for now. There would of course be a way to do the same with DX11, but let's see first, if the new system prooves to be easier to use for the majority of the tasks, while not failing at the more complex setups. When we have that proof of concept, it'll be doable to copy the concepts over to DX11. Let's wait for that first.
Depending on whether new DX11 builds shall still support older vvvversions or not, the implementation gets trickier or easier. So give us some time here to figure out what route to take. Thank you!
anonymous user login
@gregsn, this is very much appreciated and totally makes sense. this is one of the terrible things a vvvv noobs has to cope with. especially designer don't understand why the stage stretches with the window size.
in terms of the dx9 implementation, i'm not able to really thoroughly test it since we work with dx11 in projects and even at university.
i'm looking forward to the dx11 implementation, although the problem is not as visible in dx11 as in dx9 since vux did this aspect ratio layer node which solves 50% of the problem. the mouse/touch coordinates still need handling there.
seriously all this effort for a somewhat breaking change because people were lazy to do an ApplyTransform? :P
I doubt DX11 will follow though. Or at least I hope it won't make Aspect corrected projection space its default.
@u7angel: thank you sir!
@microdee: well, it's a hardnock life for us. but.. yeah: seriously all this effort, just to get things easy that should be easy. in my opinion positioning elements in spaces (world, view, projection) that are totally incompatible to input device spaces (pixels space or "normalized window space"), ignoring viewports, crops and aspect ratio transforms is just totally random, doesn't align and has been a major design flaw. You want to be able to compare positions (incoming from devices or outgoing for drawing).
I at least would of course like to see DX11 to follow. I am in contact with vux, who's been a bit sceptical, because of reasons, but let's just give it a shot in dx9 first and pick up the discussion later on.
@gregsn: as a designer i got used to the "design flaw" !
i think it is easy to understand, -1 is on the left border of your render window and 1 always the right border.
my Kids get it and some design students should get it too.
@gregsn: well now I need to use another extra node somewhere else if I want the original behavior back. but on the other hand why should I care? I haven't touched DX9 since 2013. When it's asked around on facebook I see most if not all serious vvvv users are in similar situation. Why spend time on the DX9 renderer? Leave it there as legacy. Make DX11 vanilla! Even if there's bidirectional disagreements about licensing and implementation and other stuff between vux and the devvvvs it's been 5 years since DX11 got released. Enough time imo to get onto common grounds. Unless you want to support hardware made before 2002 DX9 is pointlessly limiting and old now. It hurts vvvv's image so badly too. Whenever I want to introduce it to total stranger there's always a conversation like
I know VL is more important right now and that is good. Do that. But also include the DX11 in packs folder by default. I'm sick of this situation :P
Facebook is just another channel for keeping in touch with other vvvv guys but there's a lot of unimportant noise which would just litter this site or the riot channel. If something important arises it will end up on this site too most of the time.
I know this post is not about the technology but the interaction, the rant about the DX9 and my frustrations with it was a sidetrack.
Good to hear that efforts have been already started to make DX11 vanilla.
ehm yeah, true.
thank you for the clarification!
for reference: IFF DX11 will go for it - here is the issue for it: https://github.com/mrvux/dx11-vvvv/issues/332
@gregsn yeah this is great! Exactly the behaviour I'd want, and expect. Exactly according to feedback I get from people starting vvvv as well.
A very welcome change, even though as others have said, we aren't using DX9 anywhere any longer.
@hrovac: Of course its easy to understand, but it gets so much more complicated when you want to build a somewhat responsive GUI and have to deal with lots of elements. All of a sudden you need to make everything scale and have to worry about pixel-perfect mouse input at any renderer size or shape. That's where things get hard and any help to make it more predictable and logical (ie. no distortions, because who would want anything to distort beyond what was designed) will be helpful.