» Aspect Ratio and Projection Space
This site relies heavily on Javascript. You should enable it if you want the full experience. Learn more.

Aspect Ratio and Projection Space

Some brief statements


A quad is quadratic

This quad has a quadratic shape.


Positioning an element at the mouse position results in that element being shown at the mouse position

This small quad is aligned to the mouse.


Using touch positions for positioning results in elements drawn at your finger tips

These quads show up where i touched the screen.


Interact with objects in world space, even in complex multi screen setups. Do that with the system cursor, not a displaced rendered cursor

Star

All this wasn't something that you could take for granted. Up to now.
I had to tease you first, before going into detail. If you think about the statements above, or even don't think about it, all of the above should be just normal, no-brainers. Having a not-quadratic screen is the case 99% of the time. These cases occur that often, we should make them easier to work with.

So from now on we have

  • Auto Aspect Ratio in the renderer, so you don't need to do that AspectRatio (Transform) involving cylic graph with the 3 links
  • you can disable Auto Aspect Ratio and still feed your own for the more complex cases
  • mouse, touch, gesture nodes are now reporting positions in our notion of projection space, an undistorted space that didn't get treated by the aspect-ratio transformation. These postions are just easy to work with as you saw above.
Projection Space vs. Distorted Normalized Projection Space

The main output of the mouse is the Position (Projection) XY pin, values in the case above go from (-1.78, -1) to (+1.78, +1), reflecting that the renderer is not quadratic.


All the details

What's that Projection Space?

The underlying technology (DirectX) comes with the following spaces and transformations, to get from one space to the other:

              World T.           View T.           Proj. T.          
 Object Space   ->   World Space   ->   View Space   ->   Proj. Space  

World Transformation typically is set by the Transform pin at your "quad", that takes it from object space and places that object within the world (the 3d scene).
View Transformation is what you connect to the renderer and is about the position and orientation of your camera.
Projection Transformation is the other input on your renderer, that is for making your scene compatible to a 2d screen. It pojects that 3d stuff onto a screen.
Now, while the underlying DirectX also mixes aspect ratio into that transformation, vvvv at some point started to distinguish lense projection and aspect ratio transformation, which now feels to pay out in the end.
So here is our notion of spaces and transforms:

              World T.           View T.           Proj. T.          Aspect Ratio T.
 Object Space   ->   World Space   ->   View Space   ->   Proj. Space   ->   Norm. Proj. Space 

Our renderer comes with this additional pin Aspect Ratio (and now also comes with that auto aspect ratio feature), treating this transformation a seperate step. Since the transformations are seperated, we got an additional space that you can think in.
And this is the space you want to be in. This at least is our theory. In our projection space the aspect ratio transformation didn't get applied yet.

Let's look at some gif before we theorize further:

Operating in projection space

Here we see how to operate in projection space when a camera is attached.
With the node WithinProjection (Transform) we tell the system that we want to operate in projection space, which is the same as saying "do not care about the camera (don't apply view and projection transformation as we already are in the right space)". So the spheres get affected by the camera, the quad does not get affected by the camera.
So what you take from the lesson should be that mouse pin Position (Projection) XY goes well together with the WithinProjection (Transform) node. The node you only need if a camera is connected to the renderer.

Normalized Projection Space

Now, the next step the pipeline does is applying aspect ratio, which distort everything in a way that a quadratic space matches the rectangular window or viewport. This is just technical necessity as DirectX asks for that. We are now in normalized projection space. You know, that space where left & bottom border are at -1, and right & top border are at +1. The one that you learned in your first tutorial about.
We always thought that this is the nicest space to think in, which is obviously not true. It feels nicely quadratic in size, which just doesn't align to the fact, that your renderer typically is not. So it is a distorted space.

Several render passes

Here is how we still give it a raison d'être:
If you have several render passes you often just want to have a fullscreen quad textured by a previous render pass. Now how would you place a quad so that it goes from left to right border and bottom to top border. Well this is obviously easy to do in a space where these borders are always at a fixed position like in the normalized projection space.

not so quadratic

What if you want to use and render the mouse in an early render pass, maybe with many viewports, softedge and aspect ratio settings, while actually hovering with the mouse over the final renderer, that comes with different settings? Does this align?
Well, this is a rare case where you again need to use manual aspect ratio nodes. With them you can adjust how to map to meaningful mouse positions that make sense in an earlier render pass. Actually you just need to reason about the aspect ratio of your orginal scene to make this work nicely. Note however, that in this special case - especially when softedge is involved - system cursor position and rendered cursor position don't align anymore, as you were used to in earlier vvvversions. Note that the editors from the editing framework still work, you just need to use the Cursor node to visualize the cursor, since the system cursor is off.

Cursor gets rendered in another renderer that you hover. Softedge adds to the complexity.

Old patches and a breaking change

Patches get converted so that they now work with the new mouse positions, those in projection space.
By that all patches fit well together. We are pretty sure that the benefits outweight the cons. This however still is a breaking change. If you have a patch where you don't use the mouse position for positioning elements, but map it to something else, and experience that the new value range doesn't feel right, you need to manually switch to the old behavior. Check the mouse node to access the now hidden Position (Normalized Window) XY, to access the exact old behavior. Gesture and Touch nodes come with the same pins.
Old renderers get converted in a way that the Auto Aspect Ratio is turned off - on newly created renderers it's turned on.
Patches working with touch or gesture were complicated as they just had to correct the touch position by manually transforming it in compliance to the aspect ratio. Where with mouse you got away with showing a rendered cursor that is just displaced, touch and gesture just don't let you do the same trick. You really expect the elements under your fingers to react. Those patches get converted in a way that they still work by using the Position (Normalized Window) XY, but you should consider cleaning that up by using the standard output Position (Projection) XY and throwing away all the unnecessary aspect ratio related tweaks and hacks.

DX11

Directx 11 doesn't come with the features for now. There would of course be a way to do the same with DX11, but let's see first, if the new system prooves to be easier to use for the majority of the tasks, while not failing at the more complex setups. When we have that proof of concept, it'll be doable to copy the concepts over to DX11. Let's wait for that first.
Depending on whether new DX11 builds shall still support older vvvversions or not, the implementation gets trickier or easier. So give us some time here to figure out what route to take. Thank you!

The fact that DX11 works a bit different for now isn't a big issue. Most patches that are supposed to work for both node sets actually do work for both environments. The only difference typically is how a rendered cursor comes into view. Interaction in most cases should feel the same though. For DX11 nothing has changed and all patches should work exactly like before.
gregsn, Friday, Jan 26th 2018 Digg | Tweet | Delicious 10 comments  
u7angel 31/01/2018 - 13:08

@gregsn, this is very much appreciated and totally makes sense. this is one of the terrible things a vvvv noobs has to cope with. especially designer don't understand why the stage stretches with the window size.

in terms of the dx9 implementation, i'm not able to really thoroughly test it since we work with dx11 in projects and even at university.

i'm looking forward to the dx11 implementation, although the problem is not as visible in dx11 as in dx9 since vux did this aspect ratio layer node which solves 50% of the problem. the mouse/touch coordinates still need handling there.

microdee 31/01/2018 - 23:32

seriously all this effort for a somewhat breaking change because people were lazy to do an ApplyTransform? :P
I doubt DX11 will follow though. Or at least I hope it won't make Aspect corrected projection space its default.

gregsn 01/02/2018 - 17:10

@u7angel: thank you sir!
@microdee: well, it's a hardnock life for us. but.. yeah: seriously all this effort, just to get things easy that should be easy. in my opinion positioning elements in spaces (world, view, projection) that are totally incompatible to input device spaces (pixels space or "normalized window space"), ignoring viewports, crops and aspect ratio transforms is just totally random, doesn't align and has been a major design flaw. You want to be able to compare positions (incoming from devices or outgoing for drawing).

I at least would of course like to see DX11 to follow. I am in contact with vux, who's been a bit sceptical, because of reasons, but let's just give it a shot in dx9 first and pick up the discussion later on.

hrovac 01/02/2018 - 17:30

@gregsn: as a designer i got used to the "design flaw" !
i think it is easy to understand, -1 is on the left border of your render window and 1 always the right border.
my Kids get it and some design students should get it too.

microdee 01/02/2018 - 18:09

@gregsn: well now I need to use another extra node somewhere else if I want the original behavior back. but on the other hand why should I care? I haven't touched DX9 since 2013. When it's asked around on facebook I see most if not all serious vvvv users are in similar situation. Why spend time on the DX9 renderer? Leave it there as legacy. Make DX11 vanilla! Even if there's bidirectional disagreements about licensing and implementation and other stuff between vux and the devvvvs it's been 5 years since DX11 got released. Enough time imo to get onto common grounds. Unless you want to support hardware made before 2002 DX9 is pointlessly limiting and old now. It hurts vvvv's image so badly too. Whenever I want to introduce it to total stranger there's always a conversation like

  • "oh yeah btw the default graphics backend is DX9 and vvvv won't even start without it"
  • "uurgh, okay so there's no support for modern graphics libraries?"
  • "yeah there is actually but that's a 3rd link you have to download and unzip yourself"
  • "oh god but why?"
  • "nobody knows exactly, something something legacy I guess"

I know VL is more important right now and that is good. Do that. But also include the DX11 in packs folder by default. I'm sick of this situation :P
</rant>

gregsn 01/02/2018 - 18:52
  • "but on the other hand why should I care?" you don't need to actually. We made sure that DX11 patches work as they did. so no breaking change for you.
  • implementing it in DX9 now lets us compare the approaches side by side.
  • when there is consensus about the approach i'll be happy to be of any use on the DX11 implementation. also started that effort already.
  • "Do VL". Good to hear. Yeah we do. But i will take the freedom and still work on vvvv when i feel the urge to.
  • actually it's not about DX9 vs. DX11 at all. It's about a general interaction concept, not about a certain technology.
  • about you guys discussing everything on facebook: sure. what could go wrong.
microdee 01/02/2018 - 19:41

Facebook is just another channel for keeping in touch with other vvvv guys but there's a lot of unimportant noise which would just litter this site or the riot channel. If something important arises it will end up on this site too most of the time.

I know this post is not about the technology but the interaction, the rant about the DX9 and my frustrations with it was a sidetrack.

Good to hear that efforts have been already started to make DX11 vanilla.

gregsn 01/02/2018 - 20:16

ehm yeah, true.
thank you for the clarification!

for reference: IFF DX11 will go for it - here is the issue for it: https://github.com/mrvux/dx11-vvvv/issues/332

dominikKoller 02/02/2018 - 01:41

@gregsn yeah this is great! Exactly the behaviour I'd want, and expect. Exactly according to feedback I get from people starting vvvv as well.

seltzdesign 02/02/2018 - 15:42

A very welcome change, even though as others have said, we aren't using DX9 anywhere any longer.

@hrovac: Of course its easy to understand, but it gets so much more complicated when you want to build a somewhat responsive GUI and have to deal with lots of elements. All of a sudden you need to make everything scale and have to worry about pixel-perfect mouse input at any renderer size or shape. That's where things get hard and any help to make it more predictable and logical (ie. no distortions, because who would want anything to distort beyond what was designed) will be helpful.

  • 1

anonymous user login

Shoutbox

~14d ago

~17d ago

joreg: The Winter Season of vvvv workshops is now over but all recordings are still available for purchase: https://thenodeinstitute.org/ws23-vvvv-intermediates/

~24d ago

schlonzo: Love the new drag and drop functionality for links in latest previews!

~1mth ago

joreg: Workshop on 29 02: Create Sequencers and Precise Clock Based Tools. Signup here: https://thenodeinstitute.org/courses/ws23-vvvv-08-create-sequencers-and-precise-clock-based-tools-in-vvvv-gamma/

~1mth ago

joreg: Workshop on 22 02: Unlocking Shader Artistry: A Journey through ‘The Book of Shaders’ with FUSE. Signup here: https://thenodeinstitute.org/courses/ws23-vvvv-12-book-of-shaders/

~2mth ago

joreg: Talk and Workshop on February 15 & 16 in Frankfurt: https://visualprogramming.net/blog/vvvv-at-node-code-frankfurt/

~2mth ago

woei: @Joanie_AntiVJ: think so, looks doable