pack vive oculus VR htc
The OpenVR pack contains nodes to get pose data and render a 3D scene into any VR headset supported by SteamVR. Currently HTC Vive, Oculus Rift DK1, DK2 and CV1.
Commissioned by Marshmallow Laser Feast:
Special thanks goes to these members of the vvvv community who also supported its development with various amounts:
Louis Mustill & Will Young
Poser (OpenVR) returns all tracking poses from the headset, lighthouse stations, trackers and controllers as transformation. Also has a pin to disable wait for OpenVR sync, but the sync needs to run a few times in startup to get the compositor up and running.
Camera (OpenVR) calculates vvvv conform view and projection matrices for the left/right eye. It also outputs the recommended texture size and a hidden area mesh that masks out unnecessary pixels to safe shader performance.
Compositor (OpenVR) receives a texture handle and submits it to the HMD.
TrackedDevices (OpenVR) outputs all OpenVR events and an object oriented view on the tracked devices. Might be merged with Poser in the future.
Controller (OpenVR Split) splits a tracked controller from the TrackedDevices node into pose transformation, trackpad axis and all button states.
Renderer (OpenVR DX11) Combines Camera and Compositor into one handy node that only needs a layer to be rendered. It exposes quality settings and returns the rendered texture for preview.
HiddenArea (OpenVR DX11.Layer Viewport) constructs the hidden area mesh to cast out unnecessary pixels.
ViveController (OpenVR DX11.Layer) renders a life-like model of the vive controller. It gives a pretty good eye-hand coordination feeling when seeing the actual controller in VR.
ViveLighthouse (OpenVR DX11.Layer) renders a life-like model of the vive lighthouse tracking station.
HMD (OpenVR DX11.Layer) renders a life-like model of a generic VR headset, helps to build a third person overview on the scene.
Latest version has tracker only support, read this blog post on how to set it up: using-htc-vive-trackers-without-headset
Here is a nice set of patches from the OpenVR beginner workshop from NODE17:
Sources are available here:
If you encounter bugs, then github issues instead of comments are welcome:
1@#!@$@#$%^$^& waaaaaaaaaa! let the dogs out!
so good to see this! thanks to tonfilm, and everyone involved
how about a vveekend vvorkshop about this? who should keep it?
Well Done boys!! respect.
amazing, grand tonfilm and supporters
Trying this with an Oculus DK2. Installed SDKRuntime 0.80. OR Configuration utility seems happy but OpenVR girlpower never detects HMD.
Have I missed anything?
Oculus Rift DK2 + Leap = Vive wannabe ;)
@guest you need to calibrate oculus with SteamVR, if that's all fine vvvv will detect it. and i also think its better to use latest oculus runtime, if your graphics card is supported.
everything works fine in SteamVR/vvvv with oculus runtime 0.8 + latest Leapmotion Orion Version: 3.1.3+41910...
btw failed with the newest v1.6.0 oculus runtime (even if gtx 780ti should work)
now of course the challenging part is to make vvvv patches that runs at 75/90 fps ;)
there seems to be a calibration delay with the latest Vive HMD,
looking at a line while moving will show two lines,
then at rest point the image becomes sharp again
seems a specific problem
@ggml, yeah i saw that too
Righteous! Works great on my Sager laptop with 2x GTX870Ms in SLI. I do see the double image on motion in the Vive HMD and not on the laptop display; I think that may be because my cards are below VR spec. @ggml and @u7angel what cards are you using? I'll try it on a desktop GTX970 and see what happens...
Thanks to all, really excellent contribution!
is is possible to add any kind of 2d effect before the hmd ? like on the old dk2 wrap... ?
@andresc4 you can translate a grid on z axis and connect it to hmd pose transform
any chance for a dx11 screenshot node for virtual patching purposes ?
anyone managed to get the vive frontfacingcamera videotexture ?
The haptic pulse node seems a little odd, it only seems to work when the power is around 0.06 above that and you can hear it but not feel it, it's also quite weak, is that as high as it goes? I want a massage :)
Also the node goes red if the controller disconnects.
first of all great thanks to tonfilm and all others for bringing this to vvvv.
i am currently trying to move the controller's origin (or axis center) to another location in the controller, but no luck so far.
i edited the .obj files origin in the steam/steamapps/.../vr_controller_vive_1_5 which is used in the ViveController node, but that obviously only changes the rendered model.
i also looked at the json file in that folder but couldn't figure out if this is the right place.
if anybody could give me directions on this it would help me a lot.
Thanks tonfilm and supporters!
The Poser node is CPU intensive (8000 ticks in debug mode). I can't get a perfect 90fps with the OpenVRdemo..
When WaitForSync=0, Poser is 100 ticks, perfect 90 fps.
But I can see the double lines in motion...
The motion is perfect only with WaitForSync=1.
Any idea for a workaround or other ways to decrease the Poser CPU consemption?
What is the role of WaitForSync?
VVVV 34.2 x64
HTC Vive / Direct Mode / SteamVR
you have to disable the sync of the renderer, every node that blocks the vvvv mainloop has to release its sync. so enable "Do Not Wait" on every DX11 Renderer (immediate on DX9 Renderer) and VideoTextures.
Thanks for your answer, but I can't get good results with the OpenVRdemo at this time (it's an exageration: results are perfect, but after drawing a few lines in space, CPU is overcharged, and framerate too small...)
With this CPU budget, it's hard to develop more complex applications.
If I understand correctly (and for me it's very abstract), OpenVR use a fonction called WaitGetPoses()(the VVVV WaitForSync Poser Pin?).
This fonction seems to wait for the position of every tracked objects in the scene and it's necessary for a perfect fluidity and avoiding any judder effects.
But it has a very high CPU cost.
Seems to be a frequent OpenVR concern on Github.https://github.com/ValveSoftware/openvr/issues/64
An OpenVR user talk about rendering GetWaitPos on a specific thread to gain performance, and not to block the others CPU process.
A thing like that would be possible in VVVV?
yo guest! I've already tried it with multiple vvvv instances, one was getting openvr data in sync, the other was rendering the scene independently. it was worse than being 45 fps synced. problem is that the presentation of the image for the vive need to be at tight exact times otherwise it will be horribly glitchy and flashing in the headset. so the problem with the vvvv implementation is that it's asking for the waiting process at node evaluation and not at right before end of the frame so you can't really have a precise idea when vvvv asks for the next available state of the vr system also if anything else is not calculating afterwards, which can make missing frames. that's the reason why it is capped at 45 fps when the patch runs at 120+ fps without the VR syncing.
+ on this topic: is there a method with vvvv plugininterfaces in which I can make sure my thing will be executed last before presentation. is already one of the Mainloop events can do that?
Thanks for your clear explanation!
So the (scary) question is:
There is no way to use Vive+VVVV without judder and/or low framerate, regardless of your hardware specs?
currently if you want more than phong shaded boxes, it won't be a smooth experience in vvvv now but it can be improved in the future with 2 things: waiting done at right time, and as vux suggested making a proper renderer for the compositor (instead of resource pointer readback as done currently which also consumes a lot of overhead)
hello guest and @microdee,
as you can read on the openvr github and steam threads about WaitGetPoses, there is no way around it. but calling it as soon or as late as possible in the frame is the next thing that will be added to the pack.
also microdee has developed a compositor node that has a DX11 texture input to avoid the resource pointer read back... i will try to merge that soon.
@tonfilm: my compositor node is still using pointer readback it is only improving usability. vux pointed out that the fastest would be a clone of the preview node just without a forms window. when I was doing that node I didn't have much time to experiment with it unfortunately and currently I'm busy with fbx and commercial work.
@microdee ok, do you know in theory how to avoid resource pointer read back, or should i contact @vux for more details?
mainloop events is in the making...
yup contact vux. also try the syncing on different thread with WaitHandles for vvvv ;)
@tonfilm maybe you can do the readback similar to here (https://github.com/elliotwoods/VVVV.Nodes.ReadBack) not sure this is the ideal way for reading back only 1 pointer, but at least it's threaded.
@sebl that's something different... the texture data has not to go back to the CPU in that case, it's only about sending the pointer to OpenVR.
@microdee and others, as of V3 the WaitGetPoses is now by default after the vvvv frame, so shortly before the next frame, which is recommended by valve. if Sync After Frame is off, the call is right before the vvvv frame. you can check what works better for your setup...
@catweasel please try haptic pulse again, i think i fixed a little bug and the node is now spreadable. it is common to trigger the pulse in multiple frames (MonoFlop or so), the power pin is the duration for one single pulse from 1 to 3999 microseconds, so max 4ms.
with SteamVr autoupdate everything goes wrong!
my DK2 rig was working perfectly with vvvv.openvr V1 with early september steamVR version.
with the new update i try every combination between SteamVR, SteamVR beta, vvvv.openvr V1/V2/V3 with no luck, no more vvvv Rendering in the headset
i've this compositor error message in vvvv:
and SteamVr says:
the vr_client_Steam log file says:
sounds like a SteamVr issue more than a vvvv one!
@guest: use R8G8B8A8_UNorm (you can try sRGB variant too) texture format. only dx11
the new V3 also has a color space enumeration. not all RGB formats run with all color spaces. before V3 the default format was 16-bit float, but i changed that because of performance reasons to an 8-bit format. seems the new steamVR update does not like that particular one...
I'm new to vvvv,so maybe this question is very obvious, for that I am sorry.
I am trying to get my Rift DK2 to work, It works with the SteamVR calibration etc. - so far so good.
I am trying to use the example patches in the
However, when I open the demo patch, I have several 'red' (unloaded) objects.
I read the Using Adddons page, and I have gotten the impression that placing the openVR folder inside packs should be suffienct for letting vvvv see the content in all subdirectories.
Is there a setting which let me direct vvvv to the sub directories of the OpenVR 'nodes' folder?
thanks a lot! :)
@lassemunk did you also install the DX11 pack from:
this needs to be besides the VVVV.OpenVR pack.
or double check whether you are starting the correct vvvv.exe, sometimes another vvvv instance is registered as the default for .v4p files. you can change this setting with the setup.exe besides vvvv.exe.
also regarding the DK2, it requires 'direct mode' to be working but most laptops with two graphics cards can't do that...
Thanks a lot for your answer, I think my problem was with me installing the Oculus SDK and SteamVR wrong. Anyway - after uninstalling everything and installing again it works with the newest version of oculus sdk (1.8), steamVR etc.
I should have deleted my question, but got carried away in the new good energy eager :)
Have a good evening!
Is it possible to disable the blue ring / grid? (I think it's called the chaperone)
I think you can set chaperone mode and color in the steamVR settings with your remote
is it right that the connection to steamvr only works if the oculus patch is in focus? so if i minimize (alt+3) it, vvvv.exe "is not responding".
is there any workaround?
and furthermore, are there any more dependencies for the second girlpower patch "patch in vr"? i have a few red nodes with hard links. the title had me quiet interested, as i'm still wondering how to get my patching window into vr.
thanks in advance, huge contribution!
for your fist question, there might be a solution but i have to check that... you can somewhere tell OpenVR that the application is a background application.
the patch in vr patch is only displaying the XML of a patch, you cannot interact with it really... i might remove that.
Guys check this out!!
OpenVR does something similar for you... i think it's currently not as good, but i read somewhere that there will be an update soon.
yeah I think it's reprojection in openvr terms
I mean this
"The hardware requirements for ASW are modest. This functionality has been enabled on all current-generation AMD GPUs (RX 400 series) and previous- or current-generation Nvidia GPUs (GTX 900 or 1000 series)."
I have a 950 on a notebook and I would love to use OpenVR without my big desk pc
the problem is that the specs are for desktop cards. unfortunately the mobile GTX cards have less then half of the computing performance than the desktop variants. so it's unlikely that any VR app will run smoothly on a 950M chip. my GTX 970M was barely able to drive the VR demos from valve... VR on laptops is currently no fun, also because most of them do not support direct mode, which is required for Oculus.
@tonfilm according to nvidia, that should change with the latest 10 series GPUs, where they have dropped the mobile naming and claim that the mobile versions are within 10% of their desktop counterparts.
I've a laptop with a 10 series card (GTX 1080) and the performance is great (with VR in general- I did have some of the performance issues in vvvv mentioned above but haven't had much of a chance to tweak things).
Hey guys, how can I stream VR player on the screen with chromakey body? Like on that video:https://youtu.be/Ikz5uJ0KE3w . You see, that man is playing the game,and we also can see his body. I understand that he has something green or blue befind him, and other "VR operator" shooting him and stream it with his body? How can I do it?
This could be a useful starting point:
Is there a solution to the post from
08/10/2016 - 11:26
Does using a 16bit texture format work?
@dominikKoller yes, definitely check that.
Using a 16bit texture format didn't help..
Can't test anymore, I don't have the hardware here!
@dominikKoller with Intersect (EX9.Geometry Quad) node it is easy to get the XY coordinates of where the controller is aimed at (and create a pointer that starts at the controller and ends at the intersection point with quad). It's then easy to map interface texture onto the quad and feed the interface with XY and controller interactions. Could be a HTMLTexture (DX11) and I guess that allows you to create interactive interface and read its state.
Nodes are missing for the Patch in VR patch, is this supposed to be working now? Or what are you using to do some patching while wearing the vr set? I tried with the camera but it is too low res, maybe screenshooting?
EDIT: https://sourceforge.net/projects/screencapturer/files/ this one works at least so it is possible ot tweak the scene wearing the vrset.
I am really missing a zoom feature here for vvvv though...
@io this patch is quite old and non functional. it was just there to display the patch. no interaction...
I'm facing some lag in the vive headset view after installing the new version (18/01/17) at every movement, as if I'm still at 90 fps. I mean every movement duplicate the content for a short time. It creates a bad feeling and a real vr sickness. this was not the case in the previous version.
Maybe someone as some idea how to fix it?
can you reproduce this behavior with the exact same patch and vvvv version with two different versions of the pack?
HI tonfilm. Thanks for your reply
just made the try before to read your answer, downgraded to previous pack and relunch my patch and the issue is still there. but I did not have this behavior before ( i would obviously noticed this ). I tried to change the texture format to 16bit in the renderer but it made the "movement delay" effect worst than in 32bits.
499 reality, i had the same trying the vrdemo patch in girlpower. by deleting the renderer which renders the scene in vvvv, the stutter disappeared.
@elektromeier, many thanks for your help, just tried this and removed the renderer for preview purpose but nothing changed in the headset.
maybe the stutter was a bit faster..
@499reality I have always seen the same thing, but only when actually moving. If I just rotate my head everything is smooth.
I suspect that with just rotation the update is happening in the OpenVR software (it can just slide the texture while doing the next render), but a move means that whole new textures from vvvv are needed.
Are you sure you weren't seeing the smooth pan before and now seeing the rougher images during movement? If you just turn your head is it smooth?
I think movement double image does indicate there is a double-buffer problem in there somewhere though.
Also, what GPU are you using?
i'm using nvidia gforce 1060 everything was quite perfect before.
you are right the head rotation is quite smooth and no stutter in this case, but when moving ( walking or moving the controllers) the things are going worst. it seems to be a position refresh issue, like a delay between the real position of my controller and the position received by the computer. as the effect appears bigger when moving faster.
@499reality Another reason I think it is a vvvv -> OpenVR buffer or frame rate issue is that when I move near the edge of the space and the "fence" pops up, the fence (being rendered by OpenVR) moves smoothly but the vvvv imagery does the double-image stutter.
At least with my setup, this is not only a problem with this plugin. I just tried "The Body VR", and if I set the rendering quality to high, the framerate drops, and I see the same exact double-imaging on movement. So this seems to happen when the rendering framerate drops below the Vive display rate even with Vive apps.
this pack is amazing, thanks for sharing! I've been using these nodes while prototyping my last project. i hooked up the vive with blender through osc using vvvv. for the final demo i turned towards unreal engine as the rendering features and accessibility were much more robust.
moving on to my next project i was wondering wether this pack supports the htc vive tracker by any chance?
hallo @hhdev nice to hear that it helped you prototyping. the trackers should just appear in the list of tracked devices the same as the controllers. but i have never tested it personally and haven't heard form anyone who tried it. if you have them and it won't work it should be a quick fix. if you are located in berlin, as the video suggests, it would be helpful to get the tracker for an afternoon to test it...
hey tebjan, thanks for the reply.
i ordered one which is supposed to be delivered in early may. i'll let you know if they work out of the box. i was just curious if anyone else already gathered some experience with the trackers. in case they don't i'll get in touch with you (you are right, i'm currently doing my interaction masters degree in weißensee)
Tracker works out of the box as is.
the directory paths in ViveLighthouse and ViveController nodes should be accessible as pin since not everyone chooses the default directory for installation :)
deleting the poser node crashes vvvv all the time (with 35.7 RC)
left and right touchpad visualization is reversed in ViveController node.
sorry for spam.. :)
Great great pack. Would be nice to install it over vpm!
Is there minimum PC spec that is recommended? does it run on windows 10?
there is rather a minimum graphics card that is recommended... any card better than NVidia GTX970 should work nicely. sure it runs on windows 10.
@ottomatic it works happily for me with a GTX 960, even a GTX 870m in my Sager laptop. Of course actual results will depend on scene complexity. I think if you are able to run things like TiltBrush or Night Cafe OK on your PC then it should also handle this.
Just trying this out for the first time, but I keep getting a red node for the Camera(OpenVR) when opening the OpenVRDemo.v4p from girlpower:
SteamVR is on latest beta and running, freshly installed vvvv 35.8, latest DX11, latest VVVV.OpenVR, GTX1070. Any ideas?
SteamVR says (unresponsive) vvvv.exe
Hey all, this pack is just perfect. It works very nicely with 3d scenes and is easy and fun to use!
I have a small problem using this pack:
The final texture that gets rendered to preview ( and I think HMD ) seems to be offset in x by a small amount. This amount stays constant and is exactly half of the eye distance calculated by using the eyeToHead output.
Splitting the Previewoutput of the Composer Node in two textures and layering them on top of each other shows the offset and makes it possible to get a perfect match by texturetransforming one of the images.
As a temporary fix, I texturetransformed the images before composing them to the headset which seems to do the trick.
I found this while building a mono- and stereoscopic 360 Imageviewer, while creating spheres exactly around both eyes. The spheres can be scaled without any change to this offset which seems to indicate they are in the right place.
It would be great to hear from you. This might also be a bug but I don't know enough yet to suggest that.
Thank you ;)
this sounds a bit special and i have no HMD here to test that. please make a forum entry with the same text, i hope others can confirm (or not) that.
issues from here usually get lost...
new behaviour arrived - i start vvvv with vr running (vive) - everything works
if i will turn controller off and on while vvvv vr is running, it stops receiving all buttons from the controller, dashboard works, tracking works, i can ping the controller, but vvvv does not get buttons, only after i restart vvvv AND vr as well, can someone confirm?
same behavior here with the controllers:
@andresc4 you need to apply that to each eye's camera/render output, make a copy of the Renderer (OpenVR) and plug in the fx there. and best start a forums thread for such questions.
Is it possible to make this track without HMD?
Imtrying to use only the tracker, but the poser showes no hmd error.
hello @Takuma unfortunately i don't have a tracker to test that. i've heard from other people that they successfully used the tracker, but not sure whether they used it without the headset. maybe @u7angel or @sebl knows more?
@tonfilm, one can setup the HTC driver to run on tracker only but the vvvv plugin insists on having the headset too. tried to grasp the plugin code to make a special case for "tracker only mode" but failed so far.
i could send you the tracker to berlin if you fancy...
@tonfim @U7angel Hello!! so On steamVR there is this default.vrsettings file that you just change RequireHMD to false, and works without Headset. Im trying to find that on Open vr plugin for vvvv
@takuma, this is what i said in my last comment, there is no option like that in the plugin yet.
@takuma you should be able to use the one in the Steam\Config folder. it should affect vvvv.
@u7angel thanks for the offer, but we currently don't have a vive here, so no base stations to test it with. but i'll come back to you as soon as we have a setup.
@tonfilm, i can send you a vive too :)
Has anyone got the vive to work without headset? I've tweaked the settings file, but still can't get to run without errors :(
I've adjust the default file here C:\Program Files (x86)\Steam\steamapps\common\SteamVR\resources\settings
as dicussed in this link
Also in the file Tonfilm has mentioned above, but I get no tracking...
The node gives a HMD not connected error still. If I discconnect the headset USB steam says the controllers cant see the base stations (the app has the HMD off, which is correct and still displays an error, but that is supposed to happen)
If anyone has had success, that would good to hear!
@catweasel @u7angle and @Takuma
i've checked the repo and found the place that requires some change:
if you have a vive and someone with basic c# skills you can hack it. otherwise you have to wait until we have a vive setup here in the office and i have to time to dig into it.
Ah ok, well I have almost basic c#, so I'll give it a go, what can go wrong!!
@rogalog as soon as SteamVR works properly with it, it should work with this pack.
@tonfilm: I understand that you have already tried Riftcat and that it is not working properly yet.
Anyway, it's good news that there is a chance ... I would like to use a few (4-6) Playstation move in my project, not necessarily as a Vive substitute. Data which I need is position, rotation and location in space. For now, I managed to pair them with Win7 PC and receive a stream of data via UDP. So far, I have not managed to translate received data to useful values.
I thought maybe it will be simpler and more elegant if I use OpenVR ...
did i? i can't recall that... do you have a link? :)
maybe I misunderstood your last response....
@rogalag ah, i see what you mean. what i meant was that if you have it running with SteamVR and other games work with it, it should also run fine with the pack. but i have not tested it so far.
Hallo, I have decomposed the renderer subpatch so that I can do some post processing which works fine. But I am a bit puzzled about how to go about when post processing involves using the View and Projection Matrix from the camera, for example volumetrics etc. Should we get just one eye? The average? Or what?
anonymous user login