This is a great new feature! Is anyone having an issue where you put your controllers down to just use your hands but the controller graphic stays on you hands? It is very distracting when trying to see things. Outside of that, this is an amazing new app
Thanks for spreading the news on MR. I think the PERFECT simulation experience will be a physical panel with working knobs, customizable gauges of all manner, paper charts & kneeboards, with VR green-screen type setup for everything outside the cockpit. I have a pimax Crystal and may dump it for the Quest3 if this is possible… It would be FANTASTIC to be able to build *MY* plane’s instrumentation, combined with the MSFS VR world.
They tell me that Steam makes layering video streams possible while the current OpenXR standards do not. I agree that SteamVR is a performance hog I’d rather avoid but the purpose of my mixed reality videos is to show the potential and hopefully stimulate development of OpenXR mixed reality tools.
I can't even run DCS in steam with a layer because it ends up being so jittery and choppy that it's unusable. I haven't tried Microsoft flight sim but I imagine that will be a similar situation. My system is not top of the line but it's no slouch. I have a 3090 with an I-9 9900k and 64 gigs of RAM.
This is amazing! Thank you for everything you do for the open-source community. Could you please create a step-by-step guide specifically for X-plane? I've followed all your videos on mixed reality, but I haven't been able to achieve any of the results.
Hi Russ. I tried it out in my physical home cockpit and it is absolutley the right direction for me! Did you find a way to run it in OpenXR (VDXR) over Virtual Desktop? Or is it bound to SteamVR at the moment? MSFS seems to run quite a bit smoother in OpenXR than SteamVR.
@srbarlow - I recently picked up a quest 3 and am trying to get it working like in your setup. My pass-through video looks terrible compared to yours. Do you find the bright images to be blown-out contrast-wise? I'm really thinking of exchanging this one for a different one.
hi, couple questions. you said that you can bind to joystick. however i cannot get it to bind. i can get it to work with keyboard, but not joystick. any ideas? i also noticed the colors from this app, to virtualdesktop do not match. do you know what RGB setting you had to change them too to match?
Do you have Virtual Desktop? I do, I have the same problem, i think we have to change the virtual environment to passthough on VD. Not sure though i will let you know when i test. My son is currently using the quest
Russ, both the Hand Portals and Color Panel apps require Virtual Desktop. I therfore deduce that the new Pimax Crystal Light will not be compatible with either of these options. I cannot determine from the Pimax website OR Web searches if the Crystal Light even has passthrough cameras on its basic model. Is my assumption that neither program works with the Crystal Light correct? If so, do you know if there is any hope of getting such functionality to work on the Crystal or Crystal Light?
Thanks for the update. I did download and use it and it is quite amazing. Unfortunately, steam VR slows DCS to a crawl in comparison to Open XR so it will not solve my MR challenges. Back to peeking through the nose hole :) I like where this is heading for the future.
Thanks for the info! What cpu/gpu? Yes too bad it is not OpenXR. Yanking and banking is definitely more demanding than straight and lever and pattern flying.
i5 12900k and a 4090 but even so this is a lot to ask especially via Virtual Desktop. Wish there was something like this for the Pimax Crystal :)@@rbarlow
De you need the Steam version of Virtual Desktop? And does it work IN VD like if you're just using your desktop in a VD environment and want to grab a coffee or do you have to use the passthrough VD environment?
I have a shoulder injury and my left shoulder hurts if I have my hands together on a keyboard too long so I have a split keyboard that is shoulder length apart so this would be perfect for me. Wish Immersed would do something like this, I'll suggest it on their forum.
Hey Russ, dd you see reality mixer just released a feature in beta that allows us to use a usb webcam in lieu of the headset passthrough cameras? This could be a game changer, especially since most webcams are better resolution than headset cameras.
Hi Russ, please allow one more question. You are so good in figuring out all of these details. Did you find a better pass through VR glass as the Quest 3? Is it worth spending more money on a Varjo XR3. As far as I see it here, the Quest 3 is pretty much good enough for all our needs? My goal would be to fly fully IMC in X-Plane to the minimum with it. Do you think its better to buy higher quality VR pass through glasses? Honestly I don't find better reviews on this topic. Do you heard of better pass through glasses which are worth it? Thanks a lot Russ. Appreciate all your contents. Christian
Wait for the XR-4 and Quest 3 is not bad to play with now but just not that easy to setup now … need better software like Varjo Lab Tools but for Quest
it's great !!!! so The controller is displayed on monitor when I do hand-tracking, do you know how to remove it? I can't see the controller in your video
Apologize for the inconvenience... I can't seem to find the download link. Could you post it in the "Update on ChromaTrack Ovberlay that provides hand tracked ..." section above? Thank you
Hi Russ. I am clearly missing something. I cannot even run virtual desktop without it forcing me to use my controllers. Then in steamvr it only works with controllers. How do I get steamvr to use hand tracking and not the controllers?
This looks really useful particularly for VR users flying IFR on a network like PilotEdge. (ie: CRAFT notes, ForeFlight) What headsets is this software compatible with? I use a Reverb G2.
Machine translated from the website: "It does not work with standalone HMD solids because of apps that utilize the VR overlay function of SteamVR." So it seems that it must use steam VR, which is annoying for DCS and MSFS pilots hoping to get a little extra performance by eliminating steamVR.
This is amazing! Projects like this and like the new UEVR injector make me super excited about the future of VR and MR. My question is, I use the cable commection for flight and race sim. I guess for now, there is no way to use it without a Virtual Desktop. I know there is a way to bring hand tracking to DCS but I dont know about MR.
I have used quest 2 with msfs before. Now, the only problem I did have was using the vatsim window, for chat and frequencies. I’m very good at knowing how to reach directly for my controls in vr, my brain to hand is very good. Is there any other way of having my vatsim window….maybe in the cockpit with quest 3? Or would I have to go down this route with the pass through windows to see the specific monitor with the vatsim window. Thank you.
@@rbarlow il be sure too. I’m looking at picking my quest 3 up in the next weeks. And it an ability I need with vatsim. I don’t really fly without vatsim. And I want to be in vr too. Il let you know of my findings. 👍🏻
Hey Russ, another great video thanks. Do you know if is possible to create a portal in one specific area.. for example for the A320, the light toggles are on top, and I have my button box on my desk, but I would like to when I look to the top it shows the portal with my button box. I know it could be confusing to look to the top and toggle the buttons in front of you.
Is there a way to put Microsoft FS 2020 and Xplane 11 and 12 onto the Steam platform? I bought all these directly from Microsoft and Xplane. Thanks. Really enjoy your videos and appreciate all your research!
Hi Russ, How do you get rid of the controllers. Mine is working fine (by the way no need for the steam version of MSFS I use the store version, launch the sim, then steam vr from VD and ask the sim to to go into vr - hey presto_) But when I am in VR I always see the controllers graphics from steam, can't seem to get rid of them and they block the view. This only happen here not in reality mixer by the way
There is a setting in VD controllers that says hide controllers if inactive (I think)and after they sit still they will disappear. Then just use the hand gesture (praying hands) to activate the portals.
This is the issue I'm having and I can't find the "hide controller if inactive" anywhere. When I set the controlers down, the portals snap to my hands but the controller graphic is still there. Any update on this?
As long as you have the physical cockpit on the motion rig I think it should work( of course you need motion compensation for using vr on the motion rig).
Great video. Do you know if there is being experimented with having the virtual hands and arms follow you real arms? That could be used for giving even more immersion when you hold the flight stick and flick switches. I know you would have to align you own IRL stick/yoke/wheel with the simulated one. I mostly use race sims, and when I have my IRL wheel aligned with the virtual one, my brain accepts the virtual arms as my own. But I would love for the arms to actually follow my own. That would be magic for the immersion.
First you need to launch SteamVR from the Virtual Desktop settings menu ( press menu button in Virtual Dexktop). Then you need the Steam version of MSFS and you need to launch it from steamVR library
You need to set the passthrough settings in Virtual Desktop to that same color so it becomes transparent instead of pink. Look at Streaming settings and under Passthrough select Enviroment checkbox. Click configure and then set the R,G,B values. There is now a Hands option in the Passthrough settings which, if hand tracking is active, will place see-through portals tracking your hands similar to this app in the video.
Too bad it's only for SteamVR, the overhead of SteamVr is too much for my liking, the Oculus runtime works way better for me personally on the Quest 3.
Don’t sugar coat it tell me how you really feel😀. I have owned lots of headsets including Aero and Crystal and for $500 Quest 3 isn’t a bad experience and has allowed me to experiment with Mixed Reality. Not a Meta fan for other reasons so I did compromise myself a bit when I bought it. Thanks for your comment.
Can you go over your VD streaming settings in the Quest 3 as well as VD settings on PC. I can't get VR to work with this when I use the runtime SteamVR but it goes straight in using VDXR. @srbarlow
I am quite baffled... on the site of the developer of this program there is no instruction or link to a youtube video where it shows how exactly one is supposed to use the program. In this movie and previous on this channel... same thing... lots of explanations of what it does... but... Maybe I am retarded... How?? I have steam VR. I have Virtual Desktop. I am in MSFS in VR... I start this program and all I can see are two solid pink spheres/circles no passthrough no nothing. It's amazing :D Was it supposed to be so easy to understand and use that nobody thought to explain how? edit. Found one!! th-cam.com/video/E7FjNJjyR8c/w-d-xo.html
Making amazing progress towards a truly workable mixed-reality sim setup! Thanks for all the sharing, Russ!
Thanks for sharing this update Russ. I’ve been following these videos with much curiosity. Thanks for pushing the edges for us in this space.
This is a great new feature! Is anyone having an issue where you put your controllers down to just use your hands but the controller graphic stays on you hands? It is very distracting when trying to see things. Outside of that, this is an amazing new app
Thanks for spreading the news on MR. I think the PERFECT simulation experience will be a physical panel with working knobs, customizable gauges of all manner, paper charts & kneeboards, with VR green-screen type setup for everything outside the cockpit. I have a pimax Crystal and may dump it for the Quest3 if this is possible…
It would be FANTASTIC to be able to build *MY* plane’s instrumentation, combined with the MSFS VR world.
Another step towards immersion 😊 this type of program indicates the next exciting development for Virtual Reality.
It's very cool, but I wish people would stop developing for steam VR and instead develop for openXR.
They tell me that Steam makes layering video streams possible while the current OpenXR standards do not. I agree that SteamVR is a performance hog I’d rather avoid but the purpose of my mixed reality videos is to show the potential and hopefully stimulate development of OpenXR mixed reality tools.
It's totally possible to do video letters but you have to build it yourself instead just using the steamVR api@@rbarlow
I can't even run DCS in steam with a layer because it ends up being so jittery and choppy that it's unusable. I haven't tried Microsoft flight sim but I imagine that will be a similar situation. My system is not top of the line but it's no slouch. I have a 3090 with an I-9 9900k and 64 gigs of RAM.
This is amazing! Thank you for everything you do for the open-source community. Could you please create a step-by-step guide specifically for X-plane? I've followed all your videos on mixed reality, but I haven't been able to achieve any of the results.
Very nice will definitely try it out. Also looking forward to the MR faceplate of the Pimax Crystal and use it with that!
Downloaded the software and donated for author, thanks!
Hi Russ. I tried it out in my physical home cockpit and it is absolutley the right direction for me! Did you find a way to run it in OpenXR (VDXR) over Virtual Desktop? Or is it bound to SteamVR at the moment? MSFS seems to run quite a bit smoother in OpenXR than SteamVR.
@srbarlow - I recently picked up a quest 3 and am trying to get it working like in your setup. My pass-through video looks terrible compared to yours. Do you find the bright images to be blown-out contrast-wise? I'm really thinking of exchanging this one for a different one.
hi, couple questions. you said that you can bind to joystick. however i cannot get it to bind. i can get it to work with keyboard, but not joystick. any ideas? i also noticed the colors from this app, to virtualdesktop do not match. do you know what RGB setting you had to change them too to match?
Hey Russ, can you do a how to setup this in VR? I have a Meta but my pink shows up. I can’t get it to pass through. I donated $20 when I downloaded it
Will this work with openXR? I own msfs20 & steamVR has to much baggage
Do you have Virtual Desktop? I do, I have the same problem, i think we have to change the virtual environment to passthough on VD. Not sure though i will let you know when i test. My son is currently using the quest
If i had a Vr setup, I would definitely use this!
Russ, both the Hand Portals and Color Panel apps require Virtual Desktop. I therfore deduce that the new Pimax Crystal Light will not be compatible with either of these options. I cannot determine from the Pimax website OR Web searches if the Crystal Light even has passthrough cameras on its basic model. Is my assumption that neither program works with the Crystal Light correct? If so, do you know if there is any hope of getting such functionality to work on the Crystal or Crystal Light?
Thanks for the update. I did download and use it and it is quite amazing. Unfortunately, steam VR slows DCS to a crawl in comparison to Open XR so it will not solve my MR challenges. Back to peeking through the nose hole :) I like where this is heading for the future.
Thanks for the info! What cpu/gpu? Yes too bad it is not OpenXR. Yanking and banking is definitely more demanding than straight and lever and pattern flying.
i5 12900k and a 4090 but even so this is a lot to ask especially via Virtual Desktop. Wish there was something like this for the Pimax Crystal :)@@rbarlow
De you need the Steam version of Virtual Desktop? And does it work IN VD like if you're just using your desktop in a VD environment and want to grab a coffee or do you have to use the passthrough VD environment?
Hi Russ, how do you get the combined picture into obs? Steamvr window doesn't show the portals for me in obs
I have a shoulder injury and my left shoulder hurts if I have my hands together on a keyboard too long so I have a split keyboard that is shoulder length apart so this would be perfect for me. Wish Immersed would do something like this, I'll suggest it on their forum.
Hey Russ, dd you see reality mixer just released a feature in beta that allows us to use a usb webcam in lieu of the headset passthrough cameras?
This could be a game changer, especially since most webcams are better resolution than headset cameras.
Fascinating add on!
Have you tried this with a glass cockpit in Msfs. My G1000 screens freeze as soon as I enter VR. Still trying to get it to work 😢
Does this work in Xplane12 or just MSFS?
Great stuff thanks for sharing.
Hi Russ, please allow one more question. You are so good in figuring out all of these details. Did you find a better pass through VR glass as the Quest 3? Is it worth spending more money on a Varjo XR3. As far as I see it here, the Quest 3 is pretty much good enough for all our needs? My goal would be to fly fully IMC in X-Plane to the minimum with it. Do you think its better to buy higher quality VR pass through glasses? Honestly I don't find better reviews on this topic. Do you heard of better pass through glasses which are worth it?
Thanks a lot Russ. Appreciate all your contents. Christian
Wait for the XR-4 and Quest 3 is not bad to play with now but just not that easy to setup now … need better software like Varjo Lab Tools but for Quest
Wow, that's impressive.
Wow... AR is really comming!
Thanks for this video. What system (PC and headset) are you using?
it's great !!!!
so The controller is displayed on monitor when I do hand-tracking, do you know how to remove it?
I can't see the controller in your video
Apologize for the inconvenience... I can't seem to find the download link. Could you post it in the "Update on ChromaTrack Ovberlay that provides hand tracked ..." section above? Thank you
Hi Russ. I am clearly missing something. I cannot even run virtual desktop without it forcing me to use my controllers. Then in steamvr it only works with controllers. How do I get steamvr to use hand tracking and not the controllers?
quick question: are you able to select a knob in vr and then use knobster once it's selected?
This looks really useful particularly for VR users flying IFR on a network like PilotEdge.
(ie: CRAFT notes, ForeFlight)
What headsets is this software compatible with? I use a Reverb G2.
Is it mandatory to use Steam VR ?
Machine translated from the website:
"It does not work with standalone HMD solids because of apps that utilize the VR overlay function of SteamVR."
So it seems that it must use steam VR, which is annoying for DCS and MSFS pilots hoping to get a little extra performance by eliminating steamVR.
This is amazing! Projects like this and like the new UEVR injector make me super excited about the future of VR and MR.
My question is, I use the cable commection for flight and race sim. I guess for now, there is no way to use it without a Virtual Desktop. I know there is a way to bring hand tracking to DCS but I dont know about MR.
Pressing 2 hands together in japanese means gassho or to appreciate its used lots especially in traditional martial arts like karate
I have used quest 2 with msfs before. Now, the only problem I did have was using the vatsim window, for chat and frequencies. I’m very good at knowing how to reach directly for my controls in vr, my brain to hand is very good. Is there any other way of having my vatsim window….maybe in the cockpit with quest 3? Or would I have to go down this route with the pass through windows to see the specific monitor with the vatsim window. Thank you.
I think this is what you want vpilot.rosscarlson.dev/ and sim-efb.com/
Read here www.reddit.com/comments/rnvzb6?
@@rbarlow thank you very much. I shall check this out. I enjoy your informative video’s. Keep up the good work.
@@Test-Pilot1194let me know your results. I haven’t tried this myself.
@@rbarlow il be sure too. I’m looking at picking my quest 3 up in the next weeks. And it an ability I need with vatsim. I don’t really fly without vatsim. And I want to be in vr too. Il let you know of my findings. 👍🏻
Hey Russ, another great video thanks.
Do you know if is possible to create a portal in one specific area.. for example for the A320, the light toggles are on top, and I have my button box on my desk, but I would like to when I look to the top it shows the portal with my button box. I know it could be confusing to look to the top and toggle the buttons in front of you.
Sounds like rreality Mixer is what you want. Did you see my two previous videos on that. It creates toggle-able fixed portals.
Is there a way to put Microsoft FS 2020 and Xplane 11 and 12 onto the Steam platform? I bought all these directly from Microsoft and Xplane. Thanks. Really enjoy your videos and appreciate all your research!
Hi Russ, How do you get rid of the controllers. Mine is working fine (by the way no need for the steam version of MSFS I use the store version, launch the sim, then steam vr from VD and ask the sim to to go into vr - hey presto_) But when I am in VR I always see the controllers graphics from steam, can't seem to get rid of them and they block the view. This only happen here not in reality mixer by the way
There is a setting in VD controllers that says hide controllers if inactive (I think)and after they sit still they will disappear. Then just use the hand gesture (praying hands) to activate the portals.
This is the issue I'm having and I can't find the "hide controller if inactive" anywhere. When I set the controlers down, the portals snap to my hands but the controller graphic is still there. Any update on this?
Will mixed reality on the quest 3 allow usage with a motion simulator? Or will pass thru windows not move with the motion ?
As long as you have the physical cockpit on the motion rig I think it should work( of course you need motion compensation for using vr on the motion rig).
yes I do. Thanks for the quick reply and all your awesome videos!@@rbarlow
Great video. Do you know if there is being experimented with having the virtual hands and arms follow you real arms? That could be used for giving even more immersion when you hold the flight stick and flick switches. I know you would have to align you own IRL stick/yoke/wheel with the simulated one.
I mostly use race sims, and when I have my IRL wheel aligned with the virtual one, my brain accepts the virtual arms as my own. But I would love for the arms to actually follow my own. That would be magic for the immersion.
I think Meta even has full body tracking for the Quest 3 but the game developer would have to take advantage of the technology in the game.
Russ: I'm too stupid to live. I've just acquired Meta Quest 3 VR goggles. Do you think I can connect them to X-Plane 12? HELP!!!
Thank you Russ. Can we use reality mixer and this program together?
Possibly but not sure performance would be very good.
Hi Russ, do you have the MS store version of MSFS, or do you need to have MSFS installed from Steam?
First you need to launch SteamVR from the Virtual Desktop settings menu ( press menu button in Virtual Dexktop). Then you need the Steam version of MSFS and you need to launch it from steamVR library
I have the store version and it works fine, no need to launch it from steam.
@@rbarlow
OK thanks, I have the MS store version so it won't work for me.@@rbarlow
It would be great for VRChat when you’re drinking with friends.
Cheers!
Using your settings, my hands turned pink, not transparent.
You need to set the passthrough settings in Virtual Desktop to that same color so it becomes transparent instead of pink. Look at Streaming settings and under Passthrough select Enviroment checkbox. Click configure and then set the R,G,B values. There is now a Hands option in the Passthrough settings which, if hand tracking is active, will place see-through portals tracking your hands similar to this app in the video.
Too bad it's only for SteamVR, the overhead of SteamVr is too much for my liking, the Oculus runtime works way better for me personally on the Quest 3.
Has anyone in here been able to get the Pico 4 to work with this?
Pico 4 doesn't support hand tracking on PCVR, so doing this on MSFS VR is not going to work with a Pico 4.
I wouldn't give 10 cents for anything QUEST... RIPOFF and their goggles are overpriced glorified phone stuck in some shitty goggles..
Don’t sugar coat it tell me how you really feel😀. I have owned lots of headsets including Aero and Crystal and for $500 Quest 3 isn’t a bad experience and has allowed me to experiment with Mixed Reality. Not a Meta fan for other reasons so I did compromise myself a bit when I bought it. Thanks for your comment.
Can you go over your VD streaming settings in the Quest 3 as well as VD settings on PC. I can't get VR to work with this when I use the runtime SteamVR but it goes straight in using VDXR. @srbarlow
I am quite baffled... on the site of the developer of this program there is no instruction or link to a youtube video where it shows how exactly one is supposed to use the program. In this movie and previous on this channel... same thing... lots of explanations of what it does... but... Maybe I am retarded... How??
I have steam VR. I have Virtual Desktop. I am in MSFS in VR... I start this program and all I can see are two solid pink spheres/circles no passthrough no nothing.
It's amazing :D Was it supposed to be so easy to understand and use that nobody thought to explain how?
edit.
Found one!!
th-cam.com/video/E7FjNJjyR8c/w-d-xo.html