This is the game changer that is needed for flight simulation in VR. VR is such an amazing advance but without being able to use my peripherals effectively it's simply a no-go for me. This is the start of something really exciting and I can see it unlocking things in the very near future.
Keep up with the awesome work, Russ! Your combination of personality, professionalism, and infectious enthusiasm make you the best resource online that I keep as a ready reference! I'm just breaking into this mixed reality enhancement. It makes me leave my real planes in the hanger sometimes or allows me to take these virtual birds to flight on days with icing IMC. All the best to you and please keep amazing us!
cool, I did like to see the user hands being passthrough and able to interact with the plane controls.
ปีที่แล้ว +8
I've been using VR since it became a thing. My first headset was the Oculus CV1 and today, I'm using the Varjo Aero. I'm still as amazed what VR brings to the table as I was on day 1. Spending the majority of time flying in VR in MSFS 2020 and absolutely love it! What I really would like is being able to get rid of my mouse and simply use my hands instead to manipulate all knobs, switches etc. Where the mouse cursor seen in MSFS 2020 would be replaced by my hand(s). Preferably "feeling" the switch between my fingers thanks to some form of haptic feedback. I know there have been a few products out there, I think one of them was called VR Gloves. Unfortunately though, my understanding is there's currently no support for this in MSFS 2020 but I would be thrilled to be proven wrong!
Great to see pass through for your hands or a custom masked area. This would also be good for sim racers to see your steering wheel, shifter, and e-brake!
This is pretty great! The first thing I thought that could make it perfect would be the ability to offset the circle. So like the pinky side of the hand would be the edge and it would show more on the thumb side where all the action is.
Good work buddy I will watch your development work with interest and I hope the industry does too. I can just imagine how much work you're doing at your computer late at night ...thank you, it allows me to have a normal balanced life again while I wait for you to perfect it.
The interest for me is not seeing low res real world cockpit items but hand tracking VR hands to use VR cockpit buttons . Have you tried this in MSFS yet?
I am impatient to experience MR so I am willing to accept the resolution limitations. The Varjo XR-4 will offer the resolution needed as well excellent software tools to make it work rather seamlessly. Pimax is supposedly working on a Mixed Reality faceplate that will add stereo color pass through video to the Crystal headset. It offers very good resolution and will create even more downward pressure in MR costs. Of course MR can reduce the weight of a cockpit to the point where a motion platform becomes more practical.
Hello Russ! I don't know if you were lucky to find other feature to switch on hands passthrough without contollers functioning. I tried this program from Japanese guy, but there was still controllers acting with hands and unstable work of mouse. Recently I've found in Virtual Desktop Discord easy solution of the problem: you should turn on hands tracking as usual but turn off "Track controllers" in Streaming. And it works perfectly, hands, clear passthrough circles, mouse or controllers for panel and CDU switches. And no need in other programs, just VD. Thank you very much for your videos!
I'm still convinced it should be totally possible to track your hands to interact with the cockpit switches. Every attempt I've seen to dayt though just kind of does finger pointing in some way, but it should be completely possible to actually track your hand movements and overlay them with the switches so that making a flip up or down motion, or turning motion actually does the same to the in cockpit switch. I'm really not sure why I haven't seen anyone tackle this unless I've just missed it.
Actually, just such a system at FlightSimCon expo last year in Houston. YawVR had a motion simulator with a blank Cessnapanel (it was black, but included all the switches and knobs and bezels and geometry) and with the hand tracking of the quest 2 I was able to reach out and touch physical switches that were matched up with VR. The have this product you can buy called InfuseVR. Expensive and limited to one aircraft type.
I'm a big hardware guy. I love having physical controls, and that is really problematic with VR. When I do use VR, I'm using the Reverb G2 with black and white passthrough. :( Ultimately, I would like to see hand tracking allowing manipulation of the controls in the virtual cockpit. When that becomes a reality though, passthrough will still be needed for certain things like taking a drink, and if the resolution was good enough, writing things down, like clearances and ATIS etc. This software is a solid incremental advancement of the tech allowing better use of physical controls in VR, and I'm excited to see it. Bravo! Thanks Russ, for keeping us apprised of these advancements!
I know that is possible through OpenXR toolkit. I don't know about MSFS but I have heard it works, but I have my Q3 that it also supports working exactly like this in DCS, being able to flip switches and push buttons naturally.
Perhaps the best solution I’ve seen so far. If the ‘’switch’’ could be proximity related that would be even better. You say you purchased the Steam version? Does this mean the store version will not work with it? Thanks for the video and Happy Xmas.
Merry Christmas Mark! I’m no expert on SteamVR but I know you need to launch the sim from SteamVR. You may know if that is possible but I couldn’t get it to work until I purchased the Steam version. As I said in the video this is early beta and needs some work to be really practical. We need something that can run directly on OpenXR and simplify things. Varjo has a webinar on their Varjo Lab Tools that demonstrates the ”gold standard“ of Mixed Reality software that we should be shooting for.
Developer note; "It cannot be used if it does not meet the requirements. Due to SteamVR overlay API by Unity and Virtual Desktop, There are no plans to develop on other platforms or devices and OpenXR".
Not a fan of mixed reality myself. My perfect solution would involve VR gloves replacing the VR controllers. X-Plane would read the gloves and put avatars of my hands in the cockpit, positioned, articulated and lit appropriate to the scene. I'm thinking something similar to VTOL VR's inverse-kinetic body simulation.
if anyone is wondering about if reality mixer works with DCS and warthunder, it does! thanks russ for these vids! just uploaded a short if anyone wants to see it in action with an f16 *gotta realighn my mfds lol) =p the future is bright!
Hey Russ. Just wanted to drop a little tidbit. Since this was designed for helping eat. In Japanese culture right before they eat they put their hands together, similar to prayer, and say "itadakimasu" which means "I humbly receive". This is why the activation gesture seems so clunky.
Can this ChromaTrak app be combined with reality mixer? I have my iPad with Foreflight mounted permanently on my sim cockpit: can I cut a permanent hole out for that in Reality Mixer, and then still use ChromaTrak to track my hands as needed on the rest of my controls?
Interesting Russ. As stated before in your other videos on this subject, for cockpit builders the developers should reverse the process. What in the end is needed is a green/pink screen outside the cockpit that shows the outside view while the passthrough shows your physical cockpit. As you said, the devs do not know what (virtual) pilots need. Might it be an idea that someone like you, with your longtime experience as a pilot and being a 'flight sim evangelist', reaches out to them and put them on the right track?😅 You've done that with the knobster. But first, have a Merry Christmas and a Happy New Year.
I have a question, sir. If you are still using the quest three, how do you keep it charged? I have a bobo VR headset and I enjoy the fan plus the battery. How can you get a connection to the PC wired while keeping your bobo VR loveliness intact. Thank you.
Wow! That's impressing!! Does it work only with oculus headsets?? I use HP reverb 2. Would it work with it? Would you show us in detail how you get it working? Thanks! Subscribing and liking!!
No because the cameras are black and white so no chroma key possible. Reality Mixer will work but the quality is not great see my video th-cam.com/video/FBkbTo_pSXg/w-d-xo.htmlsi=-favPogZLqLoMedN
Russ. Great video and product. I downloaded and executed, but do not know how to activate it when starting x-plane through virtual desktop. Any advice? Thank you
Hi, I'm using Quest 3. But I made a camera box, but I can't get a pass-through image. I've tried camera setup, but all the buttons are inactive. What's the problem? If you know anything, please reply
I couldn't get this to work for me (I'm 66 years old) has there been any progress from when you first showcased this software? I would love to get it to work. I couldn't get Reality Mixer to work either. I know I'm probably missing something simple. (that's usually my problem).
Amazing, wish we could get this in OpenXR. Is Quest color pass through ? Very cool technology I have some physical mfds for DCS, I would love to be able to see them. Blindly hunting for them is not so great today
Quest 3 is color pass through in stero and with LIDAR ranging. Amazing tech for $500. The resolution may be a bit short of being able to read the MIPs but no problem with the MIP buttons and other controls. It does work with DCS using SteamVR.. proof see th-cam.com/users/shortsGqjJ0GOB1AM?si=uwZrRvUBe3kZOwZc
No way!!!! this is so huge for VR man! I must ask though.....you say it doesn't break the immersion very much. How do I ask this.....on a scale of 1 to 10 where VR is a 10 and flat screen is a 1 how much does this pass thru change the 10?
I absolutely loved this demonstration you made. It made me want to go out and purchase the MetaQuest 3 right away. My HP Reverb G2 just does not give the immersion offered by hand tracking. Two questions... 1) Are you still as satisfied with it as you were in the demo video? 2)Would you get the 128GB or 512GB model of Quest 3?
@@rbarlow Do you have to change some settings in virtual desktop streamer? It always tells me, that it's waiting for wlan connection even with link cable active
I would kill for some kind of app that could use my phone’s camera to track my hands to allow for this same kind of feature to be used with my Varjo Aero.
So Russ, now for the most important question: is the Quest 3 resolution high enough to allow the proper reading of all the instruments, especially the GPS fine text? That’s what prevents me for biting the bullet for now, as the high resolution headset such as Varjo, Pimax Crystal are really over my budget. If the Quest 3 resolution is enough, the video pass through features could be the game changer I’m looking for. Thanks
No it isn’t there yet. It is fun but I cannot read the kollsman window to set the altimeter as an example . It is very realistic and good for VMC flying but the Glass displays are not very usable. The $4000 Varjo XR-4 is what we need but we need it for a sub $2000 price tag. The Quest 3 is awesome for the price but the resolution is still not great for the finer details.
@@rbarlow Thank you for your reply Russ. That’s what I suspected. The idea being to fly A320/330 for training, VR will be the way to go when prices will drop or resolution improves
Thanks Russ for this Very interesting app allowing video passthrough. Is this also usable without VR ? Happy Christmas days to you, And Thanky for all you do for the FS Community! :)
Id LOOOOVE to fly msfs 2020 in VR.. If they make a VR adaptation for the Series X or a cloud VR msfs 2020 Adaptation for the Meta Quest Ill buy the Quest 3 in 3 minutes..
Hi Russ, great video! Can you give details please as to how you got Reality mixer working in MSFS2020. I can start MSFS, and the passthrough works, but immediately I turn on VR mode in MSFS, the Reality Mixer boxes disappear. Any help greatly appreciated.
I already was using Steam VR, but i think i know what is going on. The boxes appear if i use proximity or glance options. My original desire to use a keyboard key to toggle them on or off appears to me to not work while in VR. I’m hoping the developer of Reality Mixer will respond to the Steam Community post I entered.
I use Virtual Desktop and Steam vr, but does not get any vr in my quest 3 when entering vr mode in msfs, works with airlink. The open xr is set to Steamvr. Any tips?
I don’t quite understand what you mean. Are you able to use Q3 normally with MSFS to fly in VR? Or do you mean you cannot get the chromaTrack overlay to work with MSFS in VR?
I am trying to play a pc game using virtual desktop so I have a larger screen. I want to have to boxes of where my small keyboard and mouse are so that I can see them, is this possible?
Lots of light makes the difference… direct flood light on panel also dim any LCD displays to less than 30-40% max brightness so the white doesn’t bloom
If you mean hand gestures in the air The tactile inputs are missing and trying to accurately turn a virtual dual concentric knob like those on a radio or the G530 is not easy. I tried InfuseVR by Yaw at Flight Sim Expo in Houston and they had a “blank” Cessna 172 panel with only the knobs and buttons and using the Quest 2 hand tracking it worked pretty well. Just have to build a 100% accurate model of the real panel so things are there when you reach for them and keep the panel and VR world perfectly aligned.
@@Redridge07I think unless some extremely lightweight gloves with haptic feedback could simulate the tactile feel of the physical world which isn’t impossible especially where we have come even on the last 10 years
It's what we need, and is a great idea...but to make the SteamVR app a requirement is a huge mistake. Most serious flyers are using Virtual Desktop and it's VDXR implementation of OpenXR and this is due to the fact the SteamVR offers terrible performance in MSFS as compared to VD and the XR Toolkit combo. There are now 3 different products like this, and they are all requiring SteamVR. What's really needed is an option to run these sorts of apps WITHOUT SteamVR.
Does Reality Mixer work with ILS2? I can't get Reality Mixer to come on with it when using Virtual Desktop and Quest 3. Is there a detailed video about how to make it work?
It should work but you need to do several things in the settings to set up the cameras(actually deactivate them) and instead enable the chroma key boxes. You also need to edit the boxes. In a week or two when I have things fully figured out I will try to do a tutorial step by step guide unless someone else has already done one. I explained the process is a bit complicated and required a few steps be done in a specific order (I think) . I know what works for me but there could be some flexibility but I am not sure.
Capture on the Quest 3 headset and then download to PC via link cable With headset on disconnect link and reconnect. When you get a quick notification that a usb device is connected click to accept and the quest storage will appear as a drive on your computer
this is cool , I've been using Reality mixer for this exact concept already, except without the hand tracking simply drawing the boxes over the peripherals i want to see completly like my keyboard, mfd etc ..the reality boxes activate when i move my hand close to my real device and disappear when i move my hands away
Quest 2 experimented with hand tracking, and they have continued it through to the Quest 3 which uses the cameras on the headset to track your hands without the controllers.
Maybe the reason those RGBA settings aren't the same as the ones in the desktop is perhaps they are setup for ppl with different kinds of color blindness rather then trying to setup a color, but that's just a guess. I still have to get on this band wagon Desperately lol. Asked Santa for a quest 3 but he didn't hook it up LMFAO.
You say you find this doesn't break immersion much but from just looking at a video it seems much less immersive. Are you meaning comparing it to continuously taking your headset off or are you comparing it to the passhthrough sim cockpit setup you tried in an earlier video?
What I mean is that in your normal cockpit scan the passthrough areas are mostly in your peripheral vision and not very noticeable. Obviously it is less obtrusive than taking your headset off and on. I did try YawVR’s Cessna cockpit with InfuseVR which maps the hand tracking to a dummy cockpit with physical switches and knobs. It was pretty immersive visually and the hardware was there when I reached out to grab the controls. But it is not cheap to build and is limited to a single model airplane.
I attempted this level of viewer immersion with green screen for 3 years straight, everyday and could not get it how I wanted it so I stopped making videos. This is awesome and inspiring Russ! 🎉
SteamVR: PUKE Seriously, can we get past SteamVR and just go pure OpenXR at this point? SteamVR is just sub optimal when it comes to performance, ESPECIALLY in DCS.
I have been told OpenXR doesn’t currently support layered video needed for this. SteamVR is a vehicle that supports the video layer on top to nahe these things work. Hopefully OpenXR will adopt such technology. In the meantime take some Dramamine and try it😀
@@rbarlow SteamVR is a nogo for me. The overhead cost isn't worth it. I mainly do DCS, and VDXR or straight OpenXR is the way forward. OpenXR not supporting layers is absolute bullshit, just devs being fucking lazy and not wanting to conform to the new standard.
@@rbarlow Yeah sorry to rush you lol :) I know Hollidays I was just Frustrated by the fact that I did not get it how to make this work and Usualy I do lol so yeah no worries If it was not of you I would not have Heard about this so Thanks again :)
Ahem, I did watch to the end and caught you saying the Steam version of MSFS worked [which surprised you], but my [perhaps unclear]question was focused on the downloaded version of MSFS from Microsoft, not the Steam version. Can you affirm it works that version, too? Thanks for your video and an agreeably swift reply to my question.@@rbarlow
This is the game changer that is needed for flight simulation in VR. VR is such an amazing advance but without being able to use my peripherals effectively it's simply a no-go for me. This is the start of something really exciting and I can see it unlocking things in the very near future.
Keep up with the awesome work, Russ!
Your combination of personality, professionalism, and infectious enthusiasm make you the best resource online that I keep as a ready reference! I'm just breaking into this mixed reality enhancement. It makes me leave my real planes in the hanger sometimes or allows me to take these virtual birds to flight on days with icing IMC. All the best to you and please keep amazing us!
This is super cool. I just found your reality mixer video yesterday, the hand tracking really takes it up a notch over rectangular regions.
cool, I did like to see the user hands being passthrough and able to interact with the plane controls.
I've been using VR since it became a thing. My first headset was the Oculus CV1 and today, I'm using the Varjo Aero. I'm still as amazed what VR brings to the table as I was on day 1. Spending the majority of time flying in VR in MSFS 2020 and absolutely love it!
What I really would like is being able to get rid of my mouse and simply use my hands instead to manipulate all knobs, switches etc. Where the mouse cursor seen in MSFS 2020 would be replaced by my hand(s). Preferably "feeling" the switch between my fingers thanks to some form of haptic feedback.
I know there have been a few products out there, I think one of them was called VR Gloves. Unfortunately though, my understanding is there's currently no support for this in MSFS 2020 but I would be thrilled to be proven wrong!
Interesting concept to use the hand tracking. Another great video, keep up the great work!
Great to see pass through for your hands or a custom masked area. This would also be good for sim racers to see your steering wheel, shifter, and e-brake!
This is pretty great! The first thing I thought that could make it perfect would be the ability to offset the circle. So like the pinky side of the hand would be the edge and it would show more on the thumb side where all the action is.
Good work buddy
I will watch your development work with interest and I hope the industry does too. I can just imagine how much work you're doing at your computer late at night ...thank you, it allows me to have a normal balanced life again while I wait for you to perfect it.
We really do live in amazing times! Thanks for sharing this. :)
This is such a great and informative video, Russ! I think this is a solution to a lot of simmers for both MSFS and DCS World.
Downloaded Chroma and got it working earlier this evening. Thanks!
WOW, very cool. That passthrough might be the ticket. I reley alot of musle memmory but there are times where I forget where something is.
The interest for me is not seeing low res real world cockpit items but hand tracking VR hands to use VR cockpit buttons . Have you tried this in MSFS yet?
Brilliant Russ, you always know how to find new ways, great🎉
I am impatient to experience MR so I am willing to accept the resolution limitations. The Varjo XR-4 will offer the resolution needed as well excellent software tools to
make it work rather seamlessly. Pimax is supposedly working on a Mixed Reality faceplate that will add stereo color pass through video to the Crystal headset. It offers very good resolution and will create even more downward pressure in MR costs. Of course MR can reduce the weight of a cockpit to the point where a motion platform becomes more practical.
Hello Russ! I don't know if you were lucky to find other feature to switch on hands passthrough without contollers functioning. I tried this program from Japanese guy, but there was still controllers acting with hands and unstable work of mouse. Recently I've found in Virtual Desktop Discord easy solution of the problem: you should turn on hands tracking as usual but turn off "Track controllers" in Streaming. And it works perfectly, hands, clear passthrough circles, mouse or controllers for panel and CDU switches. And no need in other programs, just VD.
Thank you very much for your videos!
I'm still convinced it should be totally possible to track your hands to interact with the cockpit switches. Every attempt I've seen to dayt though just kind of does finger pointing in some way, but it should be completely possible to actually track your hand movements and overlay them with the switches so that making a flip up or down motion, or turning motion actually does the same to the in cockpit switch. I'm really not sure why I haven't seen anyone tackle this unless I've just missed it.
Actually, just such a system at FlightSimCon expo last year in Houston. YawVR had a motion simulator with a blank Cessnapanel (it was black, but included all the switches and knobs and bezels and geometry) and with the hand tracking of the quest 2 I was able to reach out and touch physical switches that were matched up with VR. The have this product you can buy called InfuseVR. Expensive and limited to one aircraft type.
I'm a big hardware guy. I love having physical controls, and that is really problematic with VR. When I do use VR, I'm using the Reverb G2 with black and white passthrough. :( Ultimately, I would like to see hand tracking allowing manipulation of the controls in the virtual cockpit. When that becomes a reality though, passthrough will still be needed for certain things like taking a drink, and if the resolution was good enough, writing things down, like clearances and ATIS etc. This software is a solid incremental advancement of the tech allowing better use of physical controls in VR, and I'm excited to see it. Bravo! Thanks Russ, for keeping us apprised of these advancements!
I know that is possible through OpenXR toolkit. I don't know about MSFS but I have heard it works, but I have my Q3 that it also supports working exactly like this in DCS, being able to flip switches and push buttons naturally.
That would be great to see! Using a Q3 in DCS and have yet to get passthrough to work at all. 😢
Perhaps the best solution I’ve seen so far. If the ‘’switch’’ could be proximity related that would be even better. You say you purchased the Steam version? Does this mean the store version will not work with it? Thanks for the video and Happy Xmas.
Merry Christmas Mark! I’m no expert on SteamVR but I know you need to launch the sim from SteamVR. You may know if that is possible but I couldn’t get it to work until I purchased the Steam version. As I said in the video this is early beta and needs some work to be really practical. We need something that can run directly on OpenXR and simplify things. Varjo has a webinar on their Varjo Lab Tools that demonstrates the ”gold standard“ of Mixed Reality software that we should be shooting for.
Developer note; "It cannot be used if it does not meet the requirements. Due to SteamVR overlay API by Unity and Virtual Desktop, There are no plans to develop on other platforms or devices and OpenXR".
Not a fan of mixed reality myself. My perfect solution would involve VR gloves replacing the VR controllers. X-Plane would read the gloves and put avatars of my hands in the cockpit, positioned, articulated and lit appropriate to the scene. I'm thinking something similar to VTOL VR's inverse-kinetic body simulation.
Nice video, Russ, thanks! I look forward to how this technology evolves for sure....
I am a sim racer and have a Quest 3 coming this week. This would be very helpful.
if anyone is wondering about if reality mixer works with DCS and warthunder, it does!
thanks russ for these vids!
just uploaded a short if anyone wants to see it in action with an f16 *gotta realighn my mfds lol) =p
the future is bright!
I had heard it was good with DCS on SteamVR but thanks for confirming it. Found it th-cam.com/users/shortsGqjJ0GOB1AM?si=uwZrRvUBe3kZOwZc
Nice flying!
Wow! Meta should get this guy into their development team! Good video!!!
if guy Godin is an example to go by, they would rather just steal the idea and take credit anyway .
Hey Russ. Just wanted to drop a little tidbit. Since this was designed for helping eat. In Japanese culture right before they eat they put their hands together, similar to prayer, and say "itadakimasu" which means "I humbly receive". This is why the activation gesture seems so clunky.
Any idea why I can't get it out of controller mode, it works great on controllers, the gesture won't work.
I like you highlighting the tech and how to get it to work, but man, that last takeoff made me clench really hard.
Can this ChromaTrak app be combined with reality mixer? I have my iPad with Foreflight mounted permanently on my sim cockpit: can I cut a permanent hole out for that in Reality Mixer, and then still use ChromaTrak to track my hands as needed on the rest of my controls?
Interesting Russ. As stated before in your other videos on this subject, for cockpit builders the developers should reverse the process. What in the end is needed is a green/pink screen outside the cockpit that shows the outside view while the passthrough shows your physical cockpit. As you said, the devs do not know what (virtual) pilots need. Might it be an idea that someone like you, with your longtime experience as a pilot and being a 'flight sim evangelist', reaches out to them and put them on the right track?😅 You've done that with the knobster.
But first, have a Merry Christmas and a Happy New Year.
Thanks for great video, merry Xmas 😀
I have a question, sir. If you are still using the quest three, how do you keep it charged? I have a bobo VR headset and I enjoy the fan plus the battery. How can you get a connection to the PC wired while keeping your bobo VR loveliness intact. Thank you.
With MS recent announcement about windows mixed reality being deprecated, will that cause an issue with using FS2020 like you've been doing in XP?
Wow! That's impressing!!
Does it work only with oculus headsets?? I use HP reverb 2. Would it work with it? Would you show us in detail how you get it working?
Thanks! Subscribing and liking!!
No because the cameras are black and white so no chroma key possible. Reality Mixer will work but the quality is not great see my video
th-cam.com/video/FBkbTo_pSXg/w-d-xo.htmlsi=-favPogZLqLoMedN
That IS the future of VR :)
Russ. Great video and product. I downloaded and executed, but do not know how to activate it when starting x-plane through virtual desktop. Any advice? Thank you
Hi, I'm using Quest 3. But I made a camera box, but I can't get a pass-through image. I've tried camera setup, but all the buttons are inactive. What's the problem? If you know anything, please reply
I couldn't get this to work for me (I'm 66 years old) has there been any progress from when you first showcased this software? I would love to get it to work.
I couldn't get Reality Mixer to work either. I know I'm probably missing something simple. (that's usually my problem).
Amazing, wish we could get this in OpenXR. Is Quest color pass through ?
Very cool technology
I have some physical mfds for DCS, I would love to be able to see them. Blindly hunting for them is not so great today
Quest 3 is color pass through in stero and with LIDAR ranging. Amazing tech for $500. The resolution may be a bit short of being able to read the MIPs but no problem with the MIP buttons and other controls. It does work with DCS using SteamVR.. proof see th-cam.com/users/shortsGqjJ0GOB1AM?si=uwZrRvUBe3kZOwZc
No way!!!! this is so huge for VR man! I must ask though.....you say it doesn't break the immersion very much. How do I ask this.....on a scale of 1 to 10 where VR is a 10 and flat screen is a 1 how much does this pass thru change the 10?
Fantastic Russ, keep them coming :)
I absolutely loved this demonstration you made. It made me want to go out and purchase the MetaQuest 3 right away. My HP Reverb G2 just does not give the immersion offered by hand tracking. Two questions...
1) Are you still as satisfied with it as you were in the demo video?
2)Would you get the 128GB or 512GB model of Quest 3?
Amazing. Which yoke are you using ? Thanks
will this works on other VR headsets such as HP reverb G2 on DCS?
Hi, is it possible to use it (with virtual desktop) with the usb cable on quest 3?
GOLLY your Microsoft flight simulator in VR looks very crisp did you connect with wires or is it wireless?
Link cable using Virtual Desktop
Does link cable studder/lag or is that just wireless?@@rbarlow
@@rbarlowhow can you use link cable with virtual desktop? I thought this only works via wlan?
Nope Virtual Desktop now supports the link cable connection too on Quest 3 at least not sure about Q2
@@rbarlow Do you have to change some settings in virtual desktop streamer? It always tells me, that it's waiting for wlan connection even with link cable active
I would kill for some kind of app that could use my phone’s camera to track my hands to allow for this same kind of feature to be used with my Varjo Aero.
Me too.
Trying to find a video on how to make this work do you have time , maby for a how to with quest 3
So Russ, now for the most important question: is the Quest 3 resolution high enough to allow the proper reading of all the instruments, especially the GPS fine text?
That’s what prevents me for biting the bullet for now, as the high resolution headset such as Varjo, Pimax Crystal are really over my budget. If the Quest 3 resolution is enough, the video pass through features could be the game changer I’m looking for.
Thanks
No it isn’t there yet. It is fun but I cannot read the kollsman window to set the altimeter as an example . It is very realistic and good for VMC flying but the Glass displays are not very usable. The $4000 Varjo XR-4 is what we need but we need it for a sub $2000 price tag. The Quest 3 is awesome for the price but the resolution is still not great for the finer details.
@@rbarlow Thank you for your reply Russ. That’s what I suspected. The idea being to fly A320/330 for training, VR will be the way to go when prices will drop or resolution improves
Thanks Russ for this Very interesting app allowing video passthrough. Is this also usable without VR ?
Happy Christmas days to you, And Thanky for all you do for the FS Community! :)
@RussBarlow do you know if this is working also with the Microsoft Store Version of FS2020?
Needs SteamVR to work
Id LOOOOVE to fly msfs 2020 in VR..
If they make a VR adaptation for the Series X or a cloud VR msfs 2020 Adaptation for the Meta Quest Ill buy the Quest 3 in 3 minutes..
Hello. Is this application works with cable link. I dont use virtual desktop..?
Great piece of software. Do you know if it will run under Vdxr soon?
Hi Russ, great video! Can you give details please as to how you got Reality mixer working in MSFS2020. I can start MSFS, and the passthrough works, but immediately I turn on VR mode in MSFS, the Reality Mixer boxes disappear.
Any help greatly appreciated.
I have the same problem
Hi Richard, I eventually solved this by changing the run time setting in Virtual Desktop to "Steam VR". The problem was then MSFS ran badly!
I already was using Steam VR, but i think i know what is going on. The boxes appear if i use proximity or glance options. My original desire to use a keyboard key to toggle them on or off appears to me to not work while in VR. I’m hoping the developer of Reality Mixer will respond to the Steam Community post I entered.
HI Ross , I am about to buy a Quest 3 , with the new features hand tracking pass through do I need to have the Knobester ? Thank you , Marco
No only the Quest 3, Virtual desktop for quest(not bought through Steam store), and SteamVR. Knobster is not related at all.
@@rbarlowdo you know if this works also with the Microsoft Store Version of FS2020? Would be very interesting for me. Thanks in advance, nice video
I use Virtual Desktop and Steam vr, but does not get any vr in my quest 3 when entering vr mode in msfs, works with airlink. The open xr is set to Steamvr. Any tips?
I don’t quite understand what you mean. Are you able to use Q3 normally with MSFS to fly in VR? Or do you mean you cannot get the chromaTrack overlay to work with MSFS in VR?
I am trying to play a pc game using virtual desktop so I have a larger screen. I want to have to boxes of where my small keyboard and mouse are so that I can see them, is this possible?
Yes if you can run the game in steamVR
The pass through of my quest 3 doesn’t look as good. Is there a setting that can be adjusted?
Lots of light makes the difference… direct flood light on panel also dim any LCD displays to less than 30-40% max brightness so the white doesn’t bloom
Excellent video once again!!!!!
Thanks great information 🎉🎉🎉
Now how can I have the hands with an erasable background, Consider it like a OBS style thing.
Like game overlay
I think the best solution is to have hand tracking manipulate the yoke and knobs within the sim
If you mean hand gestures in the air The tactile inputs are missing and trying to accurately turn a virtual dual concentric knob like those on a radio or the G530 is not easy. I tried InfuseVR by Yaw at Flight Sim Expo in Houston and they had a “blank” Cessna 172 panel with only the knobs and buttons and using the Quest 2 hand tracking it worked pretty well. Just have to build a 100% accurate model of the real panel so things are there when you reach for them and keep the panel and VR world perfectly aligned.
@@rbarlow copy, then the mixed reality would be the best solution
@@Redridge07I think unless some extremely lightweight gloves with haptic feedback could simulate the tactile feel of the physical world which isn’t impossible especially where we have come even on the last 10 years
It's what we need, and is a great idea...but to make the SteamVR app a requirement is a huge mistake. Most serious flyers are using Virtual Desktop and it's VDXR implementation of OpenXR and this is due to the fact the SteamVR offers terrible performance in MSFS as compared to VD and the XR Toolkit combo. There are now 3 different products like this, and they are all requiring SteamVR. What's really needed is an option to run these sorts of apps WITHOUT SteamVR.
Does Reality Mixer work with ILS2? I can't get Reality Mixer to come on with it when using Virtual Desktop and Quest 3. Is there a detailed video about how to make it work?
It should work but you need to do several things in the settings to set up the cameras(actually deactivate them) and instead enable the chroma key boxes. You also need to edit the boxes. In a week or two when I have things fully figured out I will try to do a tutorial step by step guide unless someone else has already done one. I explained the process is a bit complicated and required a few steps be done in a specific order (I think) . I know what works for me but there could be some flexibility but I am not sure.
Thanks very much Russ, I look forward to your step by step tutorial. Merry Christmas.@@rbarlow
wow that's really cool could this work with DCS?
Yes others say it first but depending on your computer SteanVR can lead to crappy performance
how do you display your hands tracking on video capture?
Capture on the Quest 3 headset and then download to PC via link cable
With headset on disconnect link and reconnect. When you get a quick notification that a usb device is connected click to accept and the quest storage will appear as a drive on your computer
The biggest downside of this (sadly) is that it requires you to use SteamVR which is not well optimized as using VDXR in MSFS.
Heck out my recent video on Color Panel
Indeed checked that, it's really great thanks for sharing this!
Somnium are soon to bring out a high end mixed reality headset.
No they're not. They're a scam.
Maybe but .. th-cam.com/video/W5KQEoD5ovY/w-d-xo.html@@jimmydesouza4375
StreamVR was the killer for me sadly... thanks for this vid tho, very cool
We have got to get rid of the red lines!
You could change the passthrough chroma color to yellow and have yellow lines😊
this is cool , I've been using Reality mixer for this exact concept already, except without the hand tracking
simply drawing the boxes over the peripherals i want to see completly like my keyboard, mfd etc ..the reality boxes activate when i move my hand close to my real device and disappear when i move my hands away
Is this still working for you? When i go into VR the boxes never show.
this looks really good. is it compatible with quest 2 or just quest 3?
Yes it should work with quest 2
Thanks I'll give it a shot!!
@@rbarlow
Would be terrible with Q2 since we dont have color passthrough cameras and only IR@@rbarlow
how does it track your hand and not the controllers? Regards,
Quest 2 experimented with hand tracking, and they have continued it through to the Quest 3 which uses the cameras on the headset to track your hands without the controllers.
Maybe the reason those RGBA settings aren't the same as the ones in the desktop is perhaps they are setup for ppl with different kinds of color blindness rather then trying to setup a color, but that's just a guess. I still have to get on this band wagon Desperately lol. Asked Santa for a quest 3 but he didn't hook it up LMFAO.
Does it for work HP reverb 2 headset?
Somewhat…see my video
POOR MAN'S MIXED REALITY Reality Mixer Demo
th-cam.com/video/FBkbTo_pSXg/w-d-xo.html
work It on dcs (Digital combat
Simulator)?
Yes confirmed by a another guy here’s his video
th-cam.com/users/shortsGqjJ0GOB1AM?si=McQJgb--qucrvtA_
can you use phone?
Portals are immersion killers, imho. Until we get great haptic gloves, I rather do muscle memory.
My understanding this will not work with fs2000 for Microsoft Store.
Watch to the end for a surprise
That is awesome ! Too bad it does only work with SteamVR ...
Check out my recent video for an alternative that doesn’t just SteamVR.
th-cam.com/video/o2vHWNf7pW4/w-d-xo.htmlsi=EoO4mF3l_lDx-xYm
@@rbarlow
Thank you Sir !
You say you find this doesn't break immersion much but from just looking at a video it seems much less immersive. Are you meaning comparing it to continuously taking your headset off or are you comparing it to the passhthrough sim cockpit setup you tried in an earlier video?
What I mean is that in your normal cockpit scan the passthrough areas are mostly in your peripheral vision and not very noticeable. Obviously it is less obtrusive than taking your headset off and on. I did try YawVR’s Cessna cockpit with InfuseVR which maps the hand tracking to a dummy cockpit with physical switches and knobs. It was pretty immersive visually and the hardware was there when I reached out to grab the controls. But it is not cheap to build and is limited to a single model airplane.
Super cool!
Portals are immersion killers, imho. Untill we get great haptic gloves, I rather do muscle memory.
I attempted this level of viewer immersion with green screen for 3 years straight, everyday and could not get it how I wanted it so I stopped making videos. This is awesome and inspiring Russ! 🎉
This is the holy grail for simmers who have a perfect replica cockpit...like a cesna or F16 replica.. amazing 🎉
@@channelvr1293 even just an alpha/bravo set up in say the kodiak for example. It's pretty damn close!!!
👍🤛🤛
SUPER!!
Immersion breaking
SteamVR: PUKE
Seriously, can we get past SteamVR and just go pure OpenXR at this point? SteamVR is just sub optimal when it comes to performance, ESPECIALLY in DCS.
I have been told OpenXR doesn’t currently support layered video needed for this. SteamVR is a vehicle that supports the video layer on top to nahe these things work. Hopefully OpenXR will adopt such technology. In the meantime take some Dramamine and try it😀
@@rbarlow SteamVR is a nogo for me. The overhead cost isn't worth it. I mainly do DCS, and VDXR or straight OpenXR is the way forward.
OpenXR not supporting layers is absolute bullshit, just devs being fucking lazy and not wanting to conform to the new standard.
so i guess no How to video then
Patience working on it. Wifey is making me take down the Christmas decorations.
@@rbarlow Yeah sorry to rush you lol :) I know Hollidays I was just Frustrated by the fact that I did not get it how to make this work and Usualy I do lol so yeah no worries If it was not of you I would not have Heard about this so Thanks again :)
Steam VR no thanks
Do you know if it works with MSFS from Microsoft, please?
Yes. You didn’t watch to the end.😀
Ahem, I did watch to the end and caught you saying the Steam version of MSFS worked [which surprised you], but my [perhaps unclear]question was focused on the downloaded version of MSFS from Microsoft, not the Steam version. Can you affirm it works that version, too? Thanks for your video and an agreeably swift reply to my question.@@rbarlow
Hm, I can get it working with acc and dcs. Unfortunately it doesn't work for me in msfs2020... 😮
Very cool!