It's exciting to see your tests with virtual production, Matt. This sort of thing would have cost a fortune 10 years ago. Imagine how cool it will be when you can get one of those big LED walls!
Matt this is AWESOME. You are getting close and I can see that inspiration you were feeling. Its getting close to the point where you can make something that looks pretty damn amazing with Virtual Production and UE.
This looks great! It makes green screens much more believable! S'il y a des français passant ici: la team JDG a très probablement utilisé ce tutoriel pour leur nouveau système de fonds verts ^^. C'est le même équipement, les mêmes logiciels, et le même niveau de rendu.
a good worklow would be to record composite and separate virtual camera+greenscreen, edit with the composite and re-compose after all editing is done. so, shooting and editing with excellent preview, while mainaining full compositing for output. this workflow would require rec/stop/save commands to be sent to unreal through sdi, and auto-naming of both unreal sequences and camera, to help with camera assistance.
LOL, your equipment is godlike compared to mine. I am using a Canon T5I and a T6S, but I am getting similar quality results. UE4 Production is the future man. I feel like a pioneer on a new frontier.
@@CinematographyDatabase gonna recap your previous vids and hop on those UE tutorials when I'm back from tour! Super excited to see what I can pull off. More excited to see what you can pull off 😁
Hey Matt, with green screen (not a video wall obviously) if you recorded the raw camera feed and tracking data you could easily tweak the UE scene after the fact and re-render. That way, if the lighting wasn't perfect, you missed something glaring in the background during the shoot, the key wasn't perfect or whatnot, you have way more flexibility in post. Doesn't stop you from also recording from the monitor.
@@CinematographyDatabase Well yeah, isn't that basically what they did on Avatar? But what I really meant is, you can still do all your minor post adjustments to the scene and rendering in UE while keeping everything else exactly as you had it in camera. Also, you'd get to throw everything UE has in it's bag of tricks at the final render by not necessarily doing it real time. I'm talking things you'd probably never notice on the monitor but will show up on a 55" or whatnot. Anyway, I'm sure you've already thought of this as well. Just got me thinking, when you said you like recording onto the monitor...
So I guess what I'm trying to say is, if you also capped the raw feed and tracking data and just did your UE programming in a smart enough way, it should be trivial to go back and fix mistakes you didn't catch on set.
so how does it work with post... Are you recording on the Ursa also, and then give the unreal engine and tracking data and raw footage to the editor for posting the feature film?
SUGGESTION: If you just get the LUT you want loaded onto the Blackmagic you can send that over the SDI, you would not have to add a grade in Unreal (saving you some processing power on that computer)
awesome as always, if you can do a vlog on tracking would be very good, I am so inspired like you, I am a screenwriter now learning to make movies virtually
An idea for lighting. Get a large TV screen, place it close to the model and send another feed of the virtual environment (taken from th eopposite direction of the camera). Tune the lighting can camera for low light so that the effect of the TV is more pronounced. Now the reflection of the light from the TV on your model makes them look more immursed. It should help to counteract all that green light pollution. It's all about proximity.
yeah we are working towards indie LED wall virtual production. I shot this demo last year, basically a mini Mando LED set - th-cam.com/video/kFBha5BE38k/w-d-xo.html
Question for you: Where did you get the greenscreen background? I've been searching on Amazon but couldn't find a decent one. I'm going to try a virtual setup like this (I'm an AI programmer, so want to do some realtime virtual characters with live actors interacting.
Hey, really enjoyed your video and in-depth and clear breakdown for a VP setup. I was wondering however, if you could share how to setup just Vive tracker + base stations without the HMD i don’t have an HMD and having an extremely hard time setting those up with unreal. I keep hearing they are super easy to configure but can’t find any easy setup videos for that. I was hoping if you could help explain that. Also tried following you along the configurations you showed in unreal here and can’t find most options you mentioned here 🙏🏽
@Cinematography Database where did you get that spaceship model from? I'd like to give it a try. The Epic virtual studio models are all very bright, I'd like to try something that looks good in dim light.
I was thinking of trying the new intel realsense tracking camera as a way of doing the tracking on the camera, rather than using the vive method. I've seen bigger production sets using inside-out camera tracking where they markup the room with tracking markers and then just have a camera pointed at the ceiling with a bit of software to position and orient the camera (which is basically what the intel tracking camera is doing, but doesn't use markers). Ideally I guess you'd have a rear projection screen behind the cam for environment lighting. Ah man, if only I had the funds I could play all day with this kind of thing.
Hi Matt! How do you output your Vlog video from Composure? Is there a straight forward way to output the composited image from Composure to a .mp4 file? Thanks!
How is the video signal coming to UE4, does UE work on the "raw" captured signal or does UE do fast compression? How good is the UE keyer, looks a bit limited. Woukd not be better to work with an external hardware keyer that does a key on a 444 signal then pass that to ue, or would that add to much latency ? Sorry for so many questions but i just started with this and i am interested to have as little as possible quality loss from lens to output.
is there a way to get this camera tracking but with just an unreal camera? composure + putting the composite plane in one place, wondering if this is possible? Love the work as always!
if you check Instagram we did some testing with backlights. I'm definitely going to do videos just on matching the live action to the virtual sets. That is the fun part.
@@CinematographyDatabase I've been looking into this quite a lot... would it be possible to use a few rear projectors to build a poor man's version of the ARWALL and lighting your actors that way?
Dear Matt; I have an exact setup like your early days. I need an answer if you please 🙏🏻. I am connected to unreal via decklink card and I get keyed preview in viewport. I use composure and also I use Media bundle projected on a plane. all are perfectly working. WHAT I CANT achieve is to record my editor viewport in the UE. Is it mandatory to use an ext recorder via decklink in order to record.? I DO NOT WANT OBS or alike as a screen capture is there a way to record in the Unreal engine like take recorder or seq rec.? APRECIATE your time and help.
Hi Matt! Hope you can help me. I have spend countless hours trying to get composure to work but something isn't right. The CG layer looks completely different compared to the actual scene. It is as if it doesn't read any of the post processing or lighting properly. Any advice on what I can do to fix this. Thanks
Hi Matt, love all your videos. Quick (newbie question) - how can I record the CG clean plate (or just get the tracking info) if I just want to use the live comp as an on set preview and do post later? I feel like there's a really simple answer for this, but I'm not sure what it is.
Gee Matt you’ve come a long way in a short time. That is unbelievably amazing. I would love to see this in person. Any plans for workshops once we are free from the pandemic? BTW I’m finally going to be able to get the RTX 2080 Ti so I can finally get to work with Cine tracer. Love your work man!!!
This video was the push i needed that made me want to learn virtual production! I can now send live video feed to a level but im struggling with two things. Is there a way to bind a hotkey that displays and hides an image or element relevant to what the actor is talking about in a live situation? Also, how can i record the final outcome of a live feed on PC? I dont have a ursa and SDI out :( Thanks!!!!
If I wanna do this live on Twitch or something, is there a way to do it without the live feed ending every time I try to play the level? I want to utilize some spider AI I coded, and their animations, but can't because this method only seems to work in the editor.
Hello Matt, thank you for the details on the video. I have a question. Right now I am shooting green screen on a much higher shutter speed than my frame rate requires for perfect motion blur, to be able to have a better keying in post. While live compositing is the way to go, what happens if I really want to add back the motion blur I sacrifice for the green screen? Match the shutter speeds of the real and virtual camera to match the green screen, record my scene and then add motion blur in post, or can it be done on the output comp from the engine? Thank you!
Hello, I'm working with Composure and the cg_layer color is way off, its bad and not as good quality as the CineActorCamera. Have you come across this issue where Composure's view is different from your CineCameraActor?
Is UE calculating the lighting in real time, or is the set "baked" lighting? Either way, looks like a fantastic start for a setup based on mostly "consumer" gear! You are really opening the door for a lot of productions!
Hi Matt, Looks great ! Can't wait to see how it connects with cine tracer. I have a request though. Could you please do a traditional newsroom setting that looks pro ?
I know it takes a little longer but would it increase quality to record the foreground and back ground sperate and then key in the background in post? I was even thinking that you could record the background on a 4k Blackmagic video assist so that it's all Blackmagic RAW.
This is definitely possible and you can record the CG background, matte, a 3D camera data all together for a traditional post finish. I primarily focus on the live workflow because it evolves into projectors and LED walls.
I'm working to move in the same direction so I'm glad you told me that before I setup my studio with the wrong workflow in mind. Love your work and excited to join in with Virtual Production.
Hi Matt. If I want to buy a video camera compatible with Unreal Engine to connect and shoot with chroma input directly after 3D creation What do you recommend to buy the type of camera compatible with the fifth version
Great videos love it!! Question for you though. I don't have an htc vive setup (yet.. covid $$ issues lol) so I'm trying to achieve similar results by using the iPhone as a tracker. Even though I match my cinecameraactor settings to my DSLR, i find they don't quite match in unreal. You don't seem to be having this issue with the vive trackers. what "information" do they give back.. Transform? world location? distance between cam and tracker? thx! Marc.
Hey man, this is amazing, thankyou very much. Just one question, once you have everything set up how do you record it? I mean, to have the render video of what you were filmed in real time. Thanks in advance!!
If you export tracking points from AE's lockdown plugin into Maya or blender you might be able to improve the lighting situation. Have each point drive a vertex on a 2d cutout in 3d space and use a 360 photo-sphere to light the mesh then re composite.
aescripts.com/lockdown/ , actually it looks like it'll generate a 3d mesh for you. ... i'd have killed for this a few years ago. PF track use to be the only thing that could do it... pretty sure this is just what you need
Thanks for sharing this. Been working with the aja and media bundles. They work but really want to use some of the more advanced keying features in composure. I understand how composure works but don't want the media plate to be attached to the camera . My question is how can I take the media plate in composure and add it to a plane that I place inside unreal like the black magic or aja bundle? Have read that 4.25 added some new features to composure and wondering if this would be one of them.
hahaha i asked him the same exact question. I found a tedious way to do it but its not live footage like his example here. Dont use composure to key the footage. Just use a material to key your footage then apply it on a plane
@@Jsfilmz That's what we have been doing with the aja bundle, read something about new composure plates. Aja's keyer isn't the best and would love to use the composure keyer. If you have a written link to the tedious way i'd love to check it out.
Question! Is it possible to monitor the composite real time, record the camera internal and UE separately, then do the composite later in an NLE? For example...Record everything, export the recorded UE sequence, bring it into Resolve, insert the green screen camera file and key in Resolve, match the timecode to overlay camera on the UE sequence? This will keep my Alexa file in the highest quality (Proress 4444), and will have greater control over the captured image for grading.
Yes. You can on top of that just record the camera tracking data and then add whatever you need in 3D later and render out the UE4 images at 4K etc. and do the comp later I’m whatever app you want.
You mention 2.0 base stations in the description, but would this work with 1.0 base stations as well? The htc website says the vivetracker works with the 1.0 base stations, but not sure about this specific use case.
I don’t know the specifics but I believe the 2.0 stations can be combined to make bigger tracked volumes easier than the 1.0. In the case of virtual production you want the best quality tracking volume as possible or the whole thing jitters. A perfectly setup space and perfect 1.0 setup would probably work, but most indie spaces have issues that are overcome with more stations, which means 2.0 stations.
Spending on it few days already, but haven't found info on how to place already keyed footage of a person in UE 3D environment to make some simple camera moves and render it out. All the tutorials are for live real time production with green. So frustrating....
At this current level ideally is a PC workstation with a modern high end CPU, 64GB+ RAM, and 2080ti. As this scales up, it requires more high end PC hardware that we will be covering.
I am trying to put foreground plate over media plate and added simple cube to foreground layer, it shows black background along with cube. ...already changed "alpha channel to linear space color only". Suggest if i am missing anything.
Is there a way to sync camera aperture value and shutter angle w/ background blur and motion? I imagine this is another limitation of the green screen vs an LED wall. Is there a way to get accurate dof based off of lens focal length, aperture values, and focus points? I can't imagine that kind of data is output by cameras and available to Unreal engine to calculate.
It is possible to sync the real world focus/iris/zoom/fov/chromatic aberrations/distortion of real world lenses and match them in Unreal Engine. LED walls have their own challenges with matching, but they are different than green screen for sure.
How are you setting up the tracker distance to the lens? Are you just estimating the distance? Because there's parallax between the camera position and the tracker position and it's not looking good atm for me
Really cool-I just ordered a Roko because all of this virtual production is too next level. Have you had any luck trying to get some sort of lightwrap going on for realtime keying?
Matt do you know if the BM Video assist can record BRAW from the Ursa mini 4.6k? not the pro but the first 4.6k hope you tech guy can answer that will be really really helpfull. thanks for your knowledge.
hola que tal, excelente el trabajo que haces y los totorales y todos tus videos, felicitaciones! queria consultarle lo siguiente: que tipo de hardware de pc (placa de video, procesador, ramón, etc) es necesario para realizar este tipo de trabajos? desde ya muchas gracias y espero tu respuesta
Did you run into any lens distortion issues? Also how accurate would you say is the measurement of real space to unreal space. Has it been 100% accurate for you?
R3ZOfficial currently I do no lens mapping or correction, which makes the tracking pretty indie/bad compared to a pro tracking solution. However we will be looking at different techniques for this both indie and high end.
Matt can you make a tutorial to a live stream with a chroma key, but with a camera movement like a last shots, when you moving the camera and the background moves too pleeeeaaaasssseeeee
yes, you put the video on a plane and then move the UE4 camera around. The illusion is pretty broken if you try to pan or tilt, but if the camera stays perpendicular it looks OK. Someone in the Facebook group is using this technique and fooled me at first into thinking it was a 3D track.
Hey Matt, I’ve been following along with your experiments and it’s very exciting to see your results. I have a question, as a Unity dev I’m not that familiar with Unreal but is there a way to add a trash matte to a wide shot in this setup? The talent would still be within the key but around that, possibly and rgb green layer to mask the rest of the studio? Thanks and keep up the great work!
It's exciting to see your tests with virtual production, Matt. This sort of thing would have cost a fortune 10 years ago. Imagine how cool it will be when you can get one of those big LED walls!
Matt this is AWESOME. You are getting close and I can see that inspiration you were feeling. Its getting close to the point where you can make something that looks pretty damn amazing with Virtual Production and UE.
Wow ... Tech has come a long way
Brilliant ... Now keep that AVR (Augmented Virtual Reality) as Ur standard background
👏👏👏
well done m8. always difficult to try something new when there is no youtube tutorials on the subject. looks great
Great result, Matt. Thank you for your videos!
This is looking great! So cool to see you spin this up so quickly.
OMG this is so insane!!! This changes cg so much
This looks great! It makes green screens much more believable!
S'il y a des français passant ici: la team JDG a très probablement utilisé ce tutoriel pour leur nouveau système de fonds verts ^^. C'est le même équipement, les mêmes logiciels, et le même niveau de rendu.
So awesome to follow your journey and see it all coming together now!
a good worklow would be to record composite and separate virtual camera+greenscreen, edit with the composite and re-compose after all editing is done. so, shooting and editing with excellent preview, while mainaining full compositing for output.
this workflow would require rec/stop/save commands to be sent to unreal through sdi, and auto-naming of both unreal sequences and camera, to help with camera assistance.
Bruh. Game changer. Even now.
So awesome!! Thank you...I really loved your explanation and demo of the composite:)
FANTASTIC! Thanks SO MUCH, MATT!! TRULY motivating!
Bro, thanks for sharing your rig, it's look really amazing!! Keep posting videos about it please!
LOL, your equipment is godlike compared to mine. I am using a Canon T5I and a T6S, but I am getting similar quality results. UE4 Production is the future man. I feel like a pioneer on a new frontier.
This is amazing. Thank you so much for sharing your process.
Did you have to track these? I saw you didnt have any tracking markerw on the greensreen but had camera movements.
I live track the camera using the HTC Vive and UE4
This is some next level stuff Matt! Would love to set up a rig like this at home for a static camera!
For static cameras it's really straight forward and really great quality. Adding camera moves makes it a bit more of a setup with the Vive etc.
@@CinematographyDatabase gonna recap your previous vids and hop on those UE tutorials when I'm back from tour! Super excited to see what I can pull off. More excited to see what you can pull off 😁
Nice you got the live key to get good results.
This is so dope! Thank you for making these videos!!
I need this in my life 👍🏼👍🏼👍🏼👍🏼👍🏼👍🏼. One person micro studio to create micro scifi short form story
Looks great. You are inspiring me to shot the short with your technics. Thx!
Ooh. Wow.
Maybe Ian Hubert wants to look at this! Because this is Soo cool!
HI Matt. You do an amazing job. I am trying to do the same with a Vive and a GH5. Could you please recommend some resources? Thanks.
Well done!! 👏🏽👏🏽👏🏽
Hey Matt, with green screen (not a video wall obviously) if you recorded the raw camera feed and tracking data you could easily tweak the UE scene after the fact and re-render. That way, if the lighting wasn't perfect, you missed something glaring in the background during the shoot, the key wasn't perfect or whatnot, you have way more flexibility in post. Doesn't stop you from also recording from the monitor.
Yup, a lot of people go this approach and just use the live comp as reference but then do a traditional post FX finish. I’m all about the live comp 😈
@@CinematographyDatabase Well yeah, isn't that basically what they did on Avatar? But what I really meant is, you can still do all your minor post adjustments to the scene and rendering in UE while keeping everything else exactly as you had it in camera. Also, you'd get to throw everything UE has in it's bag of tricks at the final render by not necessarily doing it real time. I'm talking things you'd probably never notice on the monitor but will show up on a 55" or whatnot. Anyway, I'm sure you've already thought of this as well. Just got me thinking, when you said you like recording onto the monitor...
So I guess what I'm trying to say is, if you also capped the raw feed and tracking data and just did your UE programming in a smart enough way, it should be trivial to go back and fix mistakes you didn't catch on set.
so how does it work with post... Are you recording on the Ursa also, and then give the unreal engine and tracking data and raw footage to the editor for posting the feature film?
You are awesome Matt!!
u can share the BP? i have htc camera tracker, but i cant get the same result as you in 9:15
I love your work so much !! Looking forward to a new enhance version of CineTracer with character blocking and staging system, mark their movement.
SUGGESTION: If you just get the LUT you want loaded onto the Blackmagic you can send that over the SDI, you would not have to add a grade in Unreal (saving you some processing power on that computer)
That’s a great idea. I should make a UE4 LUT 🍻
I’m a bit late but thanks Matt. You’ve earned a sub
awesome as always, if you can do a vlog on tracking would be very good, I am so inspired like you, I am a screenwriter now learning to make movies virtually
An idea for lighting. Get a large TV screen, place it close to the model and send another feed of the virtual environment (taken from th eopposite direction of the camera). Tune the lighting can camera for low light so that the effect of the TV is more pronounced. Now the reflection of the light from the TV on your model makes them look more immursed. It should help to counteract all that green light pollution. It's all about proximity.
yeah we are working towards indie LED wall virtual production. I shot this demo last year, basically a mini Mando LED set - th-cam.com/video/kFBha5BE38k/w-d-xo.html
interesting stuff. is there a lightwrap effect in the unreal compositor? that would help a lot.
Brilliant !
amazing bro, keep it up !!
Question for you: Where did you get the greenscreen background? I've been searching on Amazon but couldn't find a decent one. I'm going to try a virtual setup like this (I'm an AI programmer, so want to do some realtime virtual characters with live actors interacting.
Balls, so you even had it in the damn title. Sorry for the dumbass question :)
This is incredible
Hey, really enjoyed your video and in-depth and clear breakdown for a VP setup. I was wondering however, if you could share how to setup just Vive tracker + base stations without the HMD i don’t have an HMD and having an extremely hard time setting those up with unreal. I keep hearing they are super easy to configure but can’t find any easy setup videos for that. I was hoping if you could help explain that.
Also tried following you along the configurations you showed in unreal here and can’t find most options you mentioned here 🙏🏽
Really Cool
this is awesome
@Cinematography Database where did you get that spaceship model from? I'd like to give it a try. The Epic virtual studio models are all very bright, I'd like to try something that looks good in dim light.
easily one of the best on the marketplace - www.unrealengine.com/marketplace/en-US/product/sci-fi-modular-environment
I was thinking of trying the new intel realsense tracking camera as a way of doing the tracking on the camera, rather than using the vive method. I've seen bigger production sets using inside-out camera tracking where they markup the room with tracking markers and then just have a camera pointed at the ceiling with a bit of software to position and orient the camera (which is basically what the intel tracking camera is doing, but doesn't use markers). Ideally I guess you'd have a rear projection screen behind the cam for environment lighting.
Ah man, if only I had the funds I could play all day with this kind of thing.
Well done. And here we go!!! Off to HomeDepot to make that green screen frame. Big thanks to wify. :)
Thanks for sharing. Great job!
"Hit stop, give the media to the Downloader, who is me in my case, err"
lol amazing work!
Hi Matt! How do you output your Vlog video from Composure? Is there a straight forward way to output the composited image from Composure to a .mp4 file? Thanks!
How is the video signal coming to UE4, does UE work on the "raw" captured signal or does UE do fast compression? How good is the UE keyer, looks a bit limited. Woukd not be better to work with an external hardware keyer that does a key on a 444 signal then pass that to ue, or would that add to much latency ? Sorry for so many questions but i just started with this and i am interested to have as little as possible quality loss from lens to output.
is there a way to get this camera tracking but with just an unreal camera? composure + putting the composite plane in one place, wondering if this is possible? Love the work as always!
looks good, though a way to match the lighting in the scene better would be a big improvement.
if you check Instagram we did some testing with backlights. I'm definitely going to do videos just on matching the live action to the virtual sets. That is the fun part.
@@CinematographyDatabase I've been looking into this quite a lot... would it be possible to use a few rear projectors to build a poor man's version of the ARWALL and lighting your actors that way?
@@HardRockChart also looking into rear projection as a low budget solution :)
Louis Buys yes, this is possible and I will hopefully demo it.
@@CinematographyDatabase excited to see your demo
Dear Matt; I have an exact setup like your early days. I need an answer if you please 🙏🏻. I am connected to unreal via decklink card and I get keyed preview in viewport. I use composure and also I use Media bundle projected on a plane. all are perfectly working. WHAT I CANT achieve is to record my editor viewport in the UE. Is it mandatory to use an ext recorder via decklink in order to record.? I DO NOT WANT OBS or alike as a screen capture is there a way to record in the Unreal engine like take recorder or seq rec.? APRECIATE your time and help.
Hi Matt! Hope you can help me. I have spend countless hours trying to get composure to work but something isn't right. The CG layer looks completely different compared to the actual scene. It is as if it doesn't read any of the post processing or lighting properly. Any advice on what I can do to fix this. Thanks
Hi Matt, love all your videos. Quick (newbie question) - how can I record the CG clean plate (or just get the tracking info) if I just want to use the live comp as an on set preview and do post later? I feel like there's a really simple answer for this, but I'm not sure what it is.
Thank YOU!! Can you make the background out of focus and play with forcus in general? Best Marco
Yes you can, you need a way to track the real world camera focus distance however.
Gee Matt you’ve come a long way in a short time. That is unbelievably amazing. I would love to see this in person. Any plans for workshops once we are free from the pandemic? BTW I’m finally going to be able to get the RTX 2080 Ti so I can finally get to work with Cine tracer. Love your work man!!!
This video was the push i needed that made me want to learn virtual production! I can now send live video feed to a level but im struggling with two things. Is there a way to bind a hotkey that displays and hides an image or element relevant to what the actor is talking about in a live situation? Also, how can i record the final outcome of a live feed on PC? I dont have a ursa and SDI out :( Thanks!!!!
If I wanna do this live on Twitch or something, is there a way to do it without the live feed ending every time I try to play the level? I want to utilize some spider AI I coded, and their animations, but can't because this method only seems to work in the editor.
Hello Matt, thank you for the details on the video. I have a question. Right now I am shooting green screen on a much higher shutter speed than my frame rate requires for perfect motion blur, to be able to have a better keying in post. While live compositing is the way to go, what happens if I really want to add back the motion blur I sacrifice for the green screen? Match the shutter speeds of the real and virtual camera to match the green screen, record my scene and then add motion blur in post, or can it be done on the output comp from the engine? Thank you!
Hello, I'm working with Composure and the cg_layer color is way off, its bad and not as good quality as the CineActorCamera. Have you come across this issue where Composure's view is different from your CineCameraActor?
Is UE calculating the lighting in real time, or is the set "baked" lighting? Either way, looks like a fantastic start for a setup based on mostly "consumer" gear! You are really opening the door for a lot of productions!
This scene has baked lighting. However with RTX and SSGI fully dynamic lighting has very high quality and you only need to hit 24 FPS in this case.
Hi Matt, Looks great ! Can't wait to see how it connects with cine tracer. I have a request though. Could you please do a traditional newsroom setting that looks pro ?
I know it takes a little longer but would it increase quality to record the foreground and back ground sperate and then key in the background in post? I was even thinking that you could record the background on a 4k Blackmagic video assist so that it's all Blackmagic RAW.
This is definitely possible and you can record the CG background, matte, a 3D camera data all together for a traditional post finish. I primarily focus on the live workflow because it evolves into projectors and LED walls.
I'm working to move in the same direction so I'm glad you told me that before I setup my studio with the wrong workflow in mind. Love your work and excited to join in with Virtual Production.
Hi Matt. If I want to buy a video camera compatible with Unreal Engine to connect and shoot with chroma input directly after 3D creation
What do you recommend to buy the type of camera compatible with the fifth version
Great videos love it!! Question for you though. I don't have an htc vive setup (yet.. covid $$ issues lol) so I'm trying to achieve similar results by using the iPhone as a tracker. Even though I match my cinecameraactor settings to my DSLR, i find they don't quite match in unreal. You don't seem to be having this issue with the vive trackers. what "information" do they give back.. Transform? world location? distance between cam and tracker? thx!
Marc.
Hey man, this is amazing, thankyou very much. Just one question, once you have everything set up how do you record it? I mean, to have the render video of what you were filmed in real time. Thanks in advance!!
If you export tracking points from AE's lockdown plugin into Maya or blender you might be able to improve the lighting situation. Have each point drive a vertex on a 2d cutout in 3d space and use a 360 photo-sphere to light the mesh then re composite.
aescripts.com/lockdown/ , actually it looks like it'll generate a 3d mesh for you. ... i'd have killed for this a few years ago. PF track use to be the only thing that could do it... pretty sure this is just what you need
So this is awesome! But how do you think we can merge and sync the focus pulling between virtual and real cameras?
With high hardware/software systems (MoSys, NCam, Stype) it’s possible. At the indie level the community is looking/inventing solutions.
Please could you tell me or point to a link where i can create a material "Andy's Tutorial"?
compmaterial Do you know how to make this？8:30
Great Setup, i‘m gona try to build this in future, are you planning to connect your setup to cinedesigner ?
Anton Polinski if you mean Cine Tracer, yes.
Nice! Which graphics card was used?
Man wish there was a PDF somewhere on how to set this up for all cameras.
Is the vive tracking ready for broadcast production? Are vive making tiny random movement while tracking?
This is great but how can you record the video composed in unreal to record it for example?
Amazing thanks so much!
Thanks for sharing this. Been working with the aja and media bundles. They work but really want to use some of the more advanced keying features in composure. I understand how composure works but don't want the media plate to be attached to the camera . My question is how can I take the media plate in composure and add it to a plane that I place inside unreal like the black magic or aja bundle? Have read that 4.25 added some new features to composure and wondering if this would be one of them.
hahaha i asked him the same exact question. I found a tedious way to do it but its not live footage like his example here. Dont use composure to key the footage. Just use a material to key your footage then apply it on a plane
@@Jsfilmz That's what we have been doing with the aja bundle, read something about new composure plates. Aja's keyer isn't the best and would love to use the composure keyer. If you have a written link to the tedious way i'd love to check it out.
you use the Composite Plate / Image Plane plugin in UE4.25 to put the comp on a 2d plane to composite in world space.
Question! Is it possible to monitor the composite real time, record the camera internal and UE separately, then do the composite later in an NLE? For example...Record everything, export the recorded UE sequence, bring it into Resolve, insert the green screen camera file and key in Resolve, match the timecode to overlay camera on the UE sequence? This will keep my Alexa file in the highest quality (Proress 4444), and will have greater control over the captured image for grading.
Yes. You can on top of that just record the camera tracking data and then add whatever you need in 3D later and render out the UE4 images at 4K etc. and do the comp later I’m whatever app you want.
You mention 2.0 base stations in the description, but would this work with 1.0 base stations as well? The htc website says the vivetracker works with the 1.0 base stations, but not sure about this specific use case.
I don’t know the specifics but I believe the 2.0 stations can be combined to make bigger tracked volumes easier than the 1.0. In the case of virtual production you want the best quality tracking volume as possible or the whole thing jitters. A perfectly setup space and perfect 1.0 setup would probably work, but most indie spaces have issues that are overcome with more stations, which means 2.0 stations.
@@CinematographyDatabase thanks
Spending on it few days already, but haven't found info on how to place already keyed footage of a person in UE 3D environment to make some simple camera moves and render it out. All the tutorials are for live real time production with green. So frustrating....
Hi Great Video, Thx Which Computer configuration is necessary to run such setup ?
At this current level ideally is a PC workstation with a modern high end CPU, 64GB+ RAM, and 2080ti. As this scales up, it requires more high end PC hardware that we will be covering.
Brutal !!!!! Thanks you
I am trying to put foreground plate over media plate and added simple cube to foreground layer, it shows black background along with cube. ...already changed "alpha channel to linear space color only". Suggest if i am missing anything.
Man...this is genius
Is there a way to sync camera aperture value and shutter angle w/ background blur and motion? I imagine this is another limitation of the green screen vs an LED wall. Is there a way to get accurate dof based off of lens focal length, aperture values, and focus points? I can't imagine that kind of data is output by cameras and available to Unreal engine to calculate.
It is possible to sync the real world focus/iris/zoom/fov/chromatic aberrations/distortion of real world lenses and match them in Unreal Engine. LED walls have their own challenges with matching, but they are different than green screen for sure.
How are you setting up the tracker distance to the lens? Are you just estimating the distance? Because there's parallax between the camera position and the tracker position and it's not looking good atm for me
lens calibration isn't setup yet in this demo. In the future it will be.
pretty cool ! Amazing ! I am learning a little bit of Unreal, Can you use any digital camera, like for example a Gh4 ?
Really cool-I just ordered a Roko because all of this virtual production is too next level. Have you had any luck trying to get some sort of lightwrap going on for realtime keying?
I haven't tried light wrap yet, but I heard something about it. I'll show it if I get it up and running.
What’s the minimum size of a space required for virtual production led wall ??
Matt do you know if the BM Video assist can record BRAW from the Ursa mini 4.6k? not the pro but the first 4.6k hope you tech guy can answer that will be really really helpfull.
thanks for your knowledge.
hola que tal, excelente el trabajo que haces y los totorales y todos tus videos, felicitaciones!
queria consultarle lo siguiente: que tipo de hardware de pc (placa de video, procesador, ramón, etc) es necesario para realizar este tipo de trabajos? desde ya muchas gracias y espero tu respuesta
Did you run into any lens distortion issues?
Also how accurate would you say is the measurement of real space to unreal space. Has it been 100% accurate for you?
R3ZOfficial currently I do no lens mapping or correction, which makes the tracking pretty indie/bad compared to a pro tracking solution. However we will be looking at different techniques for this both indie and high end.
@@CinematographyDatabase very cool to hear! Everything I found so far requires programming knowledge or is outdated. Excited to see how you continue!
Matt can you make a tutorial to a live stream with a chroma key, but with a camera movement like a last shots, when you moving the camera and the background moves too pleeeeaaaasssseeeee
Nice!
how can we start a small virtual production . any simple DSLR tutorial Plz
This is fantastic. I would love to learn how to do this. So, to be clear this is UE4 Composure that you are using?
Is it possible to have a static physical camera (so no tracking info) into unreal engine and move a virtual camera around in unreal engine?
yes, you put the video on a plane and then move the UE4 camera around. The illusion is pretty broken if you try to pan or tilt, but if the camera stays perpendicular it looks OK. Someone in the Facebook group is using this technique and fooled me at first into thinking it was a 3D track.
Never got a reply but lets try once more.. I do not see color diff material option have any idea.? appreciate
Hey Matt, I’ve been following along with your experiments and it’s very exciting to see your results. I have a question, as a Unity dev I’m not that familiar with Unreal but is there a way to add a trash matte to a wide shot in this setup? The talent would still be within the key but around that, possibly and rgb green layer to mask the rest of the studio? Thanks and keep up the great work!
I haven’t done it yet, but you can add 3D garbage mattes for that effect yes.
@@CinematographyDatabase Thanks! Cheers!
Hi, if I use Take recorder for record project , how to setting ? Thank you so much