Unreal Engine Metahuman with Faceware Mark IV HMC, Xsens, Manus & GlassboxTech
ฝัง
- เผยแพร่เมื่อ 1 ม.ค. 2022
- If anyone has used Metahumans and facial motion, you understand it is challenging to capture dialogue. A short clip showing my Metahuman speaking in Greek.
This is one of the first few tests I did in Unreal Engine using the Faceware Mark IV HMC to record facial motion, with Xsens and MANUS™ to capture body and finger data, simultaneously. I used the Glassbox Technologies Live Client plugin to stream in the facial motion into Unreal from Faceware Studio.
All of this is being powered by a custom Puget Systems virtual production workstation coming equipped with the NVIDIA RTXA6000.
I am so grateful to be able to utilize all of this incredible technology in order to create in Unreal Engine. I will be releasing a series of tutorials very soon, walking you through my entire process from capturing facial motion with Metahumans to creating cinematics using a virtual camera tool, Dragonfly.
Metahumans from Epic Games and 3Lateral.
Thank you to Daniel Rodriguez Cadena for your incredible texture work on the female character body. You are a true gem!
Thank you to Bernhard Rieder a.k.a. Fatty Bull for helping me to create this gorgeous cinematic and light setup in Unreal.
And thank you Pixel Urge for modifying the body and teaching me how to modify the Metahuman face textures in Substance.
Original character by Michael Weisheim
Special thank you as always to everyone at Xsens (Katie Jo Turk), MANUS™ (Arsène van de Bilt), Faceware (Catarina M Rodrigues, Karen Chan (she/her), & Pete Busch), Glassbox Technologies (Norman Wang, Johannes Wilke, & Mariana Acuña Acosta) Puget Systems and NVIDIA Design and Visualization.
Link to my Discord: / discord
#unrealengine #virtualproduction #motioncapture #nvidiartx
#pugetsystems #epicgames #metahumans #mocap
#xsens #manusmeta #faceware #glassboxtech #metaverse - ภาพยนตร์และแอนิเมชัน
![ภาพนี้ก็ฮาเหมือนกันนะเนี้ย #2 SS8 [ พากย์นรก MEME.EXE ] | easy boy](http://i.ytimg.com/vi/RMhUgxxD49Q/mqdefault.jpg)
![[안방1열 풀캠4K] 베이비몬스터 'FOREVER' (BABYMONSTER FullCam)│@SBS Inkigayo 240707](http://i.ytimg.com/vi/_2HL-ogzzBU/mqdefault.jpg)
I remember thinking it was so revolutionary the first time I saw a game simply move a character's mouth when you would talk in voice chat. I know this requires more hardware, but it's cool to think our future little gamers will be talking and showing emotion all over some video game and they'll have no idea how big it really is.
Cool, ! my Project later today, I am taking a MetaHuman, creating Morph Targets for it (that correspond to the set that Oculus Lipsync SDK uses), then the head can automatically talk (lipsync) and smile etc, with only audio input and without camera setup
@@NextWorldVR will you make a video about it? Cause I would be willing to watch.
actually they won't know and won't care seeing these types of games and stuff will be made and coded by Thirds graders
Star Citizen already has the tech, known as FOIP (Face Over Internet Protocol). It's still quite janky but cool nonetheless.
Did you know you can link unreal engine with an app on your iphone and do this? At least the face part
This is so fascinating I can't even describe the feeling I'm having to even my self!
It's called errection , SIS boy !
@@meller13 no horny you grammer disasster!
Really cool.
Super inspiring, your unreal videos have been so helpful. It’s a wonderful time to be creative.
Very well put
Helpful?
The eyes look a little off when she turns to the side, but overall this is insanely impressive.
always love seeing your work. Seriously amazing.
Thank you so much, Kelly! All this is running seamlessly thanks to your team and you!
best setup I've seen so far - awesome work
I can only wonder what you could achieve when UE5 Metahumans will be mainstream. Your work is great anyways 😁
Everyone behind the scenes tryin to figure out how to use them easily.
Αυτή και αν ήταν έκπληξη!! Thank you for your great videos and keep up the great work!
👏..."Beautiful that's why I love the photorealistic graphics. And I wanted to know how far they will be able to reach with these visual technologies to perfection"...😍👍
for me, this fall right into my uncanny valley! other than that, everything is well lit! very cool
INCREDIBLE work!!
Fantastic job Gabby, that's awesome. Keep rocking!
Thank you!! And thank you for collaborating with me on the lighting and cinematics!!
This is amazing work. Just discovered ya on my timeline and wish ya a happy new year.
It's absolutely outstanding from a technical standpoint. But my God does it look like a robot using human skin as a mask sometimes
Thanks for your work ! It's inspire me to go futher with Unreal Engine. Nice video !
WOW , that looks amazing ! 👍
Thank you so much!
So smooth that helmet camera combo looks super light
Thank you, Jonathan!!
Unbelievable and mindboggling
Πολύ καλό, μπράβο!
Whaaat.. the expressions😳sooo mesmerising😍
Wow! So impressive!
Amazing, can't be more authentic
Τέλειο!!!
Συγχαρητήρια !!!
Beautiful!!!
Nice 🏆🏆🏆! Exceptionally impressive!
This face motion capture looks great and you speak pretty good Greek
Καλή δουλειά!
I love this, this is so cool
Nice work 👌
So great!
DUDE, THAT'S SO COOOOOL :D
Aww, thank you so much, Jags!
AWESOME!
Amazing!
Love it!
Both are beautiful.
Incredible.
Τυχαία μου το πέταξε ο αλγοριθμός. Χαίρομαι! instant follow!
*_Потрясающая техника!_*
Just subscribed this is awesome
Amazing
This is how video game controllers should be made. A helmet, suit, and gloves. Your avatar can be projected into the video game where people can see your facial expression, hand gestures, and body language. You can control your avatar using actual body movements (punching, kicking, aiming, swinging, running, etc.) rather than just pressing buttons and joysticks.
Rip tv screen....
Agreed I'd love to make that happen for everyone but I can't move fast enough solo.
so vr
soon very soon
You are asking for VR, which already exists, also, you still need a controller since you would need artificial locomotion and interfaces that are not virtual for UI and object interaction. Index controllers specifically at the peak of this.
I can't wait for the day when we won't need a head rig for this level of face capture, just the odd couple of sensors around the room like a VR setup.
Eventually you won't even need sensors. It'll just be built into your brain.
@@jreamer971 I mean why even have a brain at all?
Αυτό κι αν ήταν έκπληξη. Νομίζω ότι είναι το πρώτο metahuman παγκοσμίως που μίλησε για πρώτη φορά ελληνικά χαχα. Εντυπωσιακό :DDDDDDD. Καλημέρα από Αθήνα :)
ευχαριστώ πολύ!
@Braveheart what a silly question is this?.. She spoke in Greek because she is from Greece.
This is insane.
AWESOME))
The frame in front of the face reminds me of old machines in the 1950s. However, your work is fantastic.
Awesome
That's insane. Looms like we have our human replacements coming quick. I'm surprised the cyborgs haven't learned to create themselves and have no need for us
Imagine a machine doin carpentry lmao
Very cute, both.
So so strange. How the CG is easily believable. Yet, still, it's just not "alive". It doesn't breath. The uncanny valley is uncannily small now. No matter. Our perception of the real thing vs the avatar, driven by epochs of evolution, is beyond Master level.
Way past the uncanny valley! This is unreeeeaaaal
That's crazy how well the Meta's eyes follow hers.
We are so already in the Matrix. 🤣🤪
wow wow wow,... wow!!!
so smooth! Avatar vibes!!
Thank you!
👏👏👏👏 nice
Super cool. Imagine VR headset be able to do this.
Like a BOSS!
Thank you so much Jet!!!
Can't wait to learn C++ and start making cinematic videos for TH-cam with Unreal Engine with motion and face capture
Woooowwww, whhhoooaaaahhh
So good…
This is super cool Gabi!! She is giving you a Meta side eye!! Beautifully done!!
Aww, thank you so much, Solomon!
Super cool😂
Scary!!!!
thats so insane
Pretty cool tech. The CGI counterpart needs some catchlight in the eyes to give her more life.
nice )
Interesting. We now have realtime digital mask for videoconferencing.
That's a solid setup, lipsync is smooth as, great work, does glassbox perform better than facewares live link?
Thank you so much! I cant really say which one works better as for the time being I am using 4.26 for its stability. The Faceware Live Link works great in 4.27 though!
“Smooth as” are you kiwi or aus?
@@sonny5974 Englander
nice
wonderfullllllllllll
awesome haha
Ωραία τα Ελληνικά!!!!
Πολύ καλό Βίντεο !!!!
WOW
A peek into the metaverse
One step closer to never having to leave my room
Классно 👍
That's it!
Super cool! Would you consider a more detailed tech stack walkthrough?
Absolutely! Great idea!
For any industry animators and those behind the tech who read this. Make the change to the MetaHuman rig so that the lower jaw (with teeth) slouch beneath the lower lips when "Hi" happens. This is VERY noticeable! Also, scale the size of the teeth larger. They all make the skull inside the models head seem so small in scale to the head that outfits the face.
Ahaha terrifying. I love it.
Finally the online meetings will be cool
I can't wait until something like this is in my house, where I can put the suit on, the camera and be someone else whom I created for a game, or do what she is doing here in this video, im so jealous. I have never seen anything in 3d, or vr. Im sure this would blow my mind seeing for the very first time
I am completely ignorant on this subject, but I guess that a face closer to the actress's would make the simulation look more authentic and/or realistic.
The eyes chico! the eyes never lie!
Jeeeeeeez :)
Thank you so much!!❣
I worked with this same technology many years ago. It's ready now. The effort underway now is public acceptance...
When these are our avatars in the metaverse....shit gets real
Ah yes we have finally reached uncanny valley videogames, I cannot wait to get immersed
miko gon' be happy
I've only just come across your channel but you are doing the absolute best work I've ever seen on youtube in this area.. Can I ask, does the faceware solution provide greater detail than the ARkit 52 blendshapes? I understand metahumans control rig has more fidelity than ARkit but does faceware make use of it?
Thank you so much! You are spot on that the face control rig has more fidelity which is why I love it so much. What I have found to be the benefit of using Faceware is a few things with regards to quality, for one, you can use the multiplier in FW to create a profile for each characters face, then the motion logic blueprint in Unreal set up specific for the Metahumans also allows you to fine tune the expressions even more and add personality that would otherwise be impossible to do with UE Live link face. Lastly, it helps that you can stream in data captured at 60fps.
10/10... would wear opposing "hook" Velcro suit to infiltrate studio and attach myself to mocap actress.
I used to make Facerig 2D Avatars and it's hard to do with that as well heh. I can only imagine the intricacies of 3D models making it far more difficult.
Now *THAT'S* whats up!
"toso kala"
lol thats awesome!!
πολύ καλά indeed!
haha no way you speak Greek as well!!
@@FeedingWolves lol Nah I dont :D Google translate is my friend ;)
Hi! This looks amazing, did you do extra animation on top of the capture?
What are the gloves called? I would love to get some to make some game animations for fps hands!
Ohhh you should make videos pretending to be an alien from another planet. 😁 this is really awesome tech
i think there should be a way for the avatar to also move her tongue as well as lips and mouth, it could maybe be done just by analyzing the audio track or maybe with the camera but i think overall it would add a lot of realism to the speech
You can see the tongue moving 0:28 no idea how sophisticated it is but it isn’t ignored
Is there an ever so slight lag? And if so can it be eliminated in a later upload sync I wonder.
Greek language sounds beautifull !! peace