วีดีโอ

Great thanks! I'm happy to hear you like it ^^ I've never used CC4 personally, but afaik CC4 uses a similar skeleton hierarchy and morph targets as UE MetaHuman. My plugin helps you create accurate depth maps from your video, which can then be turned into depth footage and processed through the UE MetaHuman pipeline. So, if you're able to use any MetaHuman animation with CC4, it means you should be able to use the performance output/result with CC4 as well.

@@johnnien2 you are right 👍 if you want to rig the character to a metahuman skeleton then yes, you have more things to do 😄

@@RobertKleinSk Can you jump into our discord server let's check, it would be nice if you can share error details from Output Log

@@xlipdev LogScript: Error: Script Msg: Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Program Files/Epic Games/UE_5.5/Engine/Plugins/Marketplace/FDFMancer/Content/Python\video_info.py", line 2, in <module> import imageio_ffmpeg as ffmpeg ModuleNotFoundError: No module named 'imageio_ffmpeg' LogScript: Error: Script call stack: /FDFMancer/ActionUtilities/EUB-FileMediaSourceUtilityAction.EUB-FileMediaSourceUtilityAction_C.ExtractVideoFramesWithManager <--- /FDFMancer/ActionUtilities/EUB-FileMediaSourceUtilityAction.EUB-FileMediaSourceUtilityAction_C.Extract Video Frames

yes I will join to discord

I'm happy we fixed that ^^ Let me know if you encounter any other issue

@@xlipdev this was absolutely instant support 😀I appreciate it! Thank you!!

Very good point! And yes! I actually shared a quick example on our Discord server that you can check out. I'll share a tutorial on it later! The results look pretty nice, but keep in mind that during MH Identity creation, you need natural poses of the actor (preferably front-facing and slightly to the left and right, without any facial expression). If your video provides these frames, you can achieve decent performance outputs!

@@xlipdev thanks, looking forward to your tutorials

Great thanks! Happy to hear you like it ^^

This repository requires coding experience if you are not sure what you are doing with your python env and your system of course expecting a magical outcome doesn't make much sense, here so many people have achieved very good results with this, you could ask questions in our Discord channel I’d be happy to help, as I have for many others, even maybe checking channel messages would help. Instead, you chose to express your frustration with this comment 😅

@@xlipdev I made this possible free and without your repo, I cant share with you, but got something way better and easier, costs around 5 minuets

@@NoishiXzenTM That's great I'm happy for you ^^

🤔 what is that "zen" error? you can try deleting Zen/Install folder (In AppData/Local i showed here in the vid) unreal engine will try to create these exe files again and maybe resolves your issue

@@xlipdev I meant to say that in my case, the crash report diagnostic is only about the zen folder. If I delete it, it works, and I can launch Unreal without crashing. But when I reboot my computer, UE crashes again and I have to delete the zen data folder again. I'm still investigating because I noticed no crash when executing UE with admin-rights, but doing that, I can't drag'n'drop files into my content browser or import animations into my project.

You should rely on Unreal Engine's animation pipeline to create animations with the Unreal mannequin skeleton, as it provides access to control rigs, blend shapes, and more, making the process much easier. However, some tutorials still use Rigify and import Blender-created animations into Unreal. Keep in mind, though, that the bone/joint systems in these two software are different, which makes the process quite challenging 🥲

@@xlipdev thanks, I already tried that, unfortunatly i cannot mirrorize animations in that way, can I?

​@@nazariobiscotti8011 Nope, animation keyframes cannot be mirrored as I did for the skeleton in this video I believe. However, if you have a character rigged to the UE5 skeleton, you can use animation retargeting and various Unreal animation features, which might be more helpful than trying to import animations created in Blender

Thanks to you for your kind words ^^ I actually started developing the plugin after your comment and didn’t expect this much attention tbh 😆There are many apps and AIs out there for creating body animations, and I’ve seen some great results! Currently, I’m not planning to dive into it, but let’s see how it goes. Thanks again for the appreciation it means a lot!

You are very welcome ^^

You were the first one who comments ^^ ❤

@xlipdev Love your work 😊

You’re very welcome! ^^ Yeah, facial animation is key to bringing a character to life, and that’s exactly the idea! Any camera will work with any resolution!

Have you ever checked other facial capture related software or equipment prices...

Bro no one forces you to buy it, why are you so triggered with this

Thank you so much! From what I’ve seen, the Rokoko Facial Head Rig doesn’t quite match the performance capabilities of Unreal’s MetaHuman system. However, I’ve heard great feedback from users who utilize a head rig with this plugin and are satisfied with the results. Just to clarify, the plugin generates depth frames that can be used with the MetaHuman Animator in Unreal, so even raw video from an iPhone can be utilized effectively. That said, using an HMC (Head-Mounted Camera) makes the process much smoother and more efficient ^^

Yea it was there ^^ I use my mcbook to make sure the window in my room isn’t opened all the way. So, there are definitely some good use cases for Apple products 😅

I will make another comparison video for sure ^^

Thanks a lot!

It is learning ^^

Me too 😆

Thanks for the kind words ^^ I'll keep going 😆

Great thanks ^^

@@inteligenciafutura The editor plugin I showed in this video is a paid version, but I will update the repository I shared earlier once the release is out 👍

Thank you! It's good to hear that ^^

Great to hear you like it ^^ I wasn't too happy about the last comparison video with the iPhone😆 Very interesting question! Honestly, Frame Mancer might not be the best for creating depth maps for non-face videos, as it heavily relies on facial landmarks. I believe there are many other apps specifically designed for generating displacement maps that would do a better job ^^ But good idea! I will definitely check..

@@xlipdev Ok. Cool. The reason I bring this up is how you are able to generate the depth frames locally within unreal. Could there be a way to use an AI model (Depth Anythingv2 for example) to then batch create depth frame sequence using your setup? Anyways just spitballing, might worth a try ... 🙂

​@@damiakinbode Actually, that's a very nice idea! I also initially considered using an AI model, but most of them are licensed (I mean the models specifically trained with faces; other models don't give very good output) and cannot be used in commercial applications. However, it is possible to get a trainable model and train it with your own data locally! I'm still searching for some AI models that provide better results, but let's see how it goes.

😆 No real calibration needed btw! You just tweak the focal length adjustment, and the plugin takes care of the rest ^^ My MacBook already did its job for everyone 😂

Currently, extracted frames are scaled to 720p during extraction to align with the default calibration data for plugin processing. However, the new version will expose all settings, including FPS and aspect ratio, allowing you to customize resolution and other parameters for extraction and processing while creating custom calibration assets with the plugin ^^

Great 👍🏻👍🏻

Yes, but currently, extracted frames are scaled to 720p during extraction to match the default calibration data for plugin processing. However, in the new version, all settings will be exposed, enabling you to specify any aspect ratio (resolution) for extraction and processing as you create custom calibration assets with the plugin 👍

​@@xlipdevcool 👍🏻👍🏻👍🏻

​@@xlipdevbut what about the initial calibration process? The front view, side views and teeth view. How can that be done with a custom video clip ???

​@@malikgillani Initially, you’ll set your video and depth frame resolution/fps etc. before extraction, (there’s no longer a restriction for the 9-16 aspect ratio anymore!) Before creating the Footage Capture Data asset, you’ll need a Camera Calibration asset. With the plugin, you simply adjust the focal length, and it will automatically create and assign a Camera Calibration asset to your footage. From there, you can proceed with the regular MetaHuman pipeline for identity creation etc.

@@xlipdev please do make a tutorial video for this too. Thank you for responding 😇

Yes, in the new version, options for video/depth frame resolutions, FPS, and all other calibration data settings will be fully available👍

Hey, are you sure the exported file contains only one armature? Unreal might get confused if there are multiple armatures and won’t know which one to use for creating a skeletal mesh 🤔

@@xlipdev Thank you for your nice reply! i will checkt it again, it seem like that will be very offen happened, which the "root" name of skeleton in Blender was changed regularly.

Good point! I believe the reason was that the 2025.1.7 version of the plugin still uses a default camera calibration asset retrieved from an iPhone (which has lens distortions), so it wasn’t matching 100% with the footage which causes that. I can happily say that it’s not happening anymore and won’t be there in the new version ^^ Additionally, when I applied these animations to a MetaHuman, I didn’t see these glitches🤔

@@xlipdev not a problem brother good to know. I appreciate the honesty. That means a lot and I hope to continue to see this plugin grow and develop further. :)

@@deygus Great thanks for the kind words and support man ^^ Sure I'll make it better and better..

Thanks! Good to hear it is not boring😆

@@AecelotGrimmWyrdStudios Great to hear you like it ^^ Right now, I'm dedicating all my time to developing this plugin. Once I'm satisfied with the results and release the new version(s), I plan to continue with game dev 👍

​@@xlipdev Let me know when you do! You seem like you know what you're doing and we're always looking for collaborations!

Thanks ^^ you need to be in face mesh directory before running that script. Simply go with 'cd ./face_mesh' first and then try running

@@xlipdev im sry if this is obv but im really new to this stuff but i did this cd C:\Users\Any\OneDrive\Desktop\faceDepthAI-master\faceDepthAI-master\face_mesh> then i ran the command python create_single_sample_and_display.py still the same issue

@@One1ye19 No worries, but yea this repo requires some knowledge about coding and development, you can jump here discord.gg/r4xj4hsk I will try to help ^^

@@itsMBWAAA Great to hear you like it ^^

Could you try lowering the values for "Parallel Process Count" and "Batch Size"? If that doesn't resolve the issue, join the Discord channel so we can take a closer look together.

You're not providing much detail, my friend 😅. There could be many things that went wrong. Start by checking if the RGB and depth frames exist in the folders using a file explorer. Then, review your image sequences to ensure they open correctly, have the right dimensions, and display the expected FPS values. Finally, inspect your footage thoroughly.

Very kind of you ^^ Great to hear it helps 🤗There's more exciting stuff on the way!

Great thanks for support 🤗I hope you enjoy it ^^ This is just the beginning, and it will get even better ^^

Tbh I didn’t expect that much attention😅I’ve never been great at naming things 😄Android Face Link hmm I was thinking about an android app for real time tracking later on and actually not a bad name, let's see🤗

That's cool! I actually started with MiDaS too, but since the models aren’t trained for faces, I couldn’t get accurate depth maps. I ended up finding other models specifically trained on faces that produced much better high-quality maps. However, each frame is generated individually, which makes it incompatible with the entire sequence when using AI. And yeah, I also noticed that depth frames don’t significantly affect facial features in most cases, so I decided to stick with my current implementation. That said, there are still scenarios where they can have an impact, mainly due to camera calibration (like head distance, rotation, etc.). From my testing, it seems impossible to process the MH performance pipeline with empty or solid depth maps, it just crashes without providing any clear info. I guess the Epic team wants to keep the magic under wraps for now! 😆 Or maybe I just haven’t dug deep enough or found the right sources to investigate further.

@@LestZA That's great to hear it helped 🤗I lost many projects already so decided to make a quick video 😁

@@PowerfullUs Thanks a lot! ^^ I hope you enjoy it ^^ Round 2 is gonna be completely different 😎

@@OverJumpRally Thanks a lot! Hope you enjoy it ^^ By the way, I’ve surpassed iPhone performance internally 😆 I’m working on the new version, so keep eye on it, I’ll be sharing the details soon ^^

@@massinissa8697 Massive thanks for the support man 🥹 I'm almost there I will share the details very soon ^^