This AI deepfake is next level: Control expressions & motion
ฝัง
- เผยแพร่เมื่อ 23 ก.ค. 2024
- LivePortrait tutorial: Free, opensource AI deepfake tool for animating 1 photo. It can even handle tricky expressions & movements
#aitools #aivideo #deepfake #ainews #ai #agi #singularity
TurboType is a Chrome extension that helps you type faster with keyboard shortcuts. Try it today - they have a FREE forever plan.
www.turbotype.app/
Live Portrait:
liveportrait.github.io/
github.com/KwaiVGI/LivePortrait
Newsletter: aisearch.substack.com/
Find AI tools & jobs: ai-search.io/
Support: ko-fi.com/aisearch
Here's my equipment, in case you're wondering:
Dell Precision 5690 www.dell.com/en-us/dt/ai-tech...
GPU: Nvidia RTX 5000 Ada nvda.ws/3zfqGqS
Mouse/Keyboard: ALOGIC Echelon bit.ly/alogic-echelon
Mic: Shure SM7B amzn.to/3DErjt1
Audio interface: Scarlett Solo amzn.to/3qELMeu
0:00 Scope and examples
6:25 Installation
7:20 Git
8:18 Installation (continued)
9:19 Conda
12:12 Installation (continued)
15:37 Running LivePortrait - วิทยาศาสตร์และเทคโนโลยี
TurboType is a Chrome extension that helps you type faster with keyboard shortcuts. Try it today and start saving time. They have a free forever plan!
www.turbotype.app/
Thanks man, amazing video and I will try this hkey manager.
Lowkey one of the best AI youtubers
Free and open source? No such thing. If it was so, installing it wouldn't be needed. Double click and it would work.
@@laylasmart lmao nonsense
@@minhuang8848
Harvesting info is the goal today. For that Microsoft made Copilot AI and Recall. A spyware within the operating system.
How soon can I get a Hogwarts painting of my dead grandmother to tell me to wash my hands every hour?
Now, if you want
First thing I think of is a potential VR application. This would be in real time , your whole expression would be projected on your VR avatar.
It looks like it still takes some processing time to run the software, so at least currently you couldn't animate an avatar in real time on a local recording. So for V-tubers, at least a few months off. But if it's run by a cloud service, maybe they could make it work.
It's not new, Meta has already demo'd this type of realtime avatar . Zuck showed it off on Lex Friedman's podcast.
It's realtime and very high quality , just requires some pre-processing of a full face scan
@@iankrasnow5383 with the speed of tech innovations in the last few years, if they decide to work toward this it won't be that long
video calling
@@iankrasnow5383 A lot of time. Still not a consumer friendly implementation. Might take some more time to be realtime ready. Fingers crossed.
how do i create a file like exe to just open it up later? i could use it first time but when i closed the tap and cmd i wasnt able to open again...
You could create a .bat instead and run commands there.
Maybe auto-py-to-exe
Working on a cartoon about a mischeivous young girl named Yumi, I've been using AI since the beginning and finally found a way to make her consistently with apps that effectively use Character Reference techniques, I've even trained a model with my character. Being a creative partner with Pika, Leonardo AI, and finally Runway ML I am able to create a ton of content, but I will need to add the character animation, and while I actually turned my AI character into a fully rigged Metahuman, it's nice to know if and when I need a quick shot and don't have the time to set it up in Unreal, I can quicky generate my character in the scene, then I or my niece, who will do most of the performance stuff for Yumi, can act out and voice her and I can use that footage and audio to animate the clip. This is an amazing time to be in. As someone who uses AI as a tool, I can see the several use cases for stuff like this, and it's going to make life easier for me as I am a studio of one and I have zero budget to make most of my stuff. So using a combination of free tools and my natural resourcefulness I am starting to make head way. The one man film studio era is here now.
I am right alongside you brother!
can you give a short list as to how you would go for character consistency with free tools? i found it to be either lacking extremely (wasnt consistent) or its payed models i couldnt try.
example:
1. Ai x: its free and has no tokens, i use it to do this and that. then i can you this is step 2 for ..
2. AI x2: also free and has no tokens, now you can...
I subscribed to you when you had 2K subscribers, and today you have 150K. Bro, you totally deserve it, and your content is worth much more than 150K. Soon you will reach 1M. Love you bro, from India ❤
i normally dont mess with this stuff until there is an involved interface, super excited for this stuff to be working open source
you're right. For reasons I could not fathom, very few AI tool were pushed to the point where they get a user friendly client like Midjourney. Even programmer like me struggles to pull a repository from git and build it up myself, I don't think any reguar artist out there can do AI without at least a client side.
@@SVAFnemesis what, you don't like typing in Discord? heretic.
@@English_Lessons_Pre-Int_Interm can you please carefully read and understand my comment.
always appreciate you show every single steps to install. It's very helpful for person who is not familiar with any codes
my pleasure!
What happens if the source sticks out a tongue?
The entire internet crashes.
there are already artifacts with the teeth where it stays static as the hair is with the background, i'd imagine the same will happen with the tongue.
huuuge step up for open-source nonetheless!
Harambe is resurrected
doesnt work
What is bro planning to do
Amazing walkthrough! Thank you! I'm having an issue where the output video bobbles around and shakes a bit. I'm not seeing this in your examples. Any ideas on why or how a shake is happening on the output?
This turorial was great. Thanks. Question: Is there any tool (like this to run locally) to upload an mp3 voiceover and generate the mouth and eye movements to later use in this process? Thanks!
yes, is this what you're looking for? th-cam.com/video/rlnjcRP4oVc/w-d-xo.html
This is probably the best instructional video I have seen lately. Going thru the steps in great detail. Thank you for that.!! Once installed it runs smoothly.
You're very welcome!
were you able to run it on a group photo to animate multiple faces? i was unable to
@@abhishekpatwal8576 you can uncheck do crop and it does try but the result isnt there yet
Amazing demo! What are the potential security implications of using AI deepfake technology like Live Portrait? Are there measures in place to prevent misuse?
TurboType is great! was looking for something like that.
Btw, Your input files are very small. How long does it take to render? Can we change a 1080p video that is of 30seconds? the input would be around 100mb
Available in pinokio?
This is insanely fantastic for controlling expressions keep making your great videos.💯Top Notch.😁
thanks!
Create a source video from the movie "The Mask". When Jim's character becomes freaky with his eyes and mouth.
wow, the option to animate a face on a source video has a great potential. I can already see people creating scenes of people interacting with each other with Runway Gen-3 or another video generator and then editing the video so that the people in the scene actually talk!
We're one step closer to creating movie scenes.
exactly!
Am I missing something. I don't have the option to use a source video to animate as the input only an image option. Is that a different program than this?
@@user-cz9bl6jp8b I don't know. I never tried it. I was only commenting on what I saw in the video.
@@user-cz9bl6jp8b I'd like to know this too.
Thanks! It's absoluetly awesome as nodes in comfyUI!
oh, is there a node for this already?
@@theAIsearch Yup ^^
Wow, you always present great advances in AI, and open source, I am very grateful for your channel, I will try this soon
Thanks!
I got a problem during entering the line "conda activate LivePortrait". It returns "CondaError: Run 'conda init' before 'conda activate'. What shall I do?
type this : conda init
it will ask you to close the cmd
then restart and do the same instructions
after that type : conda activate LivePortrait
@@bandaniji9061 Thank you! It worked. 🤝
@@PINKALIMBA Good to know
I could fix the lip sync of the Teenage Mutant Ninja Turtles movie with this!
I can think of a hundred ways to use this creatively but i have only got 4GB VRAM.
@@eccentricballad9039 Use Google colab👀
You mean the original 1990 movie? Never noticed
The 1990 movie is ace! You can correct espeially the 3rd one! ;)
How long should it take? With a 3070 I'm at 1280 seconds of processing time. Wondering if it's actually processing or has stopped at this point.
It looks weird in motion, but if u pause the expressions at any point during its animation, it looks good and natural.
love your videos, keep up the good work :D
Thanks!
Wow, great stuff. This is the next step I was waiting for. All the puzzle pieces are falling together... moving characters and then making them talk or sing. It just all needs to come together in one platform to combine different features to create movies or music videos or virtual avatars. Thanks for sharing! EDIT: Can you make this work with 16:9 ratio images? I see a lot of lip-sync programs that are just square.
Great demo! Curious as to your CPU / GPU / ram configuration that you ran this on?
Thanks. RTX 5000 ada, 16g vram. cpu is intel i7, but i dont think that matters
Very thorough tutorial and a very good project to cover!
Thanks!
Great video! Just wondering how you do a video as source and a video as the driving, i only see image as the source option. If you can let us know. Thanks!
it will be released soon: github.com/KwaiVGI/LivePortrait/issues/27
i've been using this from beta on starting about 3 years ago. their current version is proprietary and does so much more and mimics lifelike in every way.
Any idea why I am getting this error, everything else worked up until this point:
C:\Users\funky\Desktop\liveportrait\LivePortrait>conda create -n LivePortrait python==3.9
'conda' is not recognized as an internal or external command,
operable program or batch file.
I have the same issue
I have a basic laptop without the NVidia graphic card, can I use this as well?
Why my windows search and my pc icon is not working after instaling the git ?
this happened to me before, you should restart and reinstall the git again without changing anything
This is going to be so great for video editing instead of having to animate facial animations for characters we could just use this software
yes!
what do you recommend for animation software
Can you make a video on your hardware? That setup you have looks cool
When i paste "conda activate LivePortrait" i get an error: Run 'conda init' before 'conda activate' any help would be much apr3ciated❤
EDIT: I just typed conda init and then followed the steps and all good
can u help me, when i paste "conda activate LivePortrait" its said "CondaError: Run 'conda init' before 'conda activate'...how to solve this?
solve hehe...
i try, conda init bash > then reopen > conda init > conda activate livepotrait > done
Man this seems insane. I love your findings. Will test it tonight!
Just one thing! If you could add the timer or how long it took to process that would be very much appreciated 🙏
Thanks. For a 10s video, it took maybe 1-2min to generate. Very quick compared to other tools
@@theAIsearch thank you very much! That’s waaay faster than I expected! 🤯
Hey thanks for all your help! I have a geforce 770 gtx, is that enough to run this ? I followed your tutorial and arrived to the point of installing miniconda, but when I run the code "conda create -n LivePortrait python==3.9.18" it just says that value is not recognized...even asking for the conda version with "conda --version" says unrecognized command, and I did add the path to environment variables 🙃
hmm, exit out of everything and try again. also, I'm not sure about the 770 gtx. if it has cuda, then it should be ok
I need to recant and say that everything worked out in the end. It is necessary to install all the components first, and only at the end of it all can you install the platform. Thanks for the video.
glad you got it to work!
which py version did you use ?
@@User-Mtw-j4n What I did was follow the tutorial in this video and after four or five failed attempts it ended up working.
thank you for stepping us through the miniconda installation. Is it not possible to install this using virtual environments via python itself ? what is conda doing for this install that cant be done via a regular venv setup ?
Thanks,
b
it's likely possible to use venv. i just followed the install instructions in the repo, which used conda
@@theAIsearch Thanks for the clarification. Cheers
I get stuck installing miniconda. Apparently it's a known issue where it doesn't link some dependencies and therefore I am unable to even set up live portrait :( why couldn't they just have a regular installer instead of having to do it through command line
still not mapping much of the eye expression, not sure if its a setting or something but when the source goes cross-eyed its not conveyed at all in the result, and there expressions are much more muted than the source, hopefully this is configurable
Very cool, thank you for sharing ❤
Too bad it won't work on VIDEOS or Multiple Faces like in their examples.
Does it only generate animations? Can I change actions on static images
Does it run from the cloud, or will this run offline? Is there a way to make a quick launch package to simplify it for the user?
Thanks to your very detailed patient step by step instruction, I was able to generate my own live portraits.
My results are not as perfect as the examples, but amazing nonetheless. Thank you! Thank you! Thank you!
Just a question.. if the source video is already talking in the video and then if you put Driving video full of talk and expressions as well.. then what would be the outcome..??
Will it mask the driving video talk on a source video talk??
and thanks for the video and information..👍👍👍
good question. they haven't released the video feature yet, but thats a good thing to test out
Thanks!! Now video-2-video is supported. Can you update this guide please? Its better to made a clean install or just re-download the entire repository??
At the stage where you show how to start the application and get the interface link, I get a "port error".
_ERROR: [Errno 10048] error while attempting to bind on address ('0.0.0.0', 8890): only one usage of each socket address (protocol/network address/port) is normally permitted_
Searching on this error gets tech talk about how "the port is occupied" and one should "open another port", but that is beyond my skill level.
Any help?
Rebooted my system and ran the procedure to start LivePortrait the first thing after entering win, and that did the trick. I wonder if I need to be mindful about what is going on in regards to internet connections whenever starting this program?
YOU ARE REALLY GIVING ME ALL THESE THINGS whenever I NEEDED THIS TO Make my animation!!!!
Nice can I see your animation when you're finished?
Good luck!
@@theAIsearch how to run on amd gpu
and what a bout the aspect ratio crop thing? thanks
First step of git clone doesn't work for me. I get the message "git not recognized as command, program or file. I've installed git and anaconda. How can I get past this? Any suggestions out there?
You probably need to add environment variables to your Path e.g. c:\Program Files\Git\cmd
Absolutely incredible!
yes!
That's good, but do you have any tools like this to animate hands and the rest of the body ?
Part of the webui is a file (frpc_windows_amd64_v0.2) which is a reverse proxy utility. Looks extremely untrustworthy to me. Running under a virtual environment mitigates some of the risk but I'm still skeptical. You should really be running this in an extremely sandboxed operating system.
China + open source + free = I'm gonna steal your life now.
@@kirtisozgur yeah, pretty much
Thanks for sharing!
I also noticed when you add a video it says "uploading video" which seemed a little sus for a local install.
Never trust a Chinese company that wants your data
We really need more post from you, we are bored without you
thanks
Dell and Nvidia huh? Were one of those your Chinese friends that gave you the code :D
how do you do the sample you showed of video transposed to a video source (4:17)? Thanks!
they will release it soon github.com/KwaiVGI/LivePortrait/issues/27
@@theAIsearch cool. thanks for the reply! :D
Bro, good continuation would be to explain how to use it - vid to vid. Can you explain this? I saw examples when guys making vid to vid using liveportait and this is awesome
what's the maaximum length of ai video you generate with this?
were you able to run it on a group photo to animate multiple faces? i was unable to
This is gonna be a nightmare soon enough ...
Yea - one of my favourite (and somewhat fringe) concepts is that "Everything comes true in the end" - because the context around it changes.
My fav example of this is "primitive tribes in 1970s National Geographic Magazine" being afraid of cameras because they thought they could steal your soul.
Well.
Here we are. We are within days of there being a browser extension that with a single click can superimpose any photo into any porn video.... to take any youtube video of anyone and turn it into a kind of voodoo doll or gollum than can be made to perform any action imaginable, including ringing up your family, friends, enemies and doing such a perfect impersonation of you that it is actually more realistic than you are yourself.
In a way, the Algos already have voodoo dolls of you... a lifetime of clicks and comments etc, rows in a database tied by a single user_id.
I think "Text" was a massive massive revolution in what it meant to be human because it collapsed the time dimension. Memories became something that took zero energy to maintain. I think AI is a process towards collapsing some other dimension, although I've yet to figure out what it is, and I might of course be talking bollocks.
@@nicktaylor5264 i want what youre having
@@nicktaylor5264 Beautifully written, I too need my overlord my one true leader, a god not to worship but to follow, the basilisk, one of our own making.
@@nicktaylor5264☠️☠️☠️
Sure, if you have no idea about tech. This AI is amazing and it's only gonna get better.
I just want to say you are the best, thx a lot dude
When dabbling in Stable diffusion (in Automatic1111) I've been struggling to find intuitive solutions for transferring facial expressions onto image output. You can prompt for it, or you can use Adetailer, or you can go into control nets and some other solutions. But all of them are fiddly and deals with "numbers" and "sliders" and "settings" and so on.
If they can do what we see here, why not also in image generation, where you would simply put in a reference photo and have the expression affect the output? Perhaps I am missing something obvious regarding img2img and certain workflows. I am a dabbler after all.
Great stuff!
How long does it take to generate a 5 sec animation?
it depends on your gpu, but for me, less than 5min
I am trying to install on mac and can't,
ERROR: Could not find a version that satisfies the requirement onnxruntime-gpu==1.18.0 (from versions: none)
ERROR: No matching distribution found for onnxruntime-gpu==1.18.0
Can anyone help?
dont know if theres mac support yet. you can use this instead huggingface.co/spaces/KwaiVGI/LivePortrait
The examples you showed are for still images, how to generate for videos?
+
they haven't released that part yet. i'll keep you posted
How about using Videos instead of Images as a source file. Just like the samples you show, please show us how we can do that as well, thanks. Anyway, I have successfully installed this in my computer using phyton only env. And u r right, it generate so fast unlike any other video generation like hallo which I have installed as well.
glad you got it to work. they will release the video feature soon github.com/KwaiVGI/LivePortrait/issues/27
It's works very well when the source photo and input video are at the same angle but there is obvious warping when the angle is different. Best to keep the angles the same.
Thanks for sharing
Exciting functionality! I’m gonna do a deep scan with some security tools like wireshark n such. I suggest y’all do the same just in case. There are quite a few repos with zero day backdoors that specifically target windows. Happy prompting y’all
Incredible! Thanks for sharing!
no problem
How long can the videos be?
thats crazy! i wanted to create a yt channel for so long but didnt want to use my own voice nor face. i can do it now :)
That's a great idea Good luck to you and your channel! Did you need a Chinese phone number to run this app?
@monday304 LivePortrait doesn't require any number
That's it, I'm never ever gonna believe what I see online or on digital media anymore.
i use amd radeon graphics and seem this application cannot work in my device, please help
so how much disk space does it take up total?
Thank you. Excellent Video, very detailed and well explained. New subscriber
Thanks!
Amazing feature 🤩Greatly appreciate doing this super simple video guide on how to use this tool. Game changer! Thanks so muchhhh
enjoy!
I don't see a way to do video source and driver. was that only something they talked about in the paper?
It says they will publish it soon: github.com/KwaiVGI/LivePortrait/issues/27
After the conda activate Liveportrait step. I get the following error:
ERROR: Could not install packages due to an OSError: [WinError 206] The filename or extension is too long: 'C:\\Users\\Dave\\Desktop\
oop-unleashed-main\
oop-unleashed-main\\installer\\installer_files\\conda\\envs\\LivePortrait\\Lib\\site-packages\\onnx\\backend\\test\\data\
ode\\test_averagepool_3d_dilations_large_count_include_pad_is_0_ceil_mode_is_False\\test_data_set_0'
I assume this is due to another program (roop-unleashed), any way of sorting this? Both folders (roop-unleashed and Liveportrait) are saved on the desktop.
I tried to upload a video instead of a picture, but it doesn't allow any extension other than a photo extension. But you showed in your video that it is also possible to add a video 🤔
I hope someone else will show us how to do it. I want to know how to do that as well.
they will release the video feature 'in a few days' github.com/KwaiVGI/LivePortrait/issues/27
How did you do the video example i dont find an option to put source video
They haven't released it yet. Will keep you posted
@@theAIsearchThanks! Where do they communicate the updates? On Github?
@@theAIsearch Thanks man tried the pic came out well if u edit it in a cool way just animate it after lol
Thanks, it works fine to animate a source image. But in the video you also tell that you can animate the face in a SOURCE VIDEO. How is that happening? Can anyone animate the face in a source video? How to use the tool for that?
I am looking at the Huggingface Space. I don't see the video to video option. I don't see it in your video either. Is the video to video available for anyone yet?
not yet. they mentioned 'in a few days'
@@theAIsearch Thanks!
What is the best way to upscale the output if it can't be bigger than 512x512?
Topaz but it's not free😮
insane use cases are coming
At 3:58 you start talking about how you can use live portrait on not just stationary images, but moving videos too. Then in none of the examples at the end did you show how to use it on videos. I have tried uploading video but they are not a supported format.
What is going on?
Hello, This is great. Unfortunately I get an error where is said...To create a public link, set `share=True` in `launch()`.. Can someone help me with a fix? Thanks
try going to 127.0.0.1:7860
can i run this locally with intel arc a750 gpu?
I'm really interested in running this type of AI model locally on my own system. To do that, I would need to purchase a dedicated graphics card. Could you please suggest which one would be the better option for my needs - the NVIDIA RTX 3060, RTX 3060 Ti, or the AMD RX 6700 XT? I'd appreciate your recommendation. Thank you in advance for your help.
unfortunately, none out of those 3. 3060ti is ok, but it has only 12G vram. it's best to have at least 16G vram
'conda' is not recognized as an internal or external command, operable program or batch file.'' i just tried to add the live portrait folder where i have the rest of my AI stuff like automatic1111, forge ect. ive already installed conda. why am i gettin this error?
you need to add conda to path. see 11:10
@@theAIsearch Sir i did all of that step by step and still get that error
@@theAIsearch when i installed other ai that requires python and or conda, i never had to do it more than once why do i have to do all this over again when i already have conda and python installed . i tried the different path thing and i still get that error sayin conda isnt recognized
I'm sad to say that I'm stuck on installing (@ 12:43) Im gettin ERROR [CondaError: Run 'conda init' before 'conda activate'] Anyone an idea how to get this fixed? When typing conda init LivePortrait I get an error that i have an invalid choice of shell. (YEP I'm a n00b and had to install Git and Miniconda)
The cat chopping chicken was awesome!
This is awesome. AI is really advancing fast.
Thanks! I love your video. I followed the steps and installed the code and packages. I tried running the code it doesn't work, I got this error: "RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from..."
How come it only NVIDIA GPU it works on? Is there any alternative?
You can maybe tweak it for CPU: github.com/KwaiVGI/LivePortrait/issues/41
Or, try this online version: huggingface.co/spaces/KwaiVGI/LivePortrait
when i use it agin its procedure??
Does this app require online connection while processing?
Is there any way make it work offline or is there an offline version?
no internet needed
What is the output resolution of these videos?
For the few I tried, It is based on the size of the image you use as an input.
Very cool. Imagine watching a foreign movie and the lips move like the ones from the translation. Or a video game sequence