Samsung’s AI: Megapixel DeepFakes! 📷
ฝัง
- เผยแพร่เมื่อ 12 ส.ค. 2022
- ❤️ Check out Lambda here and sign up for their GPU Cloud: lambdalabs.com/papers
📝 The paper "MegaPortraits: One-shot Megapixel Neural Head Avatars" is available here:
samsunglabs.github.io/MegaPor...
❤️ Watch these videos in early access on our Patreon page or join us here on TH-cam:
- / twominutepapers
- / @twominutepapers
🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew Melnychuk, Benji Rabhan, Bryan Learn, B Shang, Christian Ahlin, Eric Martel, Geronimo Moralez, Gordon Child, Ivo Galic, Jace O'Brien, Jack Lukic, John Le, Jonas, Jonathan, Kenneth Davis, Klaus Busse, Kyle Davis, Lorin Atzberger, Lukas Biewald, Matthew Allen Fisher, Michael Albrecht, Michael Tedder, Nevin Spoljaric, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Rajarshi Nigam, Ramsey Elbasheer, Steef, Taras Bobrovytsky, Ted Johnson, Thomas Krcmar, Timothy Sum Hon Mun, Torsten Reil, Tybie Fitzhugh, Ueli Gallizzi.
If you wish to appear here or pick up other perks, click here: / twominutepapers
Thumbnail background design: Felícia Zsolnai-Fehér - felicia.hu
Károly Zsolnai-Fehér's links:
Instagram: / twominutepapers
Twitter: / twominutepapers
Web: cg.tuwien.ac.at/~zsolnai/
#DeepFake - วิทยาศาสตร์และเทคโนโลยี
Whoa this is wildly good.. and I didn’t even realize Samsung did excellent AI research
Samsung basically owns SK along with the other 3 big families, they're great at both dodging the laws and dabbling in every single tech field
The actual research is made by the Russians. I wonder how things will go with sanctions and everything, whether Samsung's funding will stop and they will be funded by someone else or not.
samsung is 20% of South Korea's economy.
If you wanna stay relevant you have to do AI research :shrug:
@@mnomadvfx loooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo ऊपूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूूपप्प
Those are great results! To answer your question about what actors I would like to see revived: none, as we should rather look for new talents from younger generations. These legends have already left a great legacy behind in my opinion.
The potential is still mind-blowing, though!
Just the looks does not review those people I would only use it for continuity reasons.
This technology will replace actors completely. There will not be "younger generations" unless you are counting the research papers based on these techniques.
Allot of actors also loose enthusiasm for acting as they get older, sometimes brought on by illness of cause.
Aging sucks, doesn't benefit anyone for the whole world to get old and unable to function _well_ for a good portion of their lifespan... just my 2 cents.
@@markfeeer2149 there ARE no continuity reasons. whatsoever.
A character being re-cast by a new actor is nothing new, nothing unseen, and has always been well tolerated by audiences. Take James Bond or Batman or Superman or Billy the Kid or... who ever, until Disney decided we were too stupid to understand it's a MOVIE and characters get re-cast.
And no, aging doesn't benefit............... anyone; except... you know, the young people. Who get jobs after the older actors get older. Like, imagine if Arnold, van Damme, Bolo Yeung, Sigourney Weaver, Dolph Lundgren, Sly Stallone... were still as they were in their 30ies; would Chris Hemsworth, Chris Pratt, John Cena, Charlise Theron... ever gotten a chance? If these current actors become easily and cheaply replicated, are we just going to continue to have 57 new movies with CGI replicated Hemsworth Thor and no actor even having hope to get into that role, and in turn the role never having hope to be freshened up and renewed?
( Like imagine all the Batmans and James Bonds' we had in the past decades. Yes, but every new actor brought something new different and interesting to the role. Imagine if all the movies were just made by replicating Lewis G. Wilson's or Sean Connery's performance. Would we have missed out? In a word: YES. )
What I'm saying is, life has to happen. We shouldn't stop aging and dying because then we'll stop developing and growing as a species. I'm not enthusiastic to age or die; but I can say with 100% certainty that, should I have lived for 400 years, I would have changed, developed and DONE less than 5 different people replacing "me" in order during those 400 years,each continuing roughly where the last one stopped. New replacing the old is a STRENGTH, not a Pitfall.
@@scheuerman This could very well be the case in the long run. However, at least in the short-medium term, I think both actors and deepfakes will be used, a bit like today's movies using mixes of CGI and practical effects.
The fact that its source is a 2d image, and it´s a real-time deepfake is impressive.
A realistic fully digitize 3d models of our face and the reference would be the next level (real-time aswel)
And who knows, maybe one day we will fully digitize each individual 100k hair :D
Complete with a database collected from say .. YT of any and all persons, characters, monsters, and anything else that can be remotely anthropomorphized for instant use in zoom meetings and beyond.
Now where's my Torgo costume?
nvidia is going to add head and hair reconstruction to audio2face
what would be the point digitizing 100k hair when you can simulate them ? none
@@fabricejaouen378 lol true, but it would be nice to simulate the physics to the 100k level 😃
When I first started an AI and Robotics meetup years ago, one of my first expressed desires and predictions: the creation of Synthetic Prince. First in virtual form, then in robotic form. Secondly, to have all episodes of Star Trek: The Original Series re-done in VR, where all of the environments and people off-camera are inferred along with the on-camera elements, allowing anyone to BE THERE and experience everything as if an invisible member of the crew. A future version would go beyond observation to allowing you to be a new crew member that evokes new real-time interactions, which means all of the character personalities will need to be inferred. Outlandish today; reality tomorrow.
The Synthetic Lifeform Formerly Known As Prince!
What would be really interesting is to train an AI based on a characters interactions so the AI then makes the same decisions given the same situation. if you do this for every character, you could then throw them in a new situation and essentially you could create new content. though this would likely only work for well defined characters which are not that common these days. lots of "because plot" for the reasons characters do what they do. i look forward to the day when AI can make movies based on our own scripts. i think we will see a massive increase in quality content and all these hack writers and directors can finally be thrown away.
@@jessiejanson1528 Great thoughts, and this is precisely what I meant when writing "...that evokes new real-time interactions, which means all of the character personalities will need to be inferred." "Inferred" = output from an AI trained to do so.
The Holodeck cometh!
This genuinely scares me. Now, if enough audio/video of a given person exists, evidence can easily be fabricated to discredit them or to ruin their lives. Detection will only get worse, as adversarial training is a big part of how these models work.
Yes but how could it ruin someone’s life if enough people know how easy it is to fake? Video evidence would become untrustworthy generally. But I guess in some countries especially it would be worse than others
@@ilikenicethings Also even in places where deep fakes are known, the technology is still not super widely known. A fake video could still get a lot of traction before debunkers jump on it, and of course there will always be people who’ll believe it no matter what, because it supports their views.
Teaching about fire this technology can be misused is what the author of these videos is doing. He seeks out government authorities to teach them before a major misuse occurs.
We're entering a reality where nothing on a screen will be trustworthy, which will create a double-edged sword:
On one hand, normalizing the concept that the internet is no longer just biased and unreliable, but predominated by objectively fictitious imagery, will demand that the general populace will need to stop relying on social media for information, lest they be ostracized with the willful ignorance equivalent to a flat-Earther. This is assuming all of our brains aren't _complete_ porridge by the time deepfakes become indistinguishable and accessible. It's always possible ten years from now, we're brainwashed to the point of being incapable of not believing our monitors no matter *what,* but if that's the case, slanderous deepfakes are probably ineffective to the big picture.
On the other hand, this would also make a lot of _important info_ moot. Security footage will have to be considers invalid in the court of law. But I suppose overall that would not matter very much, given recordings will inevitably be outclassed by AI powered security/investigation, anyway?
I'm more concerned with subjective matters. There's always going to be black and white evidence somewhere. We aren't going to incarcerate Obama for a deepfake of him saying _"I assisted Epstein."_ What frightens me is how things like deepfakes and generative AI will taint integrity and values, and thus critical thought, in general. Even sticking to the topic of deepfakes something as simple as actors no longer y'know...acting, and receiving royalties on their likenesses, creates a really out of touch circumstance, and that's a pretty tame example. Apply that automation to everything (including art!), and a utopian society looks more like a Brave New World situation of people not giving AF about anything.
@@Sashazur As with most things what we need in response is more education / critical thinking to counter-act the negative social effects. Otherwise I'm not sure what point there is to people complaining about how this tech may be misused. There is no putting the genie back in the box.
you know when he says "Wwwhat a time to be alive!" it is truly groundbreaking
It morbs me everytime
Absolutely!
Love it!!!
@@AmedeeBoulette .
You also know it looking at the results of the new tech, duh :)
This is awesome, I particularly like the "live painting" at 2:46. I would use this result to put a screen on my wall disguised as a picture frame but with a live subject like on hogwarts! Also imagine the possibilities, if a show gets cancelled too soon (cough cough firefly), we can hire new people to write a new script (or use gpt-3 to do it for us!) and then shoot the new episodes with the actors looking as old as they were then.
+1 To a virtual Firefly continuation! 😁😁😁
I was saying this to my colleagues! There are so many satisfying ways to use it. Finishing dead projects, replacing awful casting choices, and giving people cathartic closure for 'what ifs': we can finally see Cage as Superman!
@@avidrucker sweet dork asset dirk6
That's a great suggestion. Someone needs to make me another ten seasons of Stargate ASAP.
And then on the other hand, you could probably deepfake a fictional character to get better casting for a fantasy or sci-fi show as well. Though, *cough* simply casting the right ethnicity character these days would be a major improvement over anything woke hollywood does now.
I think in a decade or so we'll be able to feed firefly fanfiction into a general purpose AI and get multiple screenplays out of the other side, have them critiqued, iterated upon and have the best one picked, multiple storyboard artist and director algorithms interviewed and the best ones for the job recruited, the same with the other tasks, and get a full movie as the output.
Oh, I recognise these artefacts! This happens when the AI gets fixated on the screen-position of fine details. When the actor moves, and the deepfake video moves, the hair stays on the same place on the deepfake, while the hair moves relative to the head. This problem is already addressed in another paper, which used an AI to create local coordinate systems for the faces, so using the two techniques together might be an easy way to fix this problem.
From these paintings it understands the light source and it generates correct shadows for the movements. The catch light on the eyes is also moving like it should but maybe not perfectly. Anyway, it's a testament to the AI and the painters too.
I can already see a business for bringing to life deceased friends and family from old photos
Future gets cooler and weirder with every step
I can't wait until they are able to recreate old TV shows. Are you ready for season 9 of That '70s Show?
Seeing the Mona Lisa moving is so satisfying
There's a whole community of people who want to do real-time streaming but they don't want to show their face on camera. So we've got these cartoon avatars that they make or have made for them. Some are more elaborate than others but they all lack any real human like emotion coming through. There is one, codemiko who has a very expensive setup and renders herself as a 3D character that looks game level realistic but its not really practical for the average streamer. I could see something like this really taking off and allowing more emotionally charged real-time streaming characters based on anime, movie or game characters. I wonder how long from this demo to it actually being a real product anyone can use with a phone filming their face. I'm guessing its months away. There's a big demand for this.
Some of these are so smooth, it's mind-boggling how detailed they continue to be! The bandwidth must be much greater than ever before, even with temporal decoherence!
I'm fully expecting next paper to fix the temporal aliasing. I wouldn't be surprised if they already had a solution ready but they published this first to be able to publish multiple papers.
the problem is the movement, because if you just pause, that picture where you paused, will look real.
Can you let us know when it is available for the public to test? I tried to reach out to the authors, but I don't know if they speak English.
I just wish all of these amazing tools would be open sourced and put together into useful applications for the hands of everyone!
They will be. Stable Diffusion, for example. Companies will want to exploit it, but others will copy the techniques and make it available.
@@cdeford Yes, that is true, but Stability AI is a rare exception unfortunately, even though I do wait to see what they will be able to accomplish in the future. I hope the landscape shifts, but so far way too many of these technologies are way too closed in my opinion. :)
Actually, I think when this technology gets perfected, it's going to get scary. Kind of like website name addresses, I feel public domain actors and actresses of the past everyone's going to suddenly jump on board to try and claim their images as property.....from studios to average people, depending on the stars. Then families of dead stars will start suing, feeling they should own their image. Lawsuits will start up everywhere in order to own the dead! That's my prediction.
The most present danger lies in that it will help make dismissal of genuine videos more credible. Adding to the disturbing trend by certain populists that shall not be named to declare ANY unfavorable claim "fake news" by default
It's interesting how you can get a picture in your head of what someone might look like based on their voice, but when you see them, they look like nothing like you imagined.
I can't wait until this technology is combined with VR to produce immersive concert experiences like seeing the Beatles at the Cavern or on Ed Sullivan etc.
2:07 i didn't read the text at the top of the screen first and thought it was the other way around for a second. that just shows how good these deepfakes are.
Your videos keep me excited for the future of technology. 👍🏻 Thanks
I've been subscribed for a couple of years and this is the best thing you have ever had on here.
Woow! 🤩 Super cool step forward! From a 2D image... mindblowing.
Wow. That is quite amazing. Thanks for sharing and educating us in this wonderful tech ;)
People cannot (and should not) trust what they see or hear anymore. Unless you see something for yourself first hand, there is too great a chance it may have been edited or altered in some way.
And seeing something first-hand ain't so great either - consider the unreliability of eyewitness testimony.
This definitely has the possibility to be very insensitive. I can imagine there would be quite an outrage if someone just tried sticking "Robin Williams" in their film today.
Now saying that, I've had an idea for a long time that but would be cool to make a modern noir thriller starring Humphrey Bogart
With all these new deepfake/text gen AIs there's a lot of controversy but I think nothing can be done about it
For well known actors using their image for anything other than fair use will require approval from the actor’s estate, otherwise lawsuit. But I’m sure some will happen anyway.
Judge : We have CCTV image showing you violated traffic signal. 🧐
Me : DEEP FAKE! DEEP FAKE! 🤓
yeah except if nobody tampered with the footage then it can't be edited 🤓
@@kw4093-v3p Next Level Quantum computing OEM DEEP FAKE ! DEEP FAKE! 🤓
It's just 1s and 0s.
Skynet did it! 🤖
@@kw4093-v3p One now also has the burden of proof that the data has not been tampered with. Difficult at best, impossible at worst. The biggest winner here is lawyers, who'll use this to invalidate evidence or indefinitely delay trials.
In either event, the use of any recorded media as slam dunk evidence is rapidly nearing a close.
The only exceptions I see involve multiple vetted independent sources used to build a full case.
As a side note I would just add the given the still fairly resource intensive nature
of this stuff, the biggest players are still going to be state actors and multinational corps, for now. However, it won't be long before little Billy-Bob is popping this up on his Christmas I-pad.
It's no longer a question of if this tech will work, but when. We need to start thinking about the consequences and prepare, or get run over by the truck when it arrives.
Fantasztikus lett a videó, csak így tovább.😊
Kíváncsiságból: hogyan lett 173 feliratkozód anélkül, hogy bármit mutattál volna a világnak?
is there any app I can use right now to test this please?
Hello - I’m working on a project and was wondering if it is deep fake technology that is used for those baby generator apps. Or are deepfakes exclusively for face swapping?
I love how exited he sounds
Thanks for your work explaining this to the public and policy makers! It's great to see someone very knowledgable making the implications of this technology accessible. DeepFake detectors do exist and AI does not need to be locked away because of "ethics concerns!"
The one of Frida Kahlo is quite beautiful. I can imagine this will empower a whole new generation of image filters for social media.
Oh God please no… but yes I’m sure that’ll happen
Károly you are amazing, thank you for another top quality video. It's always like the good news of the week when you upload. In times like these, I really need that. I hope you are doing good and I see you next time!!
I love when Dr. Karoly says "squeeze that paper" 😄
As amazingly cool as the technology is, I really am concerned about the social implications of these styles of AI application.
Not the obvious challenges like fake videos of politicians or 'deep fake porn'. But take the resurrection of dead actors, for instance: Would wearing a dead person's image like a mask tarnish the person's memory or reputation? Or what if we come to routinely bring back dead actors year upon year, would that affect the chances of new, would-be celebrities from their own opportunities to have their faces seen and so become future legends in their own right?
What about us becoming even more obsessed about our looks than we already are? We could make much more physically attractive avatars of ourselves, and so long as noone meets us in person, we can have that be our faces. Maybe that's useful, true, but could that perhaps rob us, say, the personal growth that comes from learning to love our own faces and flaws? Coming to terms with ourselves, in other words, which we all need to do at some point in our lives for the sake of wisdom and good mental health.
What will future society risk losing by all of this, and all that is to come?
Very interesting thoughts. I have no answers, just curiosity with a small pinch of worry.
About the avatar issue, I'm pretty sure you can already do that with current technology. Especially the ones that allow face synchronization for 3D avatars. However, the one in the video is better because the only thing you need as reference is just a 2D image. So even if you don't know how to create a 3D avatar for yourself, instead of paying an expert to make it for you, you can just download an image for any character you want and you will have your own avatar for streaming.
permission would be required from the actors before their death
Exactly what I thought. I understand why one can get excited by seeing such technology but the negative implications could be vast. Being a realist I'm sure there's no way to stop this growth, but as with any technology (like smartphones) we should talk a lot about the implications and actively prevent such technologies (more by self restraint than by law) from messing with our minds and societies.
If I could choose to apply correction whenever faced with a presenter that has one eye staring off in a different directions, then I could stand to watch Pete Thorn and Dave's Garage again. I like their content, but find the wandering eye extremely distracting and spend most of my time avoiding looking at the screen.
Where is the code for the last demo ?
Can we download this pre-trained model?
Wow, I've been following your channel for a while and you are much younger than I pictured in my head. Great technology and we need better understanding of science in politics, that's great 👏
Love these videos. One thing I never understand though is how do we try these out?
I wonder if this could be used for video conferencing. Instead of sending over a compressed 12fps video that is low res and delayed, send over just the co-ordinate data of the face.
Definitely the best copy deepfake so far. Better than wav2lip, first order model, and thin plate spline motion model. I hope it's easy to use and doesn't give a limit.
what Is the app (preferably free) available that is closest to this reseach?
how can I use this? is the software for sale? subscription?
What's the name of the painting on the right at 0:15 ? Thanks.
Emilia Clarke is safe for the time being. Convincing AI eyebrow emulation is a least 3 papers away.
is there any software or online mobile app?
Good sweet papers I love you. keep up the terrible puns and the great work. Hahaha
This can be download or it's just a beta render show?
AI could technically, soon, live on as actors who have passed. insane!
Where can I try this?
Finally now everyone can make their canon version of got s7
What is the software being used to produce these? He was very vague about any info to look this up
It's so funny!... Near the end when asked "What actor would you like to see revived this way?" My first and always thought - PRINCE! 😀First suggestions after the question: Prince or Robin Williams. 👍
What if you feed the output into the input?
Looks so clear!
This is will be awesome for translating language between films
Would absolutely love that for roleplaying games, also throw in a voice deepfake and we can take it to a whole other level!
I like the way you think.
Hi!
How can I test this program? How can I get it , please?
What is the name of the application to make this deep fakes
Is there any way in which I can use one of these, I’m an artist and I love to see my drawings come to life
The fact that it's real time is insane.
As an overview this is very good, a hands on lab will be great.
I can't wait to be able to use this in my Team's meetings at work :D
Making the dead dance for us is an iffy subject for both fiscal and ethical reasons. Same goes for the art generators to a lesser extent: the thousands of art-pieces the ais "trained on" are seeing no returns on their works being processed.
Gaming & simulation graphics will soon be indistinguishable from reality!
Really amazing. 1:30 lower left error with the guy's hair on the face
We see pieces of solutions across the papers, but still don’t see physics, full denoising, temporal coherence, etc. applied at the same time to generate the perfect solution. Means it should be coming soon.
thin plate spline motion model is based off first order motion model and bundled in deepfacelive, if you have a 3000 rtx gpu you can run it in real time. it's not as pretty as megaportraits but it's fun to play with
Great content and energy!
That is pretty amazing. The Mona Lisa is incredible.
How can I experiment with this ?
What a time to be alive! Lol Love his enthusiasm
1st scene of Terminator 4 brought a young Schwarzenegger, in a indistinguishable "real life" video quality. Mind boggling impressive.
You had 11k views with 1k likes after 1 hours. I like this!
If i go back in time to show DaVinci the deepfake Mona Lisa, he would be like: i have already sketched the code for this, it must be somewhere scattered up the attic.
if only JOKER had studied to be an AI Researcher, he would have made everyone smile
Man look at how fast this stuff is improving. What is the next ten years going to look like.
Thank you for sharing these papers videos.
They are much enjoyed.
Seems VR will be hyper realistic pretty soon, instead of shapes made of triangles .
Honestly, as impressive as this is, this is extremely worrying to me. I love your videos but it'd be cool if you talked about how this could be misused as well.
Well it’s not like we believe everything we see on internet
@@hombacom a significant portion on the population always does
so where's the software?
Is this software available?
I find it great for animating art myself, not exactly Deepfake level but I have been using Crazytalk on my Midjourney ai generated art to bring characters to life, that of course is obviously puppetry but I know some use Faceapp etc on it, the ABBA characters are actual rigged 3D models AFAIK and professional face marker mocap used
The EYES!! They look so realistic and beautiful.
am i able to use this?
is the code available to the public?
This looks amazing. Is this an app? What type of app is this? I would love to do one of my grandma for my mom.
I absolutely love your scientific curiousity, it's that true l'art pour l'art vibe, but with deepfakes getting better and better, I find it mildly horrifying, and this is the point to paraphrase the great Jurrasic Park question: "the scientists were so preoccupied whether they could, they forgot to ask themselves whether they should". I also think no actors should be brought back and I would also be a bit concerned on making filmmaking and other arts more accessible to create with these, with quantity outnumbering quality, uniqueness, the challenging process of creativity would lead to art and creativity fatigue and inflation. Not to mention the security issues, etc.
To be fair, this would definitely could bring in a new way of understanding history. I loved the paintings part, how awesome is that!
ohhh coool celebrities!!!!! they definitely won't use this stuff to track us like cattle, it will definitely be used for fun concerts!! :)
Is there a program or app that I can buy to do this myself?
A "paper" (whatever tf that is) doesn't help me.
Deepfacelab do this
I assume the current way of using doubles under a high-fi CGI face will stay in professional film production. Lower budget productions might possibly take this approach though.
I don't want any dead person being brought back as a puppet without their consent.
We love you, Doctor!!
"trying my best to teach political decision makers from all around the world to make sure that they can make the best decisions for all of us."
You can't be serious. The tech is amazing and all and not something I would see so soon, but the thought of these "decision makers" having yet another tool that helps them towards their ends is utterly terrifying.
I don't think he meant that he is giving politicians and world leaders deepfake technology. I think he was referring more to the teaching of ML awareness, safety, and ethical considerations.
@@avidrucker I’m sure you’re correct but inevitably teaching them about it will lead more to misuse it than who might have otherwise. But not teaching them is worse!
Imagine the 12 year old or younger kid who will be interacting with the professional world with an adult avatar undetected.
I want to see a movie where the main cast consist of me and my friends using deepfakes :DDD
yay - now they can stop constantly looking for new Bond actors, and just make new films with Connery and Moore.
But it’s not really them. More like a puppet that looks exactly like them. Their personality is totally gone and that can’t be faked.
what a time indeed !!
Imagine that actors will just license their face and they won't do any real acting work. And that you can revive or use any person for this purpose. This will completely change movie industry in the future.
Btw is it possible to approximately calculate how much does it cost to create those models and to run them? Generally for every demo we can see here. It's good indicator how long does it take before it start to be used in practice. Thanks.
Why even license real actors when they can generate the exact face they want using another model
how tf you comment before the video comes out