This Neural Network Combines Motion Capture and Physics
ฝัง
- เผยแพร่เมื่อ 7 ก.ย. 2024
- ❤️ Watch these videos in early access on our Patreon page or join us here on TH-cam:
- / twominutepapers
- / @twominutepapers
📝 The paper "DReCon: Data-Driven responsive Control of Physics-Based Characters" is available here:
- montreal.ubiso...
- static-wordpre...
🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Alex Haro, Andrew Melnychuk, Angelos Evripiotis, Anthony Vdovitchenko, Benji Rabhan, Brian Gilman, Bryan Learn, Christian Ahlin, Claudio Fernandes, Daniel Hasegan, Dan Kennedy, Dennis Abts, Eric Haddad, Eric Martel, Evan Breznyik, Geronimo Moralez, James Watt, Javier Bustamante, John De Witt, Kaiesh Vohra, Kasia Hayden, Kjartan Olason, Levente Szabo, Lorin Atzberger, Lukas Biewald, Marcin Dukaczewski, Marten Rauschenberg, Maurits van Mastrigt, Michael Albrecht, Michael Jensen, Nader Shakerin, Owen Campbell-Moore, Owen Skarpness, Raul Araújo da Silva, Rob Rowe, Robin Graham, Ryan Monsurate, Shawn Azman, Steef, Steve Messina, Sunil Kim, Taras Bobrovytsky, Thomas Krcmar, Torsten Reil, Tybie Fitzhugh.
/ twominutepapers
Splash screen/thumbnail design: Felícia Fehér - felicia.hu
Károly Zsolnai-Fehér's links:
Instagram: / twominutepapers
Twitter: / karoly_zsolnai
Web: cg.tuwien.ac.a...
#gamedev
3:09
Simulated humanoid: *crushed under a mountain of digital cubes"
Karoly: "Outstanding"
Free my boy AI Walker dude
Reminds me of that scene from the Incredibles
More like "Inlying"
mr incredible swimming under so much elasticgirl's a$$
This will possibly make him the first victim of the AI uprise
When they started throwing the red blocks at blue guy I felt it became a visual metaphor for my life.
The fact that this is physics-based makes me think that it could be applied to motorized leg prosthetics.
Maybe have a scan to the person for measures, approx weight distribution and a basic movement tracking.
Then you let the personalized leg AI learn with the digital version of the person.
At this point you should have a good start from the beginning of the leg's install.
And from then you let the AI keep learning, with the user rating the leg performance on their own.
Love the way your thinking! So many applications to jump start big performance hurdles in different fields.
So in the future, we can see npc ragdolling around because they tripped over a small physics object. Amazing.
Yeah, but it won't have been explicitly programmed, it will have LEARNED how to trip over like an idiot! ;)
@@PinataOblongata Just like a person
Imagine that, Euphoria A.I.
@@PinataOblongata It’s not explicitly programmed in _right_ now either, it’s just that today’s physics engines still often have edge cases that are unaccounted for.
@@PinataOblongata That makes me want to launch gta 6 and push every npc I encounter :D
3:00 - actual footage of me trying to get through the week
You're very good at this. I heard it's state of the art, so everybody else is basically doing worse!
KKKKKKK Às vezes me identifico com isso.
Hahaha
2:30 As someone who easily sprains his ankle this was hard to watch
It hurts? I barely feel it. Not the fall tho. I can definetely feel that
Glad it wasn't just me that recoiled from that hahaha
"these edibles ain't shit"
20 minutes later : 2:56
2:31 "Yup, that's me. You're probably wondering how i ended up in this situation."
I don't have the time to calculate your average video length, but it's more than 2 minutes *and I love it*
this is just cruel to the AI.. we need ethical standards for this kind of experiments!!
Yeah I heard they delete it after they finish. #justicefortheAI
AIs do have souls #dontthrowcubesonme
Any ai must be a consentual test subject "programs the ai to consent"
@@trueredlucky954 they are kidding
@Solve Everything of course not. This is cruel just because it's an artificial intelligence does not mean it's not intelligent. We should put the same ethical standards we use on humans on ais. I mean a blonde who died there hair brown are artificial intelligence. But we still have the same ethical standard. We need to be above this anti robot racism.
3:00
So now we can simulate the age-old gag of throwing tomatoes at the presenter!
1:13 laughed pretty hard, lol. also i'm loving this comment section
This will be such a huge jump in fluidity in games. Looking very much forward to this being implemented into upcoming games & movies, etc. Thanks for sharing, as usual. Such incredible work! I shared your channel in my latest upload (just an hour ago) sharing my top 10 TH-cam channels. You're in the top 5. 🥰🤗 Keep up the amazing work, and again thank you!
3:09
AI: *gets stoned with digital rocks"
AI: "God, please end my suffering."
2:54 is basically a The Legend of Zelda player upsetting a cucco for the first time and trying desperately to escape. There's no escape.
They need to start modelling protection of the head. Thing faceplants without throwing its hands up, doesn't put its hands up to protect from objects flying at face, etc. I guess that's just another example of the same sort of learning in this scene, but why not make it look awesome by simulating both? ;)
Piñata Oblongata I’m going to assume that the footage it is trained on was mainly about walking around, less so about protecting itself.
What's cool is that, we can train ANY behavior this way. Jumping, shooting a bow, carrying things. Including things like self-preservation, prideful behavior, fear, bravery. Man, this is way deeper than it seems.
How's life going?
Me: 3:02
Lol same
"Looks very realistic!"
"Outstanding!"
I liked that moment when a bunch of randomly shaped boxes fall towards the robot.
Damn, my lungs hurt from laughing at the 3:08 mark!
3:10
TMP: Outstanding!
Blue Guy: **Fucking Dies**
Congratulations to UBISOFT and thanks for sharing Karoly!
Imagine a game which combines all the papers you’ve reviewed, it would be not that slow and 100x better than most games.
WaliWorldX you put that way better than I did
I love you guys, you're as a whole persistently improving video games for everyone
3:05 Stop! Stop! He's already dead!
And 1 year later we have Hellish Quart implementing this in a playable game :) What a time to be alive!
I believe this should be implemented into the player controlled character as well, making the decisions far harder. If you see some rough terrain you will think twice. RPG could use it, like dexterity increase makes the player's character much more capable to overcome terrain hurdles.
dude how’re u not cracking up
2:24 sumotori dreams
YES, I played that game when I was a kid and the characters couldn't even move 2 steps without falling over. This paper is like a super improved version of sumotori.
3:06 me getting bombarded with homework
Dude your videos are incredible. Crazy interesting topics, short and insightful and well researched. You are a true Gem to the TH-cam community!!
When animation will become realistic, our brain will not be able to recognize that is only a game, and you could react more emotionally to the game, get more angry, or even go crazy, especially if it will be used in combination with VR. People will go crazy because of games, forget to eat... It's nice that you can simulate psychics but, imagine that kind of very deep fakes. It will be used with big data from social networks, public and hidden cameras, microphones, it will create model of your brain, the way you think and it will be able to simulate or predict your next action. Imagine when someone will get mad at you, use very deep fakes to create realistic video of some crime and accuse you of something that never happened.
I can see the use for this. Because motion capture has many out of world problems. Sensor accuracy or different terrain than the studio or unique obstacles. I can't wait till Blender gets the addon to allow us to make some fun animations.
2:45 *catches cube thrown at him*
Blue looks good until they fall over and they don't have any sort of human-like response. No hands out to cushion the blow(s) and protect the head, for example. The scary part is that I can't think of a way to put that into the model without assigning what amounts to pain levels to different parts of the anatomy. i.e. it's going to hurt more to land on your face than your hands and knees, and that will discourage you from landing on your face. Also, hands don't get a concussion.
1:05 I'M LAUGHING WAY TOO MUCH AT THIS LMAO
it'd be fun to train an agent in how to deal with having blocks thrown at it by adding various inputs and fitness algorithms to make it so that hitting various parts too hard 'hurts' and that way the AI, upon tripping, will try to protect itself instead of trying to continue to walk.
Poor blue man
Throwing blocks at him until he falls down and has a seizure
This reminds me of Naturalmotion's Euphoria engine, also its usage in GTA IV but that was 14/12 years ago. It's not new tech at all, Euphoria might have been programmed manually rather than simulated and learnt but it looked and felt just as responsive.
Hayal euphoria has been used for years, in RDR2 it was used extensively and looks great... but not as good as a deep learning algorithm probably could accomplish
Looks like Euphoria physics (GTA 4)
Have you noticed how nice the animations in Assasins Creed Odyssey already are?
yes ;)
You're the one and only channel I watch every last video from.
2:56
I just find it funny how 2mp is just calmly narrating while the blue guy gets overwelmed by red boxes
Make him do my taxes and vacuum my house!
Introducing Roomba and Mr accountant
You have to do your own taxes in the USA?
@@lategamer6684 not exactly, it's more like taxes do you
Reminds me of those people kicking their real life robots. Now they have death threats by people who emphasized too much with their robot. I don't know if they got death threats, though, but people were really angry. I also felt for the figure.
I'd like to see it adapt to an avatar with different body proportions from the mocap -- either training with a bot of different proportions, or after training, give it different proportions.
[When I'm drunk]
White - How I think others see me
Blue - How they really see me
2:45 CATCH!!
2:45 When you suck at Dodge Ball and finally catch one of the balls.
1:07
White guy - GER
Blue guy - Diavolo
Search a game called "Exanima" one of the best physics based combat I have ever played
Put in a few algorithms of psychology data tracking how People would react to things, like say an approaching pedestrian on a road, and You have a CGI Actor
It still looks tipsy and bow-legged to me but impressive nonetheless.
looks real to me. I always trip over those damn red blocks
I wonder if Boston Dynamics used the same approach to make their bots stable under pressure.
Thank you for the great videos, this video had me clutching my papers hard with excitement! This is amazing work and I can't wait to see where it goes from here!
When I first saw this I was blown away! Amazing work indeed!
The movement should not be restricted to joints. Human body generates momentum every centimetre and it can travel linearly within at nonlinear rate. Simulating this would yield more realistic and accurate stylish movement.
3:03 me vs homework
I´m dying to get videogames with this technology. 120GB of sounds, textures and models; 2TB of data from the AI that know how to walk.
When computers become sentient we are so dead lol. "You taught my ancestor to walk, and then threw hundreds of red cubes at his cowering frightened frame.".
"We only did it to make you... stronger.."
Click.
"You succeeded."
Bang.
3:05 when your grades are dropping and your trying to figure it out but you keep getting more and more homework and its just overwhelming you
subject buried alive
*Outstanding*
2:45 catch
I laughed too hard at the little quip at 1:15 😂
This AI already operates in the mind of the most advanced drunkards!
its still a long way from looking and behaving realistically in my opinion but its such a big leap and is exciting in potential
That's what I meant when I suggested "AI boxing"...
So basically they're working on a movement AI where they don't have to train all the 'assassin creed climby stuff' into it? It can just figure it out on its own? This work could pay for itself many times over. They hire literally hundreds of people for world design. Being able to say half their labour hire spend, or double their output (purely random figure), would be an outstanding result for the industry.
I wanna see this in a cod game or something, it'd be sick for moviemaking
can't wait to actually see stuff like this on actual big games
This will be the future of games, i am sure of it
I can't wait until my character trips on small objects in game! That could actually be cool for some games as long as it doesn't happen too often.
Looking forward to this technology in the next Assassin's Watch Far Recon Ghost Creed Dogs Cry
I'm gonna watch again because at some point I couldn't listen anymore, neither breath
It's amazing, I wouldn't say it looks very realistic tho
That footage of drunk ai is gold
Did not anticipate this video to be so damn funny.
Hi there, I am very fascinated by the neural network concept... How could I do some of those experiments myself?
Does anyone know the name of the paper that implemented neural firing delays and relaxation resulting in more natural motion?
I feel like rockstar has already achieved what the last simulation shows, especially in rdr2, but even as far back as gta4.
I really hope they make this open source. Such a shame to have such a revolutionary technology remain proprietary.
IMO still needs a bunch of work, to me the animation based character add a much better feeling, way more than the AI + physics one. Btw something akin to this, was invented before and is used in GTA 5, some Star Wars games and some sport games, is not used for the entire character state only when he dies, don't remember the name of the system but the technique itself can be described as active ragdolls that try for example cover their face, head or grab things, when they go flying.
NaturalMotion Euphoria. It's dead, and if you try to reimplement it, Zynga might patent troll you.
It was a collection of handcrafted behaviours (animation blends essentially) that would be triggered by a given condition, plus the classic ragdoll augmented by a spring system that would try to maintain some kind of pose if the character loses control. But it could never perform standing actions with the physics active, self-stabilisation wasn't there, the little that was there was purely visual.
Is RL and computer graphics the more popular topic in recent AI research? There seems to be less research papers covered on this channel on NLP, Image classification etc.
that's really fun. i guess robots do evolve to conquer human 'cuz while they are being thrown millions of boxes, i'm just being a couch potato watching youtube. btw, can you train a kongku master like that in the Matrix by switching from resisting red boxes to dodging bullets?
1:06 this joke was funny =)
Yeah, strange no one mention this :D
I envision a day soon where games are almost fully driven by AI. A procedural world and believable world, no two buildings or characters or objects ever looking the exact same. Limitless voice acting, any line conceivable could be present. All this with almost no performance costs... the potential is endless. Think of what a community of modders could do with something like that
Haha really interesting. So the difference with DeepMimic is the additional user controller which can respond to real-time joystick, right?
Cool. Very interesting!
Why don't these RL-puppets have a smashes-head-on-the-floor penalty term...? Would make for some more realistic fails, I guess.
The original model that the AI learned to imitate... was that obtained through motion capture?
So the stuff i got really excited for when the first assassins creed game might actually be in a game in a couple few years
Finally haha
3:06, when the person throwing the blocks smokes some weed.
What do you mean? When they smoke weed they get super accurate and one-shot the target?
0:55 GTA IV walking and drunkmode
1:13
Liking this video for that alone .. lol
can anyone suggest a starting point for doing this kind of Physics-based work? I currently work in IoT and Computer Vision + DL mostly with Python, C# and Arduino/ESP32 C++, I have used Unity before for a class but didn't have a proper project to do in it afterwards so I abandoned it.
I’m calling Peta for domestic violence, simulated slavery, cruel abusive punishment and glorified torture!
Now simulate 10 to 20 others and have a fight between them or a jumping against one.
jumping a meter in the air and landing in a T pose seems like the ultimate flex honestly
I love all of the game tech you've been reviewing, and it sounds like much of it is already viable in terms of performance and stability - but when will we see any of it in actual games?
Maybe talk to some game studios and see if they'd sponsor a special episode talking about real applications of this tech they might be working on? 🙂
I wonder about the potential application of this in training robots i.e. Boston Dynamics.