NCsoft’s New AI: The Ultimate Stuntman! 🏋
ฝัง
- เผยแพร่เมื่อ 13 ม.ค. 2025
- ❤️ Check out Weights & Biases and sign up for a free demo here: wandb.com/papers
📝 The paper "Learning Time-Critical Responses for Interactive Character Control" is available here:
mrl.snu.ac.kr/...
❤️ Watch these videos in early access on our Patreon page or join us here on TH-cam:
/ twominutepapers
/ @twominutepapers
🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bryan Learn, Christian Ahlin, Eric Martel, Gordon Child, Ivo Galic, Jace O'Brien, Javier Bustamante, John Le, Jonas, Kenneth Davis, Klaus Busse, Lorin Atzberger, Lukas Biewald, Matthew Allen Fisher, Mark Oates, Michael Albrecht, Michael Tedder, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Peter Edwards, Rajarshi Nigam, Ramsey Elbasheer, Steef, Taras Bobrovytsky, Thomas Krcmar, Timothy Sum Hon Mun, Torsten Reil, Tybie Fitzhugh, Ueli Gallizzi.
If you wish to appear here or pick up other perks, click here: / twominutepapers
Thumbnail background design: Felícia Zsolnai-Fehér - felicia.hu
Wish to watch these videos in early access? Join us here: / @twominutepapers
Meet and discuss your ideas with other Fellow Scholars on the Two Minute Papers Discord: discordapp.com...
Károly Zsolnai-Fehér's links:
Instagram: / twominutepapers
Twitter: / twominutepapers
Web: cg.tuwien.ac.a...
#gamedev
Some very skilled athletes do something called "recovery" where they may land a trick off balance, but recover their balance using another trick. Two common recovery tricks are flashkick backflip for backwards balance offset and webster for forwards offset. If a NN can pull that off WHILE falling dynamically, then there is no doubt it's much better than any static animation could ever hope to do!
I'm pretty sure video games and movies are going to go through a revolution.
I'm really excited for well made dynamic / physics applied character animations in games.
I want dynamically trip over rocks!
Hi Kleric, love your vids!
When I first saw GTA IV, this is really where I hoped game animations were going. Becoming more dynamically and physics generated. I love this stuff!
Well, just let you know, this is not a work for physics based animation, those policy only outputs animations
i want dynamic movement with vr too
We've got death stranding
This is one of the most exciting things I expect to see in AAA games soon. I wonder if this can even work in a multiplayer context, and correctly synchronise motion between clients.
Good news, AI is deterministic.
Bad news, its sensitive to starting conditions/input(chaotic).
But if it could be handled server side(costly) and just lerped on clients machine that would be a working solution.
Or it could just work, i know nothing.
@@lupsik1 No one knows anything about AI. Don't feel bad.
The paper was implemented in unreal engine.
I’ve been hoping for this too. Also, realistic clothing and weapons that never clip into each other. It doesn’t seem to be a high priority for gaming companies though. New World, for example, has less realistic behavior of capes than WoW and tons of clipping issues and armor that just stretches instead of moving. Some of the pre-baked animations are decent but overall no moving the tech forward from this wealthy gaming company.
@@diamond_h0us You mean cloth physics simulations? Two Minutes Papers got a lot of those vids. There's compute inexpensive one that uses AI to simulate real life physics.
It depends on whether the game company is looking for stuff to put in their game. In WoW, is the implementations worth the effort only for just capes? I mean those are just cosmetics that aren't contributing to gameplay mechanics.
What is amazing, the student network actually makes the same changes for kicks, as a student martial artist would do. You can see in the examples, the Student ai, to meet the timing requirements, pulls back end of the kicks 4:10 for example (the teacher animation has wider kicks, with more stretched movements).
This is so cool !!
This is so cool! Imagine an RPG where training agility actually improves your characters agility! 😯
this'll be amazing for parkour RPGs like Dying Light or Assassin's Creed series. Imagine your parkour movements get more concise and more responsive as you level up. This can work with fighting games like Sifu too
@@alexdoan273 I'm imagining a GTA with a better police ai that spends more time chasing you instead of just shooting, and this would give the game a reason to bring back training so that you can be come a parkour master and have epic cop chases around town. And the less trained you are, the more likely you are to stumble on whatever you are jumping over and fall to the ground. Kind of like how Oblivion used Athletic and acrobatics to improve your characters maneuverability, but more visually pleasing.
I was hoping to see the AI horse do a spinning back kick!
Yes, my disappointment is immeasurable and my day is ruined
You get the the mocap data for that and they'll make it happen😂
Just wait 2 more papers down the line. 😂
Then I'd squeeze my paper.
It would be very interesting to train the agent how to do a set of dance moves and learn how to synchonise them with the beat of music. The next level would then be to learn partnered dance moves and to improvise combinations in time to music. That would be really impressive!
Check this one for music sync dancing. th-cam.com/video/V8MlYa_yhF0/w-d-xo.html
Great stuff! However, there some clipping artifacts. The quadruped was cleverly placed in a grass field to cover up the foot sliding issues which had a LOT of! That area definitely needs a lot of improvement. Waiting for the next paper down the line! :)
A previous 2 minutes paper suggested that implementing animations in a game engine with physic simulation such as Nvidia's could solve this problem!
@@mnomadvfx indeed for a 5min training material its really impressive!
bit of foot IK, tweaking root motion and problem solved
This series of work will go along way to improving games visually. Animations are one of the few things that have lagged behind other graphical improvements and you only get realistic animation in precanned mocap stages.
This stuff will have the ability to not only improve animation blending, but also give animation variety, and more realistic environment to character physics based variations. So no more fighting games where you do the same exact motion over and over. The ability for stamina to change each subsequent moveset will look so much better.
Just need to get it to a point where it's performant to run across numerous characters on screen. I'd imagine they might be able to adapt this to do a LOD style animation frame approximation (further objects don't need as high of a frames/sec to look good).
But at the same time, I question the decision to utilize purely physics-based animations to be used in gameplays. I'm probably missing something, but as seen in vehicle maneuvering and robotics, there are dead-lock positions, where the controllable object in question is unable to move from a certain situation no matter what their output motion is. For example, you can't flip cars over if you happen to land them on the backside, as often seen in games like GTA V.
I would understand if you are trying to make a sandbox game, and you are trying to incorporate characters that abide by the same laws of physics as all of the other objects in the game, but if I were to be presented with a story-based game, like Bioshock, I would expect my character to move in directions that are not necessarily physically accurate. I might want to jump over cliffs and bumpy terrain that human-shaped characters would not realistically be able to go through without constantly stumbling over, while expecting them to walk through them without any problem due to our exposure in and subconscious understanding of video-game logic (Death Stranding).
I'm probably missing something because I'm obviously in the minority for holding this opinion.
Yes, a game development paper! 🤩
How to implement it?
Imput lag is actually an important game mechanic, being able to remove it is good but it isn't needed on every move
Oh my. I've just read the whitepaper; this can go much further. What a gem!
In this video: Károly demonstrates that he doesn't play video games. Or else he's so ahead that he's modded the Witcher to be Street Fighter or something.
3:52
For me the most cool part is that behind these neural networks and AI in general, lies just pure math
It's probably good enough for any game. By nature, as of now, games tolerate a certain amount of foot sliding already, as we use the same animations for several speeds of movements. Switching from producing X number of fixed animations and using them with standard transitions to producing X numbers of movement types animations and having an AI interpret them with time-critical responses and smooth chaining is already a great step for game development use. Now, if somehow we can find a way to reduce the footprint further using the tensor cores on nvidia cards, then it would be perfect.
@@mnomadvfx Games are fine with spending most of the computational budget on a key feature, contrarily to most other pieces of software. Your client base doesn't use windows XP computers running on pentium too, as you can expect them to more or less use recent hardware, so you're not cutting yourself out of that big an audience by requiring a powerful computer.
I would hope it could run on more general GPU compute, as can't guarantee what kind of hardware it runs on, Nvidia RTX, GTX, AMR RX, and Intel Iris Xe all handle this a little differently to my knowledge, but can run the same programs.
I like the slower animations, they feel more natural, as if they have inertia from suddenly changing direction.
This will totally fix the UFC games, no matter how much they try they lack the flow of a real fight but with this it will be perfect, in fact any fighting game will be improved by this.
Thanks Karoly!
@@mnomadvfx true, two moving objects colliding and trying to rebalance themselves might be more complex but testing the model as it is would worth the shot and if it doesn't you know what Karoly says...Two more papers down the line! 👍
A parkour game with this would be awesome! Hope it becomes reality one day.
You have become, dare I say, one of the most important channels on the platform
5:33 I assume they used grass to hide the sliding of the horses feet.
Yeah, the horse def looked a bit wonky
It's a little surreal when the videos in reference at the beginning are all videos i watched here.
What a time to be alive.
I’m sure the developers of Trespasser are in tears that this kind of technology is coming to maturity, it truly is a time to be alive.
The pace of the jump is so well matched.
This is mind blowing! Imagine when Epic releases something like this for free in their unreal engine, so much time saved for such a great result.
Policy Distillation WOW. This methodology very much reminds me of an idea I had many many years ago now. Pretty much where and an AI code could create copies of itself which could then create other copies of itself. Ad Infinitum. Keeping the main modules the same but only allowing certain parameters to change. Like a Mandelbrot Set. I shouldn't have stopped coding all those years ago. Oh well.
Would it be possible for the AI to develop better high jump techniques than we currently have?
yes, but it appears the flop is already the best possible
Not at all in the industry, and have no papers to grab; -but i love keeping abreast of current technology! Love your channel, Doctor!
Yes yes yes! I'd love to see neural networks in charge of character movement in all games!
imagine in GTA 6 you can train your character in mixed martial arts to fight like this
Holy shit! Now imagine leveraging this improvement process as a game mechanic. So the player ends up _actually_ training their avatar. 🤯
It would be cool to see these type of movements in npcs in future vr games, even just the realistic movements would be a huge improvement to suddenly shifting directions over one frame.
But the question here is: do they fall to the floor if we bomb them with flying boxes?
It'd make sense to have the jump button be a charge up - hold it down to enter "gonna jump soon" mode, and let go to actually jump. We don't jump from a standstill, after all.
Amazing video -was confused when I scrolled down and saw only 52 likes on the video then realized this was just posted :D
Neo: "I know kung-fu."
Agent Smith: "Funny you should say that..."
Now I want to see that available to make a parkour game oO
A lot of these studies only give small amounts of data to the AI and showcase the results, which is great and all. But I would love to see some examples of what its capable of when the AI is fed many hours of data. Is it that the AI is not capable of handling it? or are they just not interested in going that far, since the impressive results with miniscule data gets the 'message' across thus they are not motivated to go further...?
Should the "Back Flip" at 2:23 be "Back Spin" as shown later?
0:16 What's a bunch of VLCs doing in there?
This seems to be as big leap as inverse kinematics was to the old static way of doing animations. Looks like it has a lot of potential.
This video would make for great source material for a documentary when the terminators come... 🥶🥶🥶
It would be so good of the response time budget descreases as the character improves their skills!
le potentiel est juste hallucinant !! merci pour le partage
He actually said the word "gamer"
Amazing work. Excited to see more of it
so there will. be no more pre rendered animation for taking a hit by opponent??
Your videos keeps me on my toes about what's going on with AI and sparks creativity within me. Thanks.
This is wicked good-looking btw. I can barely wait for upcoming NCSoft games that utilize this - NCSoft makes games that revolutionized movement in their genre, such as Aion (with its responsive flight an glide mechanics that just filled my heart with joy) and Blade and Soul (wall-runs!)
Let's hope we'll get to play them here in the west before we die of old age - by the time Blade and Soul became officially available, I've grown a beard that reached down to my chest, SHAVED IT, then grown it again. And that's not an exaggeration.
So how do I implement this now in Unreal or Unity ?
Looks great, I hope it will be used to improve the face animations too.
This one was SICK!!!!! One step closer to see these neural networks used in actual games which currently suck at everything that is not visuals.
I absolutely love this channel, all the new AI/tech stuff is mind blowing! But I can never figure out how to use any of these tools :(
(I know most of the time they are not public but some of these at least should be available to the public right? Would help game development immensely)
Sometimes the authors share their code, in this case, the links can usually be found in the paper text or on a project page. Or you can email the authors and check with them if they have an implementation
I love this channel, so glad I found it!
"Did you press the button, but the action hasn't happened in time?"
Yes, I did play GTA 4. :D
That stylish delay is why I prefere responsive controls, even when they look a bit unrealistic.
Amazing video, thank you.
It appeared that the Unreal Engine mannequin was used in the first part of the video (approx 01:40), so, does anyone know the workflow between, weights and blends & Unreal Engine?
If so, could they let me know to discuss further, please?
Thank you =]
good video. how many params in the student network?
Any recommendations on getting started with ai on something like unity?
Also, to add to e.g. in comments - an algorithm trained to breakdown complex dance patterns and teach a user in VR. It should start from learning user movement limitations and teach along the way in a continuous nuanced manner. Now swap dance patterns with Martials Arts, self-defense, EVEN better help disabled people learn to be mobile again.
When might we download this? I really want to try this out for myself, and I'm sure others would too
I wonder if this physics-driven simulations can be used to optimize real world robot movement.
im looking forward for the video games in a couple of years, its gonne be awesome!
This is a lot of improvement. Amazing.
I wish there was a channel that showed you how to implement these types of tecniques into our games.
there should be an ai, which gets papers and example projects running in your favourite engine.
Nice, the only thing that looks weird is a ground contact. I wounder how animations will look in 5-10 years with this pace
This would be amazing for Storrors upcoming parkour game
"... which you can play almost immediately. Why almost immediately? I'll tell you... [ ~almost immediately~ ] in a moment."
Ah, what a missed chance!
I'm sorry, I dropped my papers...absolutely phenomenal 👏
How does this compare to motion matching ?
This is good news! The more modern and advanced games become, the more sluggish they feel to control. As an old fart who grew up on Doom and Quake, modern games like The Witcher series and the GTA series make me seasick because there is so much art direction and flourish animations between my button press and the actual result on screen. I had to stop playing Red Dead Redemption 2 because of this. I feel more like a director than the player in these games. At least in single player games I'd like a slider in the settings that control the amount of Hollywood choreography that is allowed between my controller input and the reaction on screen. Something like this could perhaps help with that!
"Policy distillation" seems to mirror what happens between human teacher/students. A self-taught teacher learns techniques from trial and error, and good students can then refine those techniques for even better performance because they're not wasting their time with unproductive experiences.
The FitnessGram™ Pacer Test is a multistage aerobic capacity test that progressively gets more difficult as it continues.
The 20-meter pacer test will begin in 30 seconds. Line up at the start.
The running speed starts slowly but gets faster each minute after you hear this signal.
A single lap should be completed each time you hear this sound.
Remember to run in a straight line, and run as long as possible.
The second time you fail to complete a lap before the sound, your test is over.
The test will begin on the word start.
On your mark, get ready, start.
All that beautiful visual realism that flushes away when you see batman's cape waving through walls is going to be fixed soon! :P
0:22 this is how 90% of ads advertise their games
Artificial intelligence is such a crazy thing. And really the truth super intelligent beings are those who are creating and developing all this.
Can you please produce a video that shows what AAA games use the technology from past papers?
Do you have a demo to share with us ?
1:26 what an epic fail at dancing of the previous method
Nagyon jó, hasznos és minőségi videó. Csak így tovább Károly! 👌👍❤
@Mack Haha. She said useful and quality work. keep it up.
It feels like ages since Ubisoft invested in technology outside of monetization! Maybe this will start another early Assassin's Creed era, when they're done with NFT stuff.
3:57 What we don't know at this point, is he pulling is punches or not.
This is really cool. We won't have to come up with separate animations for the different player states... just drop the rigged model onto this and let it fly.
So the student was performing shuffled actions ? it seemed as if it was learning from master the "ABCD" sequence (a fixed sequence) and enhancing the transitions into quicker ones, but i wonder if the student could also BACD or something different (without training )and do the enchancement as well...
Now this is awesome already but i think , getting kicked in the ass is also a demanding skill so what if they train the ai so it spawns the enemies and tries to place the spawns to where those kicks and punches land without going through each other of course (and eliminate any " waiting inline to be kicked situations that we get to see in B movies.)
But can they get up as well as Nvidia Isaac's combat AI?
NCsoft is doing something innovative!
So where the plugins?that i can use for my game?
im glad to see how this evolved
1:07 straight yeets themselves onto the hard ground
Which video game will be the first to implement this?
It looks pretty weird but that martial art moves sequence was sick and smooth.
These are the unreal engine default pawns, is this available in UE4 now??? I want to buy this plugin
can i and how can i use this for animation of characters in a game??
Wow. What a time to be alive!
nothing excites me more than active ragdoll animatiom improvements, I hope Unreal Engine buys these papers
Future Assassin's Creed games are going to be pretty cool with the movement if this is implemented.
I can imagine a game where the character really developes in such a way...
What a perfect thing to wake up too! Great tech as always. I'm excited to see how Ubisoft utilizes this.
Imagine a Spider-Man game using this technique!
Can we use it in our games?
What a time to be alive!
I'm disappointed that the horse didn't do a backflip