Just imagine a point and click adventure in a 3D space using these animations. I can see great use cases for almost anything demonstrated here (except for sitting :))
Sebastian, this is some spectacular work!! These solutions are very elegant and silky smooth. Congrats and thank you for sharing the project. I'm going to have to dive into it shortly to check out implications and applications.
Yeah, I too am amused by the fact that Anubis is the test subject to run through these motions. To the original poster, was this being used for a specific Egyptian themed game?
i decided to come back to this after a while and holy hell my jaw dropped when you said two hours of motion capture data. i'd forgotten just how absurd that little data is. anyways this is a fantastic paper and i can't wait to hopefully see tech like this in future games.
one minor detail that i noted especially near the end with the comparison between LSTM and NSM was that the model's hands occasionally clipped into itself, especially during certain animations. would it be possible to add something similar to the enviroment sensors to the model itself? already you guys did a great job with model clipping, but it's not perfect yet. also, how does this framework compensate (if it can) for different models? eg different height, limb length, size, etc. it would be awesome if you could resize these animations for different models and effectively use 2 hours of motion capture data for a infinite variety of bipedal models. anyways, i'll leave that there for other potential studies later on. i'm not an expert on this and certainly can't build anything like this at the moment, that was just stuff i was wondering as i watched.
The side by side comparison @ 5:10 demoing the impact of the gating was night and day. Is there any intention to provide these systems as a middleware solution for unreal engine? There are very limited offerings in this arena currently other than one motion matching system. Good luck with your endeavors and thanks for sharing your work. Fascinating.
Amazing stuff! I've actually been waiting so see AI take care of movements like this, especially when walking up and down stairs, that I would love to see. Next step, do the same for clothing to avoid clipping :)
So awesome it's sad it's for research only... and not really interested in jumping through hoops to use it in a commercial project... Will have to wait for and make do with Kinematika. I hope Unity can learn something from this project, or hire Sebastian to help or something. Would be awesome if Kinematika can reach this level of quality.
Many 3D characters come in a range of shapes and sizes, with different lengths for limbs. I wonder if there is a path that could train one set of motions then correctly apply it to all the different characters with no loss in fidelity. Hopefully a future paper :)
This is just amazing.... when can we see this released on the unity asset store for commercial use? (The github licence is sadly only for research or education purposes)
Notice how ever 10 years or so we get an amazing video like this about cool animation tech that 90% of us can't afford or ever get access to because we are to poor. And then the video just sits on you-tube for 10-15 years and we just keep scratching our heads and continue using our old tools. And what about Unreal? The most popular and most used engine.
Seen this several times before and it got the same issue. The copyright and "only for research" means this project can not be used for commercial games. It often die after a short while after Unity updates and makes the research code outdated. It's really sad this happen over and over again. A quick search in both Unreal and Unity marketplace show that these kind of advanced locomotion systems don't exists there.
they have this on their copyright " Licensing is possible if you want to apply this research for commercial use. " so if anyone wants to use this for a videogame you could make an agreement
Is it just me or is the only stiff part about these AI animations... is the ankles? The ankles just seem a little stiff in their movement as if they only bend to half the flexibility they can really achieve. Other than that however, this technology is brilliant.
Official: We do a lot of work with NSM and deep learning framework to make a in-game animation better. Real: Thank you for finance development a new open world furry-yiff game.
Cool. Still, the cloth brushes his bones every few seconds, completely destroying any immersion the whole effect, admittedly quite interesting, may have created. Cloth physics and it falling into other objects completely ruins such experiences for me but I'm afraid we're still quite far from implementing it properly.
@@sebastianstarke5023 When is it possible to see it compatible with UE 4? Say someone had more traditional moving mechanics in a game and wanted to change for this kind of system. Would it have to be remade from the ground up or could you fairly easy take out bits and pieces and retrain them?
![[SIGGRAPH 2024] Categorical Codebook Matching for Embodied Character Controllers](http://i.ytimg.com/vi/NyLRcY0c0p4/mqdefault.jpg)
![[SIGGRAPH 2020] Local Motion Phases for Learning Multi-Contact Character Movements](/img/n.gif)
I do hope big companies invest in this technology as fast as possible and develop games based on it.
Amen, All game's could be improved by this amazing software.
it's already crappily licensed, so it's kinda useless until ppl make kinda the same thing but with permissive license
probably Unreal will buy it :(
Just imagine a point and click adventure in a 3D space using these animations. I can see great use cases for almost anything demonstrated here (except for sitting :))
a game like sims will will be really amazing with this one
Sebastian, this is some spectacular work!! These solutions are very elegant and silky smooth. Congrats and thank you for sharing the project. I'm going to have to dive into it shortly to check out implications and applications.
Anubis, god of the dead, just casually moving some stuff around :D ... anyway this is really interesting stuff!
Yeah, I too am amused by the fact that Anubis is the test subject to run through these motions.
To the original poster, was this being used for a specific Egyptian themed game?
That actually a BadDragon stuff.
So good. You keep evolving since the quadruped motion of last year's Siggraph. Please keep the great work up and thanks to Taku Komura too!
what a time to be alive.
i decided to come back to this after a while and holy hell my jaw dropped when you said two hours of motion capture data. i'd forgotten just how absurd that little data is.
anyways this is a fantastic paper and i can't wait to hopefully see tech like this in future games.
one minor detail that i noted especially near the end with the comparison between LSTM and NSM was that the model's hands occasionally clipped into itself, especially during certain animations. would it be possible to add something similar to the enviroment sensors to the model itself? already you guys did a great job with model clipping, but it's not perfect yet. also, how does this framework compensate (if it can) for different models? eg different height, limb length, size, etc. it would be awesome if you could resize these animations for different models and effectively use 2 hours of motion capture data for a infinite variety of bipedal models.
anyways, i'll leave that there for other potential studies later on. i'm not an expert on this and certainly can't build anything like this at the moment, that was just stuff i was wondering as i watched.
The side by side comparison @ 5:10 demoing the impact of the gating was night and day. Is there any intention to provide these systems as a middleware solution for unreal engine? There are very limited offerings in this arena currently other than one motion matching system. Good luck with your endeavors and thanks for sharing your work. Fascinating.
This is incredible!!! This is what I expect (demand?) for next gen character movement! I never want to see a slippery foot again lmao!
Great work!!!
Amazing stuff! I've actually been waiting so see AI take care of movements like this, especially when walking up and down stairs, that I would love to see. Next step, do the same for clothing to avoid clipping :)
I'm completely blown away by this :D Currently I'm an early technical artist working with mocap and I'd love to learn the workflow for this!
Workflow: Use all CPU only for animation controller.
@@razzraziel Not really. Neural networks are super fast, and so are voxels.
When? When actual good games that use such innovations (and there are so much possibilities, not just this).
excellent work. hope in future we can see more actions
A Sims game with this system would be insane
Excellent work. Is there a plan to enhance it? For instance, to synthesize more difficult body mechanics animations, such as parkour animations
racking up those steps, the Anubis NPC - gotta hit that 10,000 step milestone!
Very nice work, Sebastian!
So awesome it's sad it's for research only... and not really interested in jumping through hoops to use it in a commercial project... Will have to wait for and make do with Kinematika. I hope Unity can learn something from this project, or hire Sebastian to help or something. Would be awesome if Kinematika can reach this level of quality.
Man this is amazing. I bet big game companies would buy the shit out of this... Must save tons of time and resources
Good improvements since 2017. Or 2016? Are you planning another asset?
Welcome to the smite god reveal: Anubis. God of walking and carrying boxes.
my jaw just dropped. amazing stuff!
very cool stuff. i wonder if this could be applied on cloth for example.
Many 3D characters come in a range of shapes and sizes, with different lengths for limbs. I wonder if there is a path that could train one set of motions then correctly apply it to all the different characters with no loss in fidelity. Hopefully a future paper :)
Irre, hat ein Spiel das bereits so implementiert ?
This is just amazing.... when can we see this released on the unity asset store for commercial use? (The github licence is sadly only for research or education purposes)
Why was VSync disabled?
Recording screen at 60Hz while running Unity on 4K plus neural network plus character controller was probably too heavy for my laptop :(
@@sebastianstarke5023 how to make in unreal engine 4
Could it be done using unreal engine 5?
The motions look good. The next thing to work on is collision detection with the clothes.
Notice how ever 10 years or so we get an amazing video like this about cool animation tech that 90% of us can't afford or ever get access to because we are to poor. And then the video just sits on you-tube for 10-15 years and we just keep scratching our heads and continue using our old tools. And what about Unreal? The most popular and most used engine.
This is impressive work.
Games are going to look fucking awesome in 2030
Seen this several times before and it got the same issue. The copyright and "only for research" means this project can not be used for commercial games. It often die after a short while after Unity updates and makes the research code outdated. It's really sad this happen over and over again. A quick search in both Unreal and Unity marketplace show that these kind of advanced locomotion systems don't exists there.
Sad but true...
they have this on their copyright " Licensing is possible if you want to apply this research for commercial use. " so if anyone wants to use this for a videogame you could make an agreement
How do you use Python modules in unity3d
Any relation to Kinematica from Unity ?
I understood some of these words
Is it just me or is the only stiff part about these AI animations... is the ankles? The ankles just seem a little stiff in their movement as if they only bend to half the flexibility they can really achieve. Other than that however, this technology is brilliant.
Official:
We do a lot of work with NSM and deep learning framework to make a in-game animation better.
Real:
Thank you for finance development a new open world furry-yiff game.
Though, it's not the fault of the people that innovate. It's the companies fault. They actually fucked the whole market.
this anubis is chill
This is huge
круто круто вау ))
crazy
It's amazing
Look at what these people are acheiving. And here is me... not dying at least
Tensorflow rocks!
I know you have this for Unity. (Buying IT) But Please Oh Please port this to the Unreal Engine. :)
GTA 6
Cool.
Still, the cloth brushes his bones every few seconds, completely destroying any immersion the whole effect, admittedly quite interesting, may have created.
Cloth physics and it falling into other objects completely ruins such experiences for me but I'm afraid we're still quite far from implementing it properly.
OWO
What the dog doin
People are no longer needed
Ъ
This stuff never makes it into the games. Contact us when you get this in a game engine like UE4
It's implemented in Unity: github.com/sebastianstarke/AI4Animation
@@sebastianstarke5023 When is it possible to see it compatible with UE 4?
Say someone had more traditional moving mechanics in a game and wanted to change for this kind of system. Would it have to be remade from the ground up or could you fairly easy take out bits and pieces and retrain them?
@@sebastianstarke5023 Do UE4 and you'll sell shitloads of this plugin! ;)
5:58 *Boston Dynamics PTSD flashbacks*