Honestly, the most impressive part of this video is the reward-graph at 1:39. There still seems to be even more space left to become better at the game for the AI. It's also quite amazing how simple your rewards were. You could have make it so much more complex. Anyway, great video, love your stuff, keep it up!
Congrats on the paper! Knew as soon as you said bleeding edge it was your own model you talked about in you past livestream😆. I just read the paper and it's really detailed, and a great advancement for the accessability of RL research as a whole! I was not aware of Manchausen and it's pretty awesome. Also this is by far the lowest replay ratio I can remember being used in SOTA RL, so that's really impressive. Your analysis of action gaps and Dormant Neurons is really insightfull and I wish more papers in the field offered similar stuff. Again congratulations and good luck with the submission! P.S. Any chance of getting more detailed results on Procgen👀? jk jk. Unless?
Really grateful for the mini-review on the paper! What more results do you want? The individual game scores are in the appendix, or were you hoping for something else? Sadly the reviewers at the conference I submitted to weren't as impressed :'(
Dude this is like a real time tool assisted speedrun. Very cool. Edit: If I’m not mistaken the ai is using a technique called momentum canceling as well. It’s air dodging after getting hit to last longer. That’s impressive.
@@viewerguy10I'm not convinced honestly. Smash is an incredibly complicated and nuanced game and one of the biggest things the AI would have to deal with is being able to adapt on the fly against an experienced player. A good player will always just be able to switch up their approach and cripple the AI immediately. AI can adapt over the long term but short term immediate adaptations aren't feasible.
Never seen a good smash bros AI before, this one is the best so far. Now we only need it to 1v1 itself on random characters to create the ultimate smash bros pro
I am really not into Smash Bros, but I knew your vid woud catch me anyways ;) Also your editing skills seem to have evolved again. Great job, keep it going :)
Id love to see an ai like this programmed to observe the believed best future decision when observing other humans playing. It would offer insight on how to possibly optimize conversions in neutral
Hi, i just found your channel, i am really intrigued by it. I am new to deep reinforement learning, can you give me some tips or some platforms where i can make AI like this one. Anyways great video, keep it up.
this might be a bad suggestion and it might take you a lot of effort but it would be interesting to see you make a Mario Kart Wii AI to play on wiimmfi with other players, but since it's an emulator it might not be possible Edit: you could use Retro Rewind if you don't want to homebrew wii and ther stuff, btut it won't know extra tracks
For your mario kart wii ai, can't you show it wr time trial runs to teach it the exact motions to do, and then combine that with all the skills it already developed through trial and error? Then wouldn't it be able to easily break wrs?
How transferable are the trained models to other characters and modes? I don't think the system would work at all if the environment changes even slightly, e.g. changing the stage.
This is amazing. I'm humbled by the fact that I don't understand why this isn't causing an uproar in the research community. Maybe because it's not a pure approach, but rather a Frankenstein of different techniques? So what? I don't get it. I skimmed the paper, and will be studying it later. I love the simplicity and feel I'll learn from it. I'm thirsty for the supplementary materials - especially the code! Where can I find it?
The supplementary material is available on the open review submission to ICLR, however I soon plan on doing a more polished repo for it. My reviewers don’t seem as impressed haha
Why endless battle? When you put it in a 1v1 battle is it just gonna try to spam up+b or will it understand the difference in damage output and received?
ngl the thumbnail is super misleading (I thought this was gonna be for ultimate given the Ultimate Mario render and Ultimate Battlefield) aside from that, this video was still super interesting
I'm curious, was it playing 4 instances of brawl each at 2x speed? Also, do you just launch the emulator 4 times to get the extra ai training, or do you have to have 4 different verdions of the program?
Its just 4 emulators so the AI can practice 4x faster. In theory I could make the agent try different things in each emulator (such as exploring more in one than another).
I may have miss the information but what do you use ass first layer ? In other words how do you give at the ia it position, enemies position, percentage and all other information on screen?
The AI is given an image of the screen, and uses a convolutional neural network to learn how to interpret images. This isn't just a sequence of actions, I could start this AI in any position and it could still play
Your voice is far too quiet in this video. It's kinda hard to hear you with the music being of similar volume as well. That said, these vids are always cool.
Find a good and very popular paper, then just use the papers it cites to find more relevant stuff. My paper is in the description if you want to read that
Honestly, the most impressive part of this video is the reward-graph at 1:39. There still seems to be even more space left to become better at the game for the AI. It's also quite amazing how simple your rewards were. You could have make it so much more complex. Anyway, great video, love your stuff, keep it up!
Yeah it always amazes me that the AI just seems to keep improving no matter how long I leave it for. Also makes me wish I had more compute!
@@aitangomake it learn smash ultimate and combos and match ups and frame data and see if players can beat it
I want you to team up with Code Bullet and Challenge Red Falcon to another Mario Kart AI vs Human duel!
Congrats on the paper! Knew as soon as you said bleeding edge it was your own model you talked about in you past livestream😆.
I just read the paper and it's really detailed, and a great advancement for the accessability of RL research as a whole!
I was not aware of Manchausen and it's pretty awesome.
Also this is by far the lowest replay ratio I can remember being used in SOTA RL, so that's really impressive.
Your analysis of action gaps and Dormant Neurons is really insightfull and I wish more papers in the field offered similar stuff.
Again congratulations and good luck with the submission!
P.S. Any chance of getting more detailed results on Procgen👀? jk jk. Unless?
Really grateful for the mini-review on the paper! What more results do you want? The individual game scores are in the appendix, or were you hoping for something else? Sadly the reviewers at the conference I submitted to weren't as impressed :'(
@@aitangowhaaat you wrote that paper? cool
Real OG's know this is a re-upload
Old video got a copyright claim unfortunately
@@aitango do liars bar
@aitango is spamming the move can be exploited
12.5 days of training is crazy! Thank you for choosing the perfect game to showcase the new "Beyond The Rainbow" algorithm!
Dude this is like a real time tool assisted speedrun. Very cool.
Edit:
If I’m not mistaken the ai is using a technique called momentum canceling as well. It’s air dodging after getting hit to last longer. That’s impressive.
Oh that’s pretty cool, I had no idea why the AI was doing that!
It would be really dope to see it fight level 9 CPUs or do Cruel mode. Also, could you do this for Melee too, or only for Wii games?
I know brawl isn't the most popular one, but holy smokes it would be wild to see this go up against an expert.
I think it would need more training at first to keep up since humans play differently. But it would probably dominate after a while
@@viewerguy10I'm not convinced honestly. Smash is an incredibly complicated and nuanced game and one of the biggest things the AI would have to deal with is being able to adapt on the fly against an experienced player. A good player will always just be able to switch up their approach and cripple the AI immediately. AI can adapt over the long term but short term immediate adaptations aren't feasible.
@@saf_Safira never played brawl but id assume mario up b is punishable, and the AI has never even seen a shield so...
@@saf_Safirahungrybox just uploaded a video of him playing against a melee ai fox. It’s pretty good, might interest you.
@@ytrqwee I watched it already LMAO
It is incredible to see how far your videos have come, your ai's are really impressive and only seem to get better!
Thank you! From the AI side things are so much better now, I can’t believe how bad they were when I started!
Never seen a good smash bros AI before, this one is the best so far. Now we only need it to 1v1 itself on random characters to create the ultimate smash bros pro
I am a simple viewer.
Ai tango uploads a vid,
I watch.
True loyal subscriber! Especially since I recognize your name haha
ah you're the guy that got rejected on open review lol
I am really not into Smash Bros, but I knew your vid woud catch me anyways ;) Also your editing skills seem to have evolved again. Great job, keep it going :)
Thanks, really great to hear!
This is really impressive. Love the content you create!
Really nice to hear, thank you!
Love your channel, so insanely interesting to see ai in Nintendo games ❤ which game you currently working on?
I'd love to see what a new AI would do vs a lvl 1 amiibo in Smash 3DS. Having them both race for development might be fun to watch.
Id love to see an ai like this programmed to observe the believed best future decision when observing other humans playing. It would offer insight on how to possibly optimize conversions in neutral
Been waiting for a new video 🎉
Hope you enjoyed it!
please post more, also maybe add a side channel or seires that is a complete tutorial on how to get into video game machine learning
I'd be interested in a behind the scenes video detailing every step of the process in setting this up. A tutorial almost.
lets watch it again
Thanks haha!
What’s a normal score to be able to compare how the ai did
Hi, i just found your channel, i am really intrigued by it. I am new to deep reinforement learning, can you give me some tips or some platforms where i can make AI like this one. Anyways great video, keep it up.
You should have trained meta knight: aka the only s+ tier character ever in smash because you are playing brawl
that mario AI spams UP B as much as a pikachu main that is annoying
I can’t really blame it, it seems to be quite effective
@@aitango no one i know like playing with that Pikachu main becasue we kinda play competitivy
this might be a bad suggestion and it might take you a lot of effort but it would be interesting to see you make a Mario Kart Wii AI to play on wiimmfi with other players, but since it's an emulator it might not be possible
Edit: you could use Retro Rewind if you don't want to homebrew wii and ther stuff, btut it won't know extra tracks
yo ai tango i've been making ai's for video games just like you bro you were my inspiration to learn how to code these things :)
That's really cool to hear, always love to hear people wanting to learn about RL and similar things
If you implemented this AI architecture yourself from the paper published only 2 weeks ago that's really impressive
I wrote the paper lol
For your mario kart wii ai, can't you show it wr time trial runs to teach it the exact motions to do, and then combine that with all the skills it already developed through trial and error? Then wouldn't it be able to easily break wrs?
How transferable are the trained models to other characters and modes? I don't think the system would work at all if the environment changes even slightly, e.g. changing the stage.
This is amazing. I'm humbled by the fact that I don't understand why this isn't causing an uproar in the research community. Maybe because it's not a pure approach, but rather a Frankenstein of different techniques? So what? I don't get it.
I skimmed the paper, and will be studying it later. I love the simplicity and feel I'll learn from it. I'm thirsty for the supplementary materials - especially the code! Where can I find it?
The supplementary material is available on the open review submission to ICLR, however I soon plan on doing a more polished repo for it. My reviewers don’t seem as impressed haha
@@aitango Cool! I'll be keeping an eye on your progress
How can I see the paper? Beyond the Rainbow and that snippet sounds really interesting
Link to the paper is in the description!
Why endless battle? When you put it in a 1v1 battle is it just gonna try to spam up+b or will it understand the difference in damage output and received?
How do you find papers on algorithms? I really want to start reading papers.
Hey what about trying this algorithm on Syoban Action? (Cat Mario)
😎👍
Do you think there's a way you could encourage it to use items? It never grabbed a single one as far as I could tell
I could give it a reward for doing so. I was quite surprised it didn’t to be honest
ngl the thumbnail is super misleading (I thought this was gonna be for ultimate given the Ultimate Mario render and Ultimate Battlefield)
aside from that, this video was still super interesting
I don't see the supplemental materials the paper references. Is that still forthcoming?
They were submitted to the conference the paper was submitted to, I’m not sure if it’s publicly available yet. If it get accepted it will be for sure
Jeez wow
Thanks!
@ No problem, that’s an incredible AI that took over a week to train nonstop! ❤️😄👍
I'm curious, was it playing 4 instances of brawl each at 2x speed?
Also, do you just launch the emulator 4 times to get the extra ai training, or do you have to have 4 different verdions of the program?
Its just 4 emulators so the AI can practice 4x faster. In theory I could make the agent try different things in each emulator (such as exploring more in one than another).
I may have miss the information but what do you use ass first layer ?
In other words how do you give at the ia it position, enemies position, percentage and all other information on screen?
The AI is just given the raw screen pixels as input. I don't explicitly given it the position or enemy position
How dose the ai see? Because it looks like a blind person playing just getting told how many opponents are ko and how long it was going on for
The AI is given an image of the screen, and uses a convolutional neural network to learn how to interpret images. This isn't just a sequence of actions, I could start this AI in any position and it could still play
Why brawl and not Melee, or P+?
I played it as a kid so am inherently biased haha
I know you got copyrighted, but what exactly caused that? I’m curious, but if you don’t know either, no biggie.
One of the songs! We thought it was non-copyright, but we were sadly wrong
Your voice is far too quiet in this video. It's kinda hard to hear you with the music being of similar volume as well. That said, these vids are always cool.
Now train one to play ultimate.
Man the vocal fry make the video really annoying to watch dispite the good job you did
🐐🐐🐐🐐🐐🐐🐐🐐🐐🐐
:)
⬆️🅱️
Too much up b base your reward system off of how Mario mains in brawl play the character
So... it got really good at Up-Special
blud didnt learn anything but spammin up b xD
how does it generalize, i.e. harder CPUs / varying opponent characters / vs human
Probably not well with no extra training. If I let it train on harder opponents it would probably be pretty good though
Why reupload?
Old video got a copyright claim sadly
re-upload?
Old video got a copyright claim :(
Whats with the reupload?
Old video got a copyright claim :'(
reuploaded for double view #scam
Old video got a copyright claim :(
@@aitango just kidding great video good sir!
invest in NNE!!!!!
How do you find papers on algorithms? I really want to start reading papers.
Find a good and very popular paper, then just use the papers it cites to find more relevant stuff. My paper is in the description if you want to read that