ฝัง
- เผยแพร่เมื่อ 1 มิ.ย. 2024
- #ai #mariokart #reinforcementlearning
This is the journey of how my AI has changed since I made my first video on Mario Kart Wii.
0:00 Intro
0:49 First Setup
4:32 First AI
5:37 Second Setup
8:20 Second AI
9:21 Final Setup
11:39 Final AI
13:56 This AI
Also thank you to my editor for editing this video!
@benji.botterill
www.benji-bott.com - วิทยาศาสตร์และเทคโนโลยี
![[TH] VCT Masters Shanghai Playoffs DAY 2 // PRX vs G2 | EDG vs TH](http://i.ytimg.com/vi/T5j2XVcDFs0/mqdefault.jpg)
![[TH] 2024 PMSL SEA GF D1 | Summer | แข่งที่ไทย ต้องใส่ให้สุด](http://i.ytimg.com/vi/Jkuf7nCS7jI/mqdefault.jpg)
What hooked me into your channel is that I really like that you train and execute the AI on a real game, instead of other videos around TH-cam that the AI is trained on a replica of the original game that doesn't replicate all of the original game's behavior
That's because without an emulator that has savestates and scripting functionality, it is much more difficult to train an AI on a given game than to make a crude version of it yourself.
I hope someday it becomes very easy to train AIs on games outside of an emulator. :)
The biggest part for me is definitely the savestates. They aren't included in most modern games, hence why people tend to make their own version. The scripting functionality massively helps speed up training, however it is possible to work around (such as the first version of my system didn't use any scripting functionality)
yoooo its the loritta guy
Now train this version for 30 hours and see the results
I'll be using this version soon for sure! However I won't be doing it just on Mario Circuit, I want to aim for something more ambitious (there may have some foreshadowing in this video about my next one :)
@@aitango "Can my AI beat all of Mario Kart?"
@@aitango "Can my AI Beat Wii Tanks"?
👀
@@aitango
2:00
"Can my AI commentate Esports?"
@@aitango Can your AI Play Dark souls? LoL :D
Thank you very much for the shout-out 😀 very interesting, as always. Especially for me since I'm more of a technical kind of guy. Keep up the nice work! Makes me happy to see that my tinkering around causes real people to achieve real awesome stuff 💪
No worries, I honestly can't thank you enough! I will continue to try and make evermore interesting projects!
Hopefully this video blows up, you're ai videos are really high quality and it's always a treat to see that you uploaded
I hope so too, thank you so much!
Great video, it's impressive to see just how much progress you've made with this AI
Thanks a lot, glad you enjoyed it
"Wii game aren't known for their square aspect ratio"
Is there a specific reason why you continued running the game in 16:9 resolution despite the fact that the game can run natively in 4:3? This would have been closer to a 1:1 ratio
Not really, dolphin rendered the game at 16:9 by default so I just went with it. Rather than wanting to change it to 4:3, I was just focussed on allowing the AI to accept any input size to remove the distortion all together
5:04 you can put the aspect ratio in 4:3 instead of 16:9 which some people preferred
Real
4 by 3 is still not quite a square, it's a little bit wider
I've been watching for so long now, this is really awesome! My university doens't have reinforcement learning classes, but watching your videos helps me see what it's like!
Really glad you like it! Mine only had one course on RL and it was in the Master's degree course. I learnt RL almost exclusively through online tutorials!
Keep up the good work man this AI MKW stuff is really cool to watch. You could def get 100k subs.
I appreciate that, thanks!
its extremely fascinating to hear about your adventure through this mariokart AI project. i found it highly inspirational to watch your problem-solving and makes me want to tackle my projects. progress on it must give the same heap of reward the mariokart AI gets using a shortcut.
i hope 2024 has been another good year of progression!
Cannot wait for future vids. Great job.
Really great to hear, I'll keep them coming!
Incredible, looking forward to seeing how far you can push this AI!
Thanks, I'll keep pushing it till I get a world record haha, gotta fulfil my younger self's dream, one way or another
100K is.. quite a few subscribers. Will be waiting patiently for this milestone though!
True, hopefully it won’t be too long haha
Hi there! I've been enjoying your videos since I discovered your channel a few days ago-really interesting content! I had a thought about the banana issue. Perhaps a foveated approach, similar to a barrel distortion effect, could emphasize objects in the center, making them appear larger without needing to increase resolution? It's a technique inspired by biological vision systems, which focus on the central field to enhance responsiveness. Just an idea that might be worth exploring. Keep up the great work, very cool stuff!
That's an interesting idea! Most research for Reinforcement Learning is done on Atari games which don't have this problem, so sadly there's limited research done on the topic. Would be something that could really help me out though and be useful to many others if it proved effective!
@aitango Good luck! Another idea I had was what if you gave the system a 45 degree rotated square clipped out of the center? That way you could have a larger starting image and then throw away parts that are unlikely to have any practical use to the AI. At least for this game where the most important features are likely not in the corners of the image.
Amazing how you improved on this over the years.
Ok, now let's see it be more strategic with power ups
This is something I really want to get working for future videos. Itemization is something I really haven't seen the AI do much of yet, so would be great if I could get it working well
If you can try to find a way to create save game states on any game and try to do it on some games that rely on itemization that'd be cool.
Thx for putting the subtitles in yt only! Great vid!
I thought I'd listen to the feedback haha
We had a few people ask, glad to hear you prefer it
its really inspiring how fun & creative yr research is!
Thanks, glad you enjoyed it!
Very happy to see this video! It's unique and gives some very good insight into the difficulties of creating an artificial intelligence like this. As for giving it more input you hinted at towards the end, I would like to know your thoughts on the importance of color in the AI's vision. In Mario Kart Wii, the road, offroad, and walls are often quite different colors, but can be very similar in lightness, making them harder to distinguish in greyscale. If RGB color is too computationally expensive, perhaps you could look into giving it an indexed palette; that is, a set of numbers that correspond to predetermined colors. This could help with distinguishing different elements on the screen, but I could also see it posing a problem with similar colors, as with a linear grayscale value, the AI may be at an advantage from being able to "intuit" similar colors based on their numerical proximity, something that is much harder to achieve with indexed colors.
Glad you liked it! I would like to use colour, but I'd have to see what the additional computation cost would be as you mention (Sadly I'm pretty certain it would be very heavy). I hadn't thought of using some different colour scheme, but that is a very interesting idea.
Been here from the start. Loving seeing the progress. Hopefully you can train an ai which can work just fine on different games. I like the fix you did to account for other events outside button presses and I still think giving it access to game objects will give it that final bit of sophistication going forward.
Love to see viewers who have stuck around for a while! I have many plans for other games…
Thanks for making all these videos😊
Thank you for watching them!
Dude you are such an inspiration for trying to create my own reinforcement learning AIs and love the videos. I am trying to build a tetris AI but still struggle trying to find the right reinforcement learning algorithm, I will get it working one day.
That’s amazing to hear! Honestly Tetris is a surprisingly hard game for reinforcement learning to learn, I remember doing something similar a couple years back
I hope this channel reaches a million subscribers.
I watched all the videos in one day. Incredibly interesting information.
Thank you so much! Really glad to hear you like the content
In bolth videos the ai don't seem to go on the inside curves. The inner radius is the shortest path therefore the fastest route that is if the racer doesn't lose too much speed doing so. In mario kart it's the opposite you can gain a boost drifting!
In Mario Kart Wii if you're good enough you can definitely still get the mini-turbo while taking a tight corner! The AI probably just needed a lot longer to figure that out since, in the time I gave it it was mostly just figuring out how to not die
I remember your first ai.
I knew this channel would blow up.
Great job man👍🏽👍🏽
Good to see a veteran fan haha! Thanks!
Very impressive stuff!
Thank you very much!
Very cool! Its neat to see your perspective on these issues - I am heavily involved in the frameworks used to train agents for Rocket League using reinforcement learning. It's a multi-agent setting so the problems aren't identical, but efforts in parallelization have come a long way.
For rocket league, there comes a point where it's faster to load the observation tensor onto the GPU and run batch inference than to run inference individually, which causes a huge issue with tradeoffs when thinking of parallelization. I'm guessing that's less of a consideration here since you have so few agents running. In rocket league we have much less bloat and so we can crank out time steps. It's not uncommon to see 8000 time steps calculated per second.
The framework I'm most heavily involved in uses Redis to send the experience buffers from generated trajectories to the learner. Have you looked into doing this? It would allow for distribution across multiple PCs - all you would need is to open the port for redis.
Sadly the way Dolphin runs has some real roadblocks to running in parallel. While I've figured most of them out on one machine, doing fully distributed across machines may be tricky. Even for a single machine though, I may look into having the actors produce actions in parallel, similarly to APE-X but with a slightly more advanced algorithm behind it.
Cool video!
Really great video and it’s interesting to see how much work you put into continual evolution and improvements. Keep up the great content and helping us mere mortals to understand a little about how ai works ! 👍
Thank you so much!
Your voice only makes watching this video even more enjoyable.
Now let it run 1 month 24/7
For most of my videos they do all run 24/7, the only reason I don't let them run for a month is because that means I would only created 1 video a month! (I'm trying to do 2 a month currently, but I also have to factor in that sometimes I try new stuff which doesn't work)
@@aitango so use another PC for such projects 😏🤍🤍
Great video! I was wondering if you plan on training with continuous actions in the near future instead of discretizing actions as a single button.
"never let them know your next move"
AI: *presses Alt + F4*
Great video, it's interesting how in the end the biggest obstacle was the emulator lol
It really was! While the AI can be a bit temperamental at times, most of the time it worked pretty cleanly. The emulator interaction was pure pain though (although getting it to work was strangely enjoyable at the same time)
Just make the world reccord of every track, you never show how mutch time it takes the IA to complete a lap
So far I've mostly focussed on getting agents to learn against CPUs, with items, multiple tracks etc rather than going for world records on time trials. Its definitely something I'd like to look at soon though!
This is greatest Mario Kart History ever.
I personally am really interested in AI and time trailing in Mario Kart Wii. Will the AI be able to beat world records some day, or even come close to TAS WR's?
Regular racing is also interesting though, but I have the intuition that it's actually extremely complex on a higher level of racing, and I especially mean clan wars. Lines, item use, strats, communication, context dependence, prediction, complex interaction between the players of and between the own and the other team. I wonder if it could some day be accomplished that an AI team beats the best clans in the scence in a war and that consistently.
Thanks for your great work!
I loved watching funky kong ai drive around mario circuit for almost 15 mins.
When I train these AI, I can see them training 24/7. It gets slightly addicting to watch tbh
Man your videos are so good i hope your channel just explodes in popularity soon
Thank you, I hope so too!
i'd love to see you give us a technical deep dive into the code!
Hopefully I hit 100k subs then I will do for sure!
I wonder if having a second AI to interpret the entire screen into what it actually sees, like another player in the middle right of the screen, a blue shell warning at the bottom, or the road bends round to the right to potentially reduce the inputs on the driving neural network and distill the information it is given so it doesn't get confused with having too much unnecessary and unhelpful data
This man explained Mario Kart AI to me, all while gargling gravel in his throat. Is there nothing he can not do?
I'm still deciding if this is a compliment, insult or both
Great work! Have you considered finetuning pretrained image networks? They may have more robust embeddings than you can achieve from a limited amount of simulation data and low frequency reward signal.
how would that work? what would the image network be trained for exactly? like are you saying some kind of feature extracting model, and then the RL agent can learn from the features instead of the pixels directly?
I've seen some research on using models trained on imagenet for RL, but they typically don't perform too well (at least on atari games, which is what most RL research is done with). Maybe for Mario Kart it would do better since its more similar to imagenet than Atari, but hard to say without trying it
@alganpokemon905 AI Tango is training them completely from scratch, so they have no concept of shapes or terrain or objects. But powerful image models have been trained on millions of diverse images. Using just a little bit of example data, these models could be adapted to MKWii. Potentially it would enable the RL to focus on the driving ability rather than its image recognition. It could also prevent overfitting. But machine learning is temperamental and there is no guarantee that it would be an improvement.
@@FuzzyJeffTheory Sir that makes sense to me but what exactly do you propose the image model should be trained for? I.e. should it be an object detector where you train it to detect mario kart objects or what?
this would be quite costly in terms of human labour
@ alganpokemon905 There are several approaches. One is to feed it human annotated data, as you say. But alternatively, there is a category of learning called self-supervised learning, where you either learn to reconstruct an image (eg. filling in a masked part of the image) or perform some contrastive task (eg. identifying which image patches came from the same image). And even if the model is not perfectly fine tuned, the RL can still potentially learn to utilize the image model embeddings better than from scratch.
Idk why i thought this was imported on the wii. I literally play melee every day...
But fr this is sick, you are a goat. Also, I've been thinking about SSBM AI since i saw you for the first time 4ish months ago. Lemme preface, i know nothing in code, but i wanna know if you think any of these topics you used could be used as a reference for that kind of game?
one issue i see is that it can't press accel and turn at the same time, which is pretty useful, IMO. Perhaps there is a activation curve for pressing the inputs, so that multiple at once can be pressed?
7:55 - im susprised your ai didn't need you to run a gpu since i noticed the difference between stable diffusion cpu vs gpu is about 15 seconds to 1.5 hours for me.
The AI does use a GPU! It's using an RTX 4090
@@aitangoLOL nevermind me! I can't believe I didn't notice it XD
The Aug, Sept, and Oct 2023 Dolphin Progress Report mentions this video!
Thanks for letting me know, that’s so cool!
:D @@aitango
I used AI to give u optimized parameters:
Choose optimal kart and character combinations: 8 points
Use power-ups strategically: 9 points
Master drifting and boost techniques: 10 points
Maintain speed and momentum: 8 points
Avoid hazards and obstacles: 7 points
Take advantage of shortcuts: 9 points
Block opponents' attacks and maneuvers: 7 points
Use item management effectively: 8 points
Learn and adapt to track layouts: 9 points
Utilize slipstreaming and drafting: 8 points
It’s hysterical that AI makes half of your content.
When ran multiple instances, how did you get each different instance to take different inputs? Did you have the input set to the same, so "w" is forward on all of them or did you have each instance have different controls? (If you did make each instance have the same controls, how did you make each instance take the correct controls)
Since im using a modded version of Dolphin emulator that allows scripting input, I can actually input controls straight into memory rather than binding to keys. In the first version I did have to use keybinds, but this version wasn't parallel.
@@aitango Are you also running multiple instances of the code, or is the ai looking at each of the instances and deciding what to do based on that?
LETS GO
Does this AI have a limiter on the amount of inputs it can do per frame, or can it perform TAS things?
You had to do a lot of setup for this course to determine progress. What if you tap in to your rank at any given moment in the race? 1st place, 2nd place etc. This could be used in multiple courses without any extra setup I imagine
Maybe there’s an integrator that adds (14-rank) each second as part of the reward function
So, are you going to give the AI direct controller access now? And perhaps rewrite the reward function to peek at the game memory?
In the second setup (the modified source code by Felk), I did start using direct controller access and looking at memory for rewards, sorry if that wasn't clear in the video. Thankfully the modded version of dolphin provided this nicely
Call me solidly surprised running dolphin in fullscreen slowed the game down at all, I've never even considered it, never seen the emulator running at less than 4K
Its not something most people have have to consider haha
Would you happen to have the link to the github repo that lets you read the ingame memory in python?
Interesting, lets see what the third iteration is capable of :D
I’m excited to find out, have only just started seeing what it’s capable of!
nice vid
Thanks!
This is a really cool project. I am just beginning to learn Python. How would you suggest learning about AI applications through Python for a beginner? Thanks
Glad you like it! I really like the module PyTorch for neural networks, so I'd suggest just finding some fun/easy projects you can do in PyTorch! There's lots of great free stuff online to help learn
You should make the AI play against real people!
That's the dream for sure haha
Maybe it would be helpful if the AI got an example of a good run so it knows what it should or shouldn't do?
Where did you get the code for the game? I have the emulator but not the game itself
I cant code for shit but this sure is spaghetti code. Great vid
i love you!!!!!!!
This may be a dumb question but how does your A.I function different then the CPU players?
Make all your Ai,s race each other in Mario kart
100K subs? That's a pretty big long shot haha.
I'm ambitious haha
so far you've gotten AI to learn many things including Mario Kart and Mario Bros Wii, now we just gotta see it learn Unfair Mario lol
No comments? Let me change that
Thank you for your service!
to be clear, the Münchhausen disorder, is named after the ludicrous contents of the book
Oh I actually didn’t know that haha
In your search for emulators, did you come across an android emulator that can play mobile games that have the same features as dolphin?
No matter how smart Ai gets, I’ll still be able to wipe its @$$ in Mario Kart
bros PC is slower than my grandma's old laptop 💀
I have an rtx 4090 lol
GG
GG ez
Welcome to the emu club
Does the ai not uses a lot of rom ?
How do you not have more subs
Even 15k is crazy to me, but I do hope things continue to do well!
Why does it matter if the screenshots are square or not?
why don’t you use something like dreamer v2?
I actually did try getting that setup, but the publicly available code did not want to play nicely with my environment
@@aitango that’s unfortunate but i think dreamer fixes the observation per second bottleneck
😎👍
Is Mario Galaxy possible?
All I'll say is I'm looking into it :)
No game is truly impossible to train an AI to play
Wonder if you could try to do it on a real wii its possible to spoof a wii controller or wii mote over bluetooth. That way your not using system resources on the emulators. and instead use a few wiis since they can be found for cheap now.
100k plus subs this time next year.
I do hope so!
Make them race each other
Soon...
100 k ???
😮😳
Slightly ambitious from me I'll admit
but the wii has native 4:3 support..
That's true, and probably something I could've looked into in the first version. Once I figured out how to get any input size into the AI it wasn't a problem
Get this man to 100 k!...
Not cuz i want the source Code i Just want this man at 100 k subs LOL
how did you learn to code AI?
Thats not at all how AI works
@ChosenFate_ of course, absolutely *no* code was used
you dont "code an AI". He just made a python script that handles everything, but the "AI" itself is not something you code. @@RubixstewYT
Its true that I don't code what the AI does, but rather let it learn. However, learn to write the learning algorithm is something I learned through online tutorials. I did university courses on neural networks and programming, but I mostly still learnt this just in my spare time using the internet. I am however now doing a PhD in Reinforcement Learning, so I get to learn lots more about improving AI all day everyday
@aitango Thats really cool.
oi
Keep evolving it for the world record ai, then team up with a youtuber
If TWD98 would do a collab with me it would be like a dream come true
:)
:)
the ai itself is good but the videos feel extremely soulless. it feels like what you are passionate about is the ai itself and then making the video is an afterthought and a chore
We wanted the video to focus on the story of the AI journey being told and also show the training of the new AI in real time. All the edits are made to show important aspects of the journey but not take away from the training at the same time as that is what most people want to see. Thanks for the feedback, any way I can improve the videos would be helpful :)
Comment
:)
@@aitango this humble one learned from Grandmaster Spiff 🤓
That's some janky shit, buddy.
The first versions were for sure, the only reason I said I'd release the source code now is because the code is remotely clean!
It's Mario not Marry-o
Not in the UK haha
You did so much work for nothing, just take Dolphin with Lua and read data (checkpoints, road surface, items, position) directly from memory...
Less vocal fry please.
Going to be looking into the audio quality for future videos since there were a few comments about this
Neat, a video that says a lot but explains almost nothing
bro stop talking like that with your fake deep voice, it is so distracting
This video got many comments about the voice so I'm going to look into some audio quality things. As for the raw audio, that is my real voice... maybe I'll have to do a fake high voice
Can you please talk with normal voice instead of vocal fry? It's really annoying and I would have stopped watching if your project weren't as interesting as it is. Please consider this as constructive criticism, not an empty insult. Also, the volume of the video is very low compared to other videos, and I got an advertisement jump scare at the end.
There was another comment about this we will look into it and make changes for future videos. Noted on the audio levels also, I will fix that too. Thanks for the feedback
@@benji.botterill would it be possible to someday mod the game and replace the CPU with the AI? Amazing work btw!
I haven't looked into creating a mod (my experience in doing that is very limited), however would love to if there are any devs out there who would be willing to put some time into it. I have looked into potentially using the AI online which I also think would be really interesting. Also we'll definitely be looking into improving audio quality!