The AI in this video received drastically less reward for really high speeds compared to the other video where the AI sandbagged, but it still didn't seem to solve it haha. I guess it worked a little though as you mentioned since it didn't completely self-destruct!
@@aitangoMaybe teach the ai some basic strategies to evolve faster including taking shortcut with mushroom or when to turn/drift etc. You can also maybe cut down the unnecessary visual of the screen in order to maybe give him colour and higher resolution, including the lap and Time count.
@@aitangoI don't think the sandbagging is a reward problem its a lack of opponent difficulty problem. Assuming you are giving it more reward for being closer to first place, the fact that the CPU opponents don't pose enough of a challenge means the only way to get more reward when you know you can beat the CPUs with driving skills alone is to fish for items.
The fact the ai tend to pickup tactics that actual people sometimes use always interests me, that and seeing them forced into weird situations and still make their way out of them. Honestly plenty of ideas to be had even just for Mario Kart.
I love seeing AIs find tactics humans use. I remember years ago a project called AlphaZero which learned chess, and while playing it would learn human openings and then discard the ones it thought were bad haha
Glad you like it, I wasn't too sure how many people actually look at it! I like to include it though, just to give some idea of what the AI is "thinking"
I think a potentially better way to give the AI visual input would be to palletize the screen - that is, instead of reducing it to black and white, reduce it to say, a 32- or 64-color palette, so the AI just receives the color index of each pixel. That way it'd be easier for it to tell the difference between offroad and road, since they're usually very distinct colors, even if they're about the same shade. Although I could see palletizing every frame could be a bit more computationally expensive.
On another note, as you may be starting to recognize me as being more critical, I want to start by saying don't take it personally obviously. I do enjoy seeing what neural networks and machine learning is capable of, and these videos ARE entertaining. The criticism I have for today is the stock footage used in the intros of each video. It's not really useful for the video, and it's often the same clips used in previous videos, which can get a bit confusing, perhaps making people second guess whether they've already watched this particular video. I don't really have a proposed better solution, but I think just having you talk over gameplay rather than random stock footage would probably work fine.
One idea is, train an AI off using just no items with CPUs alone, and then add items while keeping the values for other actions the same. So it will know how to drive and can then deal with items
That's actually what I did in a previous video! (This AI went WILD) That works pretty well, the only issue is that I have to predetermine the number of actions before I create the AI, so I would have to give it an action for using items that it wouldn't be able to use for the first part. Another options I've been considering is training it in both situations at once simultaneously, but I really have no idea what effect that will have
@@aitangoYou could concurrently run a driving model and weapons handling model using the current convolutional input setup, but give each of them an additional input set with the other networks options and current preferred action
@@aitango you could use weight masking so that the model contains the weights and outputs needed for item usage, but won't actually train or use those weights for inference until the second phase.
As far as the input data to the AI goes, I somewhat recall that one semi-common strategy is to train an autoencoder on gameplay footage, and then use that autoencoder's encoded representation as input.
I really enjoy even just listening to how you set up the AI and watching it learn. It's also really cool to hear your retrospectives on what could have been improved to help the AI be better (like higher resolution might resolve the banana issue). Gives my a ton of interesting insight into creating AI's / neural networks. 😊
Your videos are truly amazing! As someone who is striving to learn about neural networks and machine learning, you manage to make the subject incredibly engaging and enjoyable. I wouldn't mind if you dedicated a small section of your videos to provide a more in-depth explanation of how to create an AI for a game. However, I understand that not everyone may share my interest in this topic.
I'm really glad you like the videos! In future videos would like to go a little more in technical depth, however I'm trying to figure out how to do so while still keeping lots of people interested!
Really glad you enjoyed the video! I'm still blown away by how well my last video did, and its been great to see so many people explore the rest of my channel after watching that video!
It's super fascinating to see the AI actually begin to adapt to item usage and even begin to overcome the blooper, which I always assumed would just be an insurmountable obstacle to the AI due to how it's designed to take in information, a number of things I'd suggest: 1) Increase the AI's viewpoint resolution, as you mentioned in the video the AI seemed to be severely handicapped by its available resolution and this made it very difficult for it to identify and avoid things such as bananas on the track even as it learned to do so, I don't believe this'd take a large upscaling and perhaps you could sidestep this issue by simply taking a snippet of the middle of the screen, upsizing it and moving it off to the side (Perhaps the top right corner since the AI doesn't seem to need to be able to view its lap count and the race timer) so the AI has a sort of "magnifying glass" it can reference for things further down the track without increasing the overall number of pixels the AI has to process, not really sure how well this would work through. 2) Give the AI a limited number of coloured pixels over its item box so that it can distinguish between green and red shells, this would give it a better idea of how these items are separate and what they both do, since right now it views them both as the same item due to their sprites being otherwise identical. 3) Change the item input so that it's a toggle hold button instead, firing the input once causes the AI to hold the current item behind itself and firing it again causes the AI to release that item, allowing it to defend itself from items and still use them without increasing the overall number of inputs the neutral network needs to process, although the AI would need to adapt to this process of using items and it'd probably have to learn the difference between when an item is being held behind itself and when it's not, so it might lengthen the learning process a bit. 4) This probably isn't practical either way but I figured I'd throw this in the suggestions regardless; giving the AI a reward for hitting an enemy with one of its items would massively improve its item usage by outright incentivising it to use and hit enemies with items, humans use the cheer sound the driver makes to know when they've hit someone, but obviously training the AI on sounds would exponentially increase the scope of this well beyond the realm of reality, so you'd probably need to figure out some way to access MKWii's internal variables to see when this cheer sound is playing and then wire that back to the AI. I really enjoyed this video, can't wait to see more content!
Thank you very much for the thoughtful comment! Increasing the AI's resolution is something I'm actually currently looking to do, however it does present some challenges to do this without massively tanking the training speed, but its a work in progress. Finding a way to distinguish between shells would be very useful; as the AI is setup only to accept black and white, but there may be a way that rgb can be converted to grayscale such that the resulting image appears differently. Changing item usage to a toggle will be done for sure, as this is pretty critical. I think I could track down the variable for hitting a CPU with an item, however the main problem I have is that every new track I play on will have a different memory address for that value, so it means finding it over and over again if I wanted to do multiple tracks (hint hint)
@@aitangoI was wondering that instead of increasing the resolution as it will increase the weights and make the model run much more slowly to instead input RGB frames at the same resolution, this will only triplicate the weights of the first conv layer leaving the rest of the architecture identical and shouldn't slow down the model too much. Using RGB frames of course makes it much easier for the model to discern bananas as they are a yellow thing on a gray background (the road), instead of being a shade of gray on another shade of gray. It will also fix the problem with the shells.
@@aitango (Red+Blue*255+Green*255*255)/16581375 Would give you "all" colors compressed into 256, reducing the amount of color fidelity by an extreme amount. Not the most efficient approach for sure, but would be fast even in a shader. The worries are though it will lose a different level of detail that is important. You could also consider giving it "FoV" detail differences like humans, with incrementally more detailed vision the closer to the center.
Yeah in lots of the videos it picked that up pretty quickly! I guess its just something that's easy to do "accidentally", and then it can learn to do it on purpose
Would be really nice to see a behind the scenes on how you are running this training and setup. Also tools and other things needed if someone would like to try it out on its own. Great video.
I'd really love to see this type of AI used in future games for people to play against, as I think it would provide such interesting and dynamic gameplay! Obviously the purpose of AI in games isn't to obliterate the human player, but Id be interested to see what could be done by taking different versions of the AI throughout training to vary the difficulty
great content. i am interested in this kind of stuff (as i believe alot of people are), and id even be interested in pure learning videoes regarding the neural network. Seeing a combination of games and teaching makes this very easy to consume and also fun to watch
Really glad you like it! I'm trying to find a good balance between explanation and watching the AI learn! I think in each video I'm going to try and discuss something new about how the whole thing works to keep it new and interesting!
Part of the learning process is figuring out what items do and learning how and when to use items to get a new one. Those are important skills for competitive racing.
To prove to people saying it just memorizes tracks try to make it play a different track after it’s done with training (maybe a similar track visually though)
Great job! I have a suggestion for the next AI, though I'm not sure if it would work: a reward for being in first place. This might negate the reward it gets for bagging, and encourage it to actually win the race faster.
I don't know anything about Mario Kart Wii's memory, but would there be a memory address you could read that would determine how far around the track you are? I know there's invisible checkpoints you have to hit so that the game correctly counts laps (mostly to prevent you from cheating), but a "you are X units away from hitting the next checkpoint" could help with knowing when to reset. There might also be a memory address that determines if the player has been hit by an item or not. That could also help the training program determine if it's not making progress due to shit driving, or not making progress because it's being smashed with items. That could allow faster resets to squeeze in more training. And of course if you want to cheat, you could just feed those to the AI so it'd "know" without having to analyze the screen to work out what its current state is. But these sorts of videos are always fascinating, especially yours. I tried playing around with RNN with Mario World on the SNES. After a bunch of hours of training, it could finish the first level, but as soon as it hit Yoshi's Island 3 (where it would have to move upwards before it could move forwards), it just got stuck endlessly, but I still had a blast doing it.
Considering the issues the AI has perceiving bananas, you could use a two-channel image stream (luma aka brightness and chroma aka hue) instead of a single channel image stream (grayscale) so it could more easily identify the distinct yellow hue of a banana. You could also consider training the AI in two separate pieces, a visual classifier/object tracker and the actual gaming agent. The object tracker simply identifies what's on the screen and where they are, using supervised learning. The gaming agent takes the interpretations of the object tracker and decides what controller inputs to use, using reinforcement learning as you've done here.
I like how the ai at the beginning is like on the off-road and just being (probably) happy about being able to drive on the off-road and not being punished for that
After my other videos where it get instantly deleted for doing that, I imagine it must be a relief! That said, its amazing how much it slows down training though, I guess harsh teaching has its advantages
@@aitango yeah, but let the ai sometime just mess around and keep it often just in the video! because it can sometimes show how every ai has its own will when doing that!
I like to think that the AI is actually having fun sandbagging. Basically, like any of us having fun, there's an activity that gives it a reward in its brain and it just focuses on doing that, even if it doesn't win
First we got AI learning to play sports, now we get AI learning to play MarioKart. This is amazing. All that skill and awesome work all these people do. Incredible.
I may not be too experienced in the field of AI, but I think it might be a good idea to give it a small reward every frame based on its position. I think this would help it learn to efficiently use its items to sabotage opponents.
Maybe you should make it distinguish between black and white and just the color yellow? That way it should be able to tell when he's about to bump into a banana
Super smash bros AI would be really cool, and you would have clear reward triggers(stocks lost or taken, percent given or taken), although there would be a lot of inputs. It would be really cool to see tho.
I'd really love to do this one, but it would be tough. As you mention, rewards are easy, but the biggest problems are the actions and screen input. The large number of actions would slow things down, but probably wouldn't cause too many issues. The biggest issue is the low resolution images the AI is given, since in smash fairly small details like which way your character is facing is really important and hard to see in a blurry image! If I find a decent way to use larger images though, I'll give it a go!
Love your videos! Glad I found your channel. One small bit of feedback-watching your last couple videos I wish the music was a bit quieter compared to your narration. For me at least it makes it harder to understand what you are saying.
I'd love to see an AI that just has the exact same controls that a human would have while playing. I know it would be a lengthy training process but itd be amazing
It would be interesting to try to help the AI changing a bit the architecture of the network to help it structure the data on screen better. Instead of feeding it just the raw pixel footage, you can use simple scripts or networks to more accurately and less effortlessly (in terms of computation required) give it a better idea of what's happening on screen. Some of the systems that may be insteresting to implement are: - Items: decide if it has an item (and which one it is) or if it's in the process of getting an item. With the 'rolling an item', you may be able to start seeing the AI tap the item button to get the item earlier (maybe even slightly reward it). - Item alerts: whether it is a red shell or a bullet bill coming for the AI, it doesn't have much time to accurately learn about how to handle the alerts it gets. Implementing a system that recognizes and classifies the alerts, then feeds the AI their position, type, time/frames since it appeared on screen and time/frames since it disappeared (the moment you want to take out your banana to block a red shell is the moment the alert disappears and shows on screen, as you didn't make the AI able to hold an item behind itself). - Position: it is rather easy to tell the numbers apart and give the position as an input to the network. Both this and the item think will probably make it so that the AI largely ignores pixels in those areas, focusing more on the meat of the game. - Map: the map info is probably hard to digest for the AI, so implementing a system focused in just doing that will probably help the AI anticipating turns and learning how to drive under blooper much faster. - Handling of noise: instead of always giving the AI the clean input of the screen, you can block some pixels/change their color. This guarantees the AI can learn to follow the track even if its view is obstructed by the CPUs or a blooper (maybe remove this noise when a blooper is detected). This is may be helpful for chaotic situations in which the AI seems to just prefer avoiding the chaos, fall back and get an item (which is risky, especially if there are a lot of drivers really far behind) instead of out-driving the other cars. - Items on the ground: pattern-match both harmul items in the ground and reward accordingly when avoided/hit. Good items can also be accounted for, like when the AI got shocked in the sand with mushrooms but instead of using the now-on-the-ground mushroom to go back to the sand, it simply drove normally through the sand back on the track. - Do item-specific races. To test the understanding of each item of the AI, you can force it to play with a reduced itemset (just shells, just bananas, just bombs...). It would be beneficial to do if you realized the AI can't learn an item properly (so force it to deal with it) or if you simply want to test some of the above mentioned systems. I hope you find my comment helpful. Great video! :)
About bloopers, I HATE THEM like, idk how people can drive while not seeing 80% of the screen, I always end up in a wall or falling somewhere every times there is a blooper, so the ai struggling with it is normal as actual humans do too
I can't wait for him to blow up so he can run three NASA 4090ti supercomputers in a chilled server and simulate an AI beating a Mario Kart online match.
Hey Tango! I’ve been following your channel for a while, and your A.I. Mario Kart videos have really caught my interest. I had some questions about the A.I. 1. I noticed you always use Funky Kong/Flame Rider combo for these videos. Would the A.I. preform the same if given a different character/vehicle combination? Why haven’t you tried using different combinations in your videos? If you’ve done it off-camera, I’d like to hear how that faired. 2. Seeing how you feed the A.I. information through B&W minimized frames, I’m curious what would happen if you did this in vs. Would it be possible to train 2 separate A.I., or even 3 or 4? It’d be interesting to see multiple A.I. trained to compete against each other, or even just seeing how a fully trained A.I. could stand a chance against a human player. 3. Do you plan to release the A.I. for others to experiment with? I’d enjoy racing against it, and I’m sure others would too, especially if it’s able to use multiple character/vehicle combinations. (Me personally, I use a lot of miis and I always find myself wishing the CPUs could choose miis.) I look forward to seeing what you’re able to achieve in the future, I will watch your career with great interest!
Have you experimented with giving direct access to the control stick x axis/drift button rather than hardcoding drift types? Or is that too hard for the AI to learn?
For item usage, perhaps aggressive play, reward the AI for every time it trips someone up with a banana, or hit with a shell, and give and deduct points for blocking shells and getting hit by them respectively. You could probably train it to dodge blue shells too using mushrooms
These are great! I wonder if you could overlay the minimaps to give an idea of how the Ai progresses over time? It could be cool to have that to show how far the first AIs made it in the time it took the final Ai to finish the course
Thanks! I did a livestream a while ago of the training process, but I’ve never done one for the actual code development. I never thought me programming for hours would be that interesting haha
The ai learning a technique even pros would use on its own is kind of astonishing, that would mean through the time it was still pretty bad, it picked up this technique by repeatedly being in last and getting absolutely nutty items. Still cant avoid a simple banana though, i guess even the strongest have the most mundane weaknesses.
I wonder what will happen if you had more than one AI learning. It would defiantly be interesting to see. Especially if the AI got more reward for being closer to first place so they actually race against eachother when both are skilled.
It could be cool (and I know the limit is resources) to take this AI, remove another CPU and replace it with another one of these AIs, then train the model again, then eventually replace another CPU with the newly trained AI, train again, and so on. This would really give the CPU a very deep and optimized understanding of the game.
Hey man love the videos. I'm doing my Masters in Computing Science and following some Machine Learning and Deep Learning courses next semester so I'll definitly watch your future videos. I was wondering how difficult (if even possible) it would be to create an AI that learns to play one track but can also perform decent on another track.
Fantastic video as always! I'd be curious to see how far an AI could learn before it plateau's if it were to race against other similar AI's, with rewards being given to the winners. With compeition on the same level as itself, it'd be forced to learn to play even better in order to come in first, yeah? ^^
I feel like it's a bit hard to call these an AI when you give them some control prompts or ways to recognize progress. What I'd like to see is to give an AI nothing more than the ability to press all the buttons on a controller just like how a human can do it (with some minor restrictions so they can't do impossible things like left + right at the same time). Then from there let the AI figure out everything so that it learns if it's doing well or not based on what's happening on the screen. Basically what you'd get if you gave a toddler a controller who has never played a single videogame before.
I'm amazed to see that the AIs have gone from just self-sabotaging for speed to learning how to bag efficiently
The AI in this video received drastically less reward for really high speeds compared to the other video where the AI sandbagged, but it still didn't seem to solve it haha. I guess it worked a little though as you mentioned since it didn't completely self-destruct!
@@aitangoMaybe teach the ai some basic strategies to evolve faster including taking shortcut with mushroom or when to turn/drift etc. You can also maybe cut down the unnecessary visual of the screen in order to maybe give him colour and higher resolution, including the lap and Time count.
the ai already knows when to turn and when to drift lol ?@@QnjtGWonQNqVsbYyzjx4
@@aitangoI don't think the sandbagging is a reward problem its a lack of opponent difficulty problem. Assuming you are giving it more reward for being closer to first place, the fact that the CPU opponents don't pose enough of a challenge means the only way to get more reward when you know you can beat the CPUs with driving skills alone is to fish for items.
@@Jackpkmn Smart observation
The fact the ai tend to pickup tactics that actual people sometimes use always interests me, that and seeing them forced into weird situations and still make their way out of them. Honestly plenty of ideas to be had even just for Mario Kart.
I love seeing AIs find tactics humans use. I remember years ago a project called AlphaZero which learned chess, and while playing it would learn human openings and then discard the ones it thought were bad haha
I really love the graph showing the AI's decision making in real-time and how confident it is in its decisions!
Glad you like it, I wasn't too sure how many people actually look at it! I like to include it though, just to give some idea of what the AI is "thinking"
@@aitango as a data analyst, I watch multiple parts of the videos two times for that reason
I think a potentially better way to give the AI visual input would be to palletize the screen - that is, instead of reducing it to black and white, reduce it to say, a 32- or 64-color palette, so the AI just receives the color index of each pixel. That way it'd be easier for it to tell the difference between offroad and road, since they're usually very distinct colors, even if they're about the same shade. Although I could see palletizing every frame could be a bit more computationally expensive.
On another note, as you may be starting to recognize me as being more critical, I want to start by saying don't take it personally obviously. I do enjoy seeing what neural networks and machine learning is capable of, and these videos ARE entertaining. The criticism I have for today is the stock footage used in the intros of each video. It's not really useful for the video, and it's often the same clips used in previous videos, which can get a bit confusing, perhaps making people second guess whether they've already watched this particular video. I don't really have a proposed better solution, but I think just having you talk over gameplay rather than random stock footage would probably work fine.
Also detect bananas
Also might help with the bananas
Also to see bananas
The fact the blooper actually did its job correctly at 5:20 is just incredible
It's awesome to see this creator get some traction. I've been a fan for a while now but I'm happy the previous video got ~900k views
8:46 THE AI IS BAGGING!!!!!!!!!!
One idea is, train an AI off using just no items with CPUs alone, and then add items while keeping the values for other actions the same. So it will know how to drive and can then deal with items
That's actually what I did in a previous video! (This AI went WILD) That works pretty well, the only issue is that I have to predetermine the number of actions before I create the AI, so I would have to give it an action for using items that it wouldn't be able to use for the first part. Another options I've been considering is training it in both situations at once simultaneously, but I really have no idea what effect that will have
Maybe you could add rewards for time spent in a high position and items in their proximity being destroyed
@@aitangoYou could concurrently run a driving model and weapons handling model using the current convolutional input setup, but give each of them an additional input set with the other networks options and current preferred action
@@aitango you could use weight masking so that the model contains the weights and outputs needed for item usage, but won't actually train or use those weights for inference until the second phase.
The AI reacted to the blooper the way Nintendo thought people would.
As far as the input data to the AI goes, I somewhat recall that one semi-common strategy is to train an autoencoder on gameplay footage, and then use that autoencoder's encoded representation as input.
lets go more mario kart ai!! i love these videos havent watched yet but ik itll be a certified BANGER
I really enjoy even just listening to how you set up the AI and watching it learn. It's also really cool to hear your retrospectives on what could have been improved to help the AI be better (like higher resolution might resolve the banana issue). Gives my a ton of interesting insight into creating AI's / neural networks. 😊
Your videos are truly amazing! As someone who is striving to learn about neural networks and machine learning, you manage to make the subject incredibly engaging and enjoyable. I wouldn't mind if you dedicated a small section of your videos to provide a more in-depth explanation of how to create an AI for a game. However, I understand that not everyone may share my interest in this topic.
I'm really glad you like the videos! In future videos would like to go a little more in technical depth, however I'm trying to figure out how to do so while still keeping lots of people interested!
You deserve so much more attention, I found one video and basically binged your whole channel. This video kicked ass, Tango.
Really glad you enjoyed the video! I'm still blown away by how well my last video did, and its been great to see so many people explore the rest of my channel after watching that video!
The AI learning to play semi competitively is completely unexpected imo
Fun fact: the circuit used is a mariokart version of the Monza Circuit, albeit without the chicanes and a banked Parabolica
It's super fascinating to see the AI actually begin to adapt to item usage and even begin to overcome the blooper, which I always assumed would just be an insurmountable obstacle to the AI due to how it's designed to take in information, a number of things I'd suggest:
1) Increase the AI's viewpoint resolution, as you mentioned in the video the AI seemed to be severely handicapped by its available resolution and this made it very difficult for it to identify and avoid things such as bananas on the track even as it learned to do so, I don't believe this'd take a large upscaling and perhaps you could sidestep this issue by simply taking a snippet of the middle of the screen, upsizing it and moving it off to the side (Perhaps the top right corner since the AI doesn't seem to need to be able to view its lap count and the race timer) so the AI has a sort of "magnifying glass" it can reference for things further down the track without increasing the overall number of pixels the AI has to process, not really sure how well this would work through.
2) Give the AI a limited number of coloured pixels over its item box so that it can distinguish between green and red shells, this would give it a better idea of how these items are separate and what they both do, since right now it views them both as the same item due to their sprites being otherwise identical.
3) Change the item input so that it's a toggle hold button instead, firing the input once causes the AI to hold the current item behind itself and firing it again causes the AI to release that item, allowing it to defend itself from items and still use them without increasing the overall number of inputs the neutral network needs to process, although the AI would need to adapt to this process of using items and it'd probably have to learn the difference between when an item is being held behind itself and when it's not, so it might lengthen the learning process a bit.
4) This probably isn't practical either way but I figured I'd throw this in the suggestions regardless; giving the AI a reward for hitting an enemy with one of its items would massively improve its item usage by outright incentivising it to use and hit enemies with items, humans use the cheer sound the driver makes to know when they've hit someone, but obviously training the AI on sounds would exponentially increase the scope of this well beyond the realm of reality, so you'd probably need to figure out some way to access MKWii's internal variables to see when this cheer sound is playing and then wire that back to the AI.
I really enjoyed this video, can't wait to see more content!
Thank you very much for the thoughtful comment! Increasing the AI's resolution is something I'm actually currently looking to do, however it does present some challenges to do this without massively tanking the training speed, but its a work in progress. Finding a way to distinguish between shells would be very useful; as the AI is setup only to accept black and white, but there may be a way that rgb can be converted to grayscale such that the resulting image appears differently. Changing item usage to a toggle will be done for sure, as this is pretty critical. I think I could track down the variable for hitting a CPU with an item, however the main problem I have is that every new track I play on will have a different memory address for that value, so it means finding it over and over again if I wanted to do multiple tracks (hint hint)
@@aitangoI was wondering that instead of increasing the resolution as it will increase the weights and make the model run much more slowly to instead input RGB frames at the same resolution, this will only triplicate the weights of the first conv layer leaving the rest of the architecture identical and shouldn't slow down the model too much. Using RGB frames of course makes it much easier for the model to discern bananas as they are a yellow thing on a gray background (the road), instead of being a shade of gray on another shade of gray. It will also fix the problem with the shells.
@@aitango (Red+Blue*255+Green*255*255)/16581375
Would give you "all" colors compressed into 256, reducing the amount of color fidelity by an extreme amount.
Not the most efficient approach for sure, but would be fast even in a shader.
The worries are though it will lose a different level of detail that is important.
You could also consider giving it "FoV" detail differences like humans, with incrementally more detailed vision the closer to the center.
Another cool thing that the AI was doing is dodging POWs
Yeah in lots of the videos it picked that up pretty quickly! I guess its just something that's easy to do "accidentally", and then it can learn to do it on purpose
@@aitango Also thanks for reporting what the model takes as input, in the other video there was a debate about it ahah
Oh yeah probably best to settle that haha
Wow, I did not realise that your last video got so many views. Congratulations on your efforts finally paying off after a longer absence :)
Thank you very much!
What if you took all of your Mario cart AI's and put them all in a race against each other
That would be utter mayhem haha. I'm hoping to set something like that up at some point, but its definitely going to take a little while!
Would be really nice to see a behind the scenes on how you are running this training and setup. Also tools and other things needed if someone would like to try it out on its own. Great video.
Honestly wasn’t expecting this, all those calculations and RNG calls that has to be made are insane! Surely this is the way forward for future games!
I'd really love to see this type of AI used in future games for people to play against, as I think it would provide such interesting and dynamic gameplay! Obviously the purpose of AI in games isn't to obliterate the human player, but Id be interested to see what could be done by taking different versions of the AI throughout training to vary the difficulty
great content. i am interested in this kind of stuff (as i believe alot of people are), and id even be interested in pure learning videoes regarding the neural network. Seeing a combination of games and teaching makes this very easy to consume and also fun to watch
Really glad you like it! I'm trying to find a good balance between explanation and watching the AI learn! I think in each video I'm going to try and discuss something new about how the whole thing works to keep it new and interesting!
You are making me wanna get onto smaller scale silly AI projects again, and I love you for it. Keep up the beautiful AI stuff my friend!
Part of the learning process is figuring out what items do and learning how and when to use items to get a new one. Those are important skills for competitive racing.
Ah yes, I loved the last items AI video you did, I’m glad you released this one too. Keep up the good work!! 🎉
To prove to people saying it just memorizes tracks try to make it play a different track after it’s done with training (maybe a similar track visually though)
awesome video keep up the great work!
Thanks so much, really glad you enjoyed!
great video I love all of your work since the last year
This gota be good
Your editing and narration got even better !
Congrata on the great video
great video, great editing, keep it up
Great job! I have a suggestion for the next AI, though I'm not sure if it would work: a reward for being in first place. This might negate the reward it gets for bagging, and encourage it to actually win the race faster.
Finally, someone who uses visual input in these things
I think letting the AI read from memory is cheating haha
I don't know anything about Mario Kart Wii's memory, but would there be a memory address you could read that would determine how far around the track you are? I know there's invisible checkpoints you have to hit so that the game correctly counts laps (mostly to prevent you from cheating), but a "you are X units away from hitting the next checkpoint" could help with knowing when to reset.
There might also be a memory address that determines if the player has been hit by an item or not. That could also help the training program determine if it's not making progress due to shit driving, or not making progress because it's being smashed with items. That could allow faster resets to squeeze in more training.
And of course if you want to cheat, you could just feed those to the AI so it'd "know" without having to analyze the screen to work out what its current state is.
But these sorts of videos are always fascinating, especially yours. I tried playing around with RNN with Mario World on the SNES. After a bunch of hours of training, it could finish the first level, but as soon as it hit Yoshi's Island 3 (where it would have to move upwards before it could move forwards), it just got stuck endlessly, but I still had a blast doing it.
I’d love to see how this AI performs on a track it’s never seen before, if it would even work at all. Great video!
Coming Soon...
@@aitangolet me guess: teaching it to identify what could be considered a road so it can be more adaptable.
Considering the issues the AI has perceiving bananas, you could use a two-channel image stream (luma aka brightness and chroma aka hue) instead of a single channel image stream (grayscale) so it could more easily identify the distinct yellow hue of a banana.
You could also consider training the AI in two separate pieces, a visual classifier/object tracker and the actual gaming agent. The object tracker simply identifies what's on the screen and where they are, using supervised learning. The gaming agent takes the interpretations of the object tracker and decides what controller inputs to use, using reinforcement learning as you've done here.
Well said
I like how the ai at the beginning is like on the off-road and just being (probably) happy about being able to drive on the off-road and not being punished for that
After my other videos where it get instantly deleted for doing that, I imagine it must be a relief! That said, its amazing how much it slows down training though, I guess harsh teaching has its advantages
@@aitango yeah, but let the ai sometime just mess around and keep it often just in the video! because it can sometimes show how every ai has its own will when doing that!
I like to think that the AI is actually having fun sandbagging. Basically, like any of us having fun, there's an activity that gives it a reward in its brain and it just focuses on doing that, even if it doesn't win
First we got AI learning to play sports, now we get AI learning to play MarioKart. This is amazing.
All that skill and awesome work all these people do. Incredible.
Thank you so much, always amazing to hear such kind support!
Next I’d love to see the AI trained on a multitude of tracks and see if it can win a full grand prix
Bro how is your channel this underrated?
Thanks haha, although I'm not sure its still underrated after how my last video did!
Im exited!
Can't wait for the next one! Keep it up man!
This is really cool! Please let it play a harder course
I may not be too experienced in the field of AI, but I think it might be a good idea to give it a small reward every frame based on its position. I think this would help it learn to efficiently use its items to sabotage opponents.
Maybe you should make it distinguish between black and white and just the color yellow? That way it should be able to tell when he's about to bump into a banana
Imagine the hit that AI got the first time it used that bullet bill. Lol.
The high of a lifetime haha
I would love to see how the AI behaves when given more control over items! great video man
Really glad you liked it! When I do another video that uses items, I'll come up with an improved system for sure!
holy moly its a sequel, most tubers would have just not done anything for a year and then redo the original video and play it off as a sequel
Super smash bros AI would be really cool, and you would have clear reward triggers(stocks lost or taken, percent given or taken), although there would be a lot of inputs. It would be really cool to see tho.
I'd really love to do this one, but it would be tough. As you mention, rewards are easy, but the biggest problems are the actions and screen input. The large number of actions would slow things down, but probably wouldn't cause too many issues. The biggest issue is the low resolution images the AI is given, since in smash fairly small details like which way your character is facing is really important and hard to see in a blurry image! If I find a decent way to use larger images though, I'll give it a go!
Love your videos! Glad I found your channel. One small bit of feedback-watching your last couple videos I wish the music was a bit quieter compared to your narration. For me at least it makes it harder to understand what you are saying.
Neat analysis video! Thanks for uploading!
A good idea for the next video is to have more ai race against eachother
I'd love to see an AI that just has the exact same controls that a human would have while playing. I know it would be a lengthy training process but itd be amazing
oh hell yeah, can't WAIT!
You should try randomizing the tracks after every attempt and leave it running for a very long time to create an ai that can play all of the tracks
It would be interesting to try to help the AI changing a bit the architecture of the network to help it structure the data on screen better. Instead of feeding it just the raw pixel footage, you can use simple scripts or networks to more accurately and less effortlessly (in terms of computation required) give it a better idea of what's happening on screen. Some of the systems that may be insteresting to implement are:
- Items: decide if it has an item (and which one it is) or if it's in the process of getting an item. With the 'rolling an item', you may be able to start seeing the AI tap the item button to get the item earlier (maybe even slightly reward it).
- Item alerts: whether it is a red shell or a bullet bill coming for the AI, it doesn't have much time to accurately learn about how to handle the alerts it gets. Implementing a system that recognizes and classifies the alerts, then feeds the AI their position, type, time/frames since it appeared on screen and time/frames since it disappeared (the moment you want to take out your banana to block a red shell is the moment the alert disappears and shows on screen, as you didn't make the AI able to hold an item behind itself).
- Position: it is rather easy to tell the numbers apart and give the position as an input to the network. Both this and the item think will probably make it so that the AI largely ignores pixels in those areas, focusing more on the meat of the game.
- Map: the map info is probably hard to digest for the AI, so implementing a system focused in just doing that will probably help the AI anticipating turns and learning how to drive under blooper much faster.
- Handling of noise: instead of always giving the AI the clean input of the screen, you can block some pixels/change their color. This guarantees the AI can learn to follow the track even if its view is obstructed by the CPUs or a blooper (maybe remove this noise when a blooper is detected). This is may be helpful for chaotic situations in which the AI seems to just prefer avoiding the chaos, fall back and get an item (which is risky, especially if there are a lot of drivers really far behind) instead of out-driving the other cars.
- Items on the ground: pattern-match both harmul items in the ground and reward accordingly when avoided/hit. Good items can also be accounted for, like when the AI got shocked in the sand with mushrooms but instead of using the now-on-the-ground mushroom to go back to the sand, it simply drove normally through the sand back on the track.
- Do item-specific races. To test the understanding of each item of the AI, you can force it to play with a reduced itemset (just shells, just bananas, just bombs...). It would be beneficial to do if you realized the AI can't learn an item properly (so force it to deal with it) or if you simply want to test some of the above mentioned systems.
I hope you find my comment helpful. Great video! :)
Now make him compete in the cups on 150cc!!
Perhaps multiple tracks will be coming soon... :)
We all know this is gonna be good.
Day 2 of asking for a race of 12 separately trained AIs on a track none of them have ever seen before
After watching this video, I now finally know the use of the blooper item
I just watched the other, and seent he comment asking for this. I HAD to see if you did the video. Low and behold. Better turn on notis.
About bloopers, I HATE THEM like, idk how people can drive while not seeing 80% of the screen, I always end up in a wall or falling somewhere every times there is a blooper, so the ai struggling with it is normal as actual humans do too
Love the content. Can’t wait for more.
This channel will blow up.
I can't wait for him to blow up so he can run three NASA 4090ti supercomputers in a chilled server and simulate an AI beating a Mario Kart online match.
New challenge: AI learns how to play Smash Brawl through neuronetworking like those amiibo fighters in Smash Ultimate
Really neat video, im would be curious to see adverserial networks battle 1v1, although it would probably take a lot of compute
Hey Tango! I’ve been following your channel for a while, and your A.I. Mario Kart videos have really caught my interest. I had some questions about the A.I.
1. I noticed you always use Funky Kong/Flame Rider combo for these videos. Would the A.I. preform the same if given a different character/vehicle combination? Why haven’t you tried using different combinations in your videos? If you’ve done it off-camera, I’d like to hear how that faired.
2. Seeing how you feed the A.I. information through B&W minimized frames, I’m curious what would happen if you did this in vs. Would it be possible to train 2 separate A.I., or even 3 or 4? It’d be interesting to see multiple A.I. trained to compete against each other, or even just seeing how a fully trained A.I. could stand a chance against a human player.
3. Do you plan to release the A.I. for others to experiment with? I’d enjoy racing against it, and I’m sure others would too, especially if it’s able to use multiple character/vehicle combinations. (Me personally, I use a lot of miis and I always find myself wishing the CPUs could choose miis.)
I look forward to seeing what you’re able to achieve in the future, I will watch your career with great interest!
The music was way too loud Bro, anyways nice vid! Love it
We need to get this legend a proper office.
Have you experimented with giving direct access to the control stick x axis/drift button rather than hardcoding drift types? Or is that too hard for the AI to learn?
For item usage, perhaps aggressive play, reward the AI for every time it trips someone up with a banana, or hit with a shell, and give and deduct points for blocking shells and getting hit by them respectively. You could probably train it to dodge blue shells too using mushrooms
your videos are awesome ! keep it up !!!
The AI learned how to item smuggle, holy crap!
These are great! I wonder if you could overlay the minimaps to give an idea of how the Ai progresses over time? It could be cool to have that to show how far the first AIs made it in the time it took the final Ai to finish the course
My new favorite channel ❤
"AI will take over the world!!"
"hehe funny ai get destroyed in mario cart"
Nice video! In my opinion would be much interesting i you did a livestream wich shows all the process that takes you to create an ai like this
Thanks! I did a livestream a while ago of the training process, but I’ve never done one for the actual code development. I never thought me programming for hours would be that interesting haha
I’m so excited
The ai learning a technique even pros would use on its own is kind of astonishing, that would mean through the time it was still pretty bad, it picked up this technique by repeatedly being in last and getting absolutely nutty items. Still cant avoid a simple banana though, i guess even the strongest have the most mundane weaknesses.
Bro ai now knows item bagging, they be ballin out now
So much awesome. Thank you again.
I love these vids, just wished they’d be longer.
The way the AI got better than i am in mktwii 🤡
I’ve always wanted to make AI to play games like this before. Do you have any good places to look at to start learning this kind of thing?
I wonder what will happen if you had more than one AI learning. It would defiantly be interesting to see. Especially if the AI got more reward for being closer to first place so they actually race against eachother when both are skilled.
You should use one of the pretrained ais on another track so it can learn to distinguish between different road types
I was expecting the AI to just get beat up by other cpus because it doesn’t know how to use items
It could be cool (and I know the limit is resources) to take this AI, remove another CPU and replace it with another one of these AIs, then train the model again, then eventually replace another CPU with the newly trained AI, train again, and so on. This would really give the CPU a very deep and optimized understanding of the game.
Hey man love the videos. I'm doing my Masters in Computing Science and following some Machine Learning and Deep Learning courses next semester so I'll definitly watch your future videos. I was wondering how difficult (if even possible) it would be to create an AI that learns to play one track but can also perform decent on another track.
Thanks, and good to hear more people are getting into Deep Learning! Maybe you'll find out in a future video :)
@@aitango I'm excited
it should get a reward for every second it spends in 1st place
Fantastic video as always! I'd be curious to see how far an AI could learn before it plateau's if it were to race against other similar AI's, with rewards being given to the winners. With compeition on the same level as itself, it'd be forced to learn to play even better in order to come in first, yeah? ^^
i would love to see an AI that was playing on a track try to play on another track to see how it goes!
I feel like it's a bit hard to call these an AI when you give them some control prompts or ways to recognize progress.
What I'd like to see is to give an AI nothing more than the ability to press all the buttons on a controller just like how a human can do it (with some minor restrictions so they can't do impossible things like left + right at the same time).
Then from there let the AI figure out everything so that it learns if it's doing well or not based on what's happening on the screen. Basically what you'd get if you gave a toddler a controller who has never played a single videogame before.
something to consider would be not only the position of the player but also the distance away it is from others
you need to find some volunteers who have never played mario kart, sit them down and compare their learning curve to the AI.
That would be very cool to see
Mecha-Funky is slowly getting stronger and stronger...
now teach it to outrun blue shells for an entire race >:)
Yeah some things in Mario Kart never change haha. If it learned to dodge one I think I'd throw a party though