I'd love to see you pick say 5 or 6 different characters in a fighting game, train AI for each one and train them against each other and then invite some fighting game pro to play. Also be interested to know what the effect of simulating human reaction time would have on the effectiveness of the model for example random 200 to 300ms delay on when an action gets performed maybe?
Yeah doing self-play or play vs human is a really interesting idea, and something I'd like to get into soon. Simulating human reaction time is always really impressive (like alphastar), however I also love seeing the AI abuse its cracked reaction time
@@aitango another idea is to use online replays. For example in street fighter 5, you can watch top player matches, inputs can be showed on the side too
See if you can train it against real players or like bring in a real pro player at the end. See how training against CPUs creates weaknesses up against real considerate fighting game players. I think that would be more interesting than just Arcade mode
I've always wondered what an AI would do in a management/tycoon sim game like Democracy 3. Could it 'solve' politics, after some solid deep reinforcement learning?
sounds interesting to me too. i think the challenge there would be creating the reward system although it would be interesting to see what ai tango thinks. What I mean is an ai might take an action and you give it a reward because you think the action is good but then somewhere down the line that action results in a collapse of their business and they loose a reward, but it is very difficult for an ai to link that negative reward way back into the past. Alternatively you could do sparse rewards where you only give a reward at major "checkpoints" in the general game progression, but with so many variables for the ai to get there, it might also be difficult for it to know which variables and decisions helped the most.
You should leave the AI running until it stops improving. You show the charts of time and improvement in these videos, but I want to see what the plateaus in improvement are.
I noticed the AI didn’t use combos or many special moves. Is this because it’s reacting to the CPU player’s movement and deciding the best way to counter? Awesome stuff!
Extremely intriguing watching different platforms for your AI. AlphaStar on StarCraft was impressive but how long before launching did the AI train to be champion? Poker AI would be incredibly interesting to watch unfold. 😉
An AI for Spore would be pretty interesting and probably quite easy to set up the rewards for as its just a green bar at the bottom of the screen! Wonder how a bot trapped as a living creature would behave
Would it be more efficient to pass in the latents of previous frames rather than pass in the frames themselves? You could save on feature extraction layers as the model can leverage work that has already been done from the first time it encountered the frame.
Yeah the idea of using recurrent nets it reinforcement learning has been done successfully, however has some downsides. When training, it requires you to repeat the same sequence to get the latents, which can be expensive since training is usually done from random samples, not in order
This is amazing! What I'd _ESPECIALLY_ love to see is an AI learning to conquer the input-reading, literal-cheating, brutal AI of Mortal Kombat 2 - If it can do that, then I can imagine they'd make the perfect training partner Player 2!
I'd love to see how much this AI could learn on it's own given a bit more time! Armagedon is a game with an incredibly high skill celing, and I'd love to see how long it takes for it to learn some basics like special moves and parries, all the way to things like infinites
i saw u had melee. fizzi the creator of slippi tried to make a competition of ia playing melee but didnt had enough entrance for it. u could maybe be one more i think. Also about melee u have so mich tool for it. U have an hardcoded "ai" ( in the sens of cpu player) that always choose the best action regardint opponent actions, it could be really usefull to train your ia once she learn the basic oof the game
One time a friend used cheats on MK3 for SNES to make 2 bots fight with healing and unlimited time. It went on forever. We also used ro set up Super Smash Bros bots against each other this way.
This would have been great with Killer Instinct, as it's even more combo focused than what Mortal Kombat is. If I had the patience, I would build an AI and try it myself, but alas, my skills lie in desktop and web apps, and not the more mathematical side of things.
This is amazing! Is possible to use this to make cpu vs cpu in Mario Party 9 and others like Mario Party DS, 10, Island Tour... That would be very impressive!
I really would like to wacht the ai learn to play games with more complex combo systems like skullgirls, guilty gear or some game like that and then drop the ai to online matches
This AI was limited in that it could only press one key at once, meaning it couldn’t execute many combos. In the future I may looking into allowing it to perform some crazy combos!
This video makes me wonder what it'd be like if you made an AI train for a more unpredictable kind of fighting game. Maybe you could train them in M.U.G.E.N.'s Survival Mode with a bunch of random character from all over the internet, and seeing how high of a score it can get, as well as if it can beat some of the toughest AI that the community has to offer (there are some that give even pro players a run for their money). Though I also wonder how they'd play at other unpredictable and random games, like Inscryption for example.
how do you batch the learning using noisy parameters? you have a list of states, actions and rewards. when you sample them, I assume you do batch computing by using a matrix as input instead of just a vector. but with each sample having random variances applied to the weight to produce a random action, you cant use the same weight to do matrix multiplication right? you would need to do multiple vector matrix multiplications with each weight matrix accounting for the different Gaussian * variance?
Can you get the AI to play against itself? Maybe that would make it improve even faster. Also, just wondering, if someone wanted to get into making these AIs, what would you recommend?
Ight now interested can you do one with the game F zero gx? The course phantom road is one of the hardest even cpu use hacks to actually finish the race, in that gane is imposible to finish in one place in that track
@@aitango same, guy just has a way of making you question the totality of reality without an existential crisis. now if you excuse me, i have to get back to reaching my goal of unlimited Ricepudding
I'd love to see you pick say 5 or 6 different characters in a fighting game, train AI for each one and train them against each other and then invite some fighting game pro to play. Also be interested to know what the effect of simulating human reaction time would have on the effectiveness of the model for example random 200 to 300ms delay on when an action gets performed maybe?
Yeah doing self-play or play vs human is a really interesting idea, and something I'd like to get into soon. Simulating human reaction time is always really impressive (like alphastar), however I also love seeing the AI abuse its cracked reaction time
@@aitango another idea is to use online replays. For example in street fighter 5, you can watch top player matches, inputs can be showed on the side too
@@aitango😮🎉
How bout the AI learn some Tekken? That would be hella fun to watch.
This will be funny and let the ai play Jack too lol
See if you can train it against real players or like bring in a real pro player at the end. See how training against CPUs creates weaknesses up against real considerate fighting game players. I think that would be more interesting than just Arcade mode
Yeah very true, I may do another video at some point where I make it fight some humans
This channel is goated! Keep it up bro
Appreciate it, will do!
I've always wondered what an AI would do in a management/tycoon sim game like Democracy 3. Could it 'solve' politics, after some solid deep reinforcement learning?
sounds interesting to me too. i think the challenge there would be creating the reward system although it would be interesting to see what ai tango thinks.
What I mean is an ai might take an action and you give it a reward because you think the action is good but then somewhere down the line that action results in a collapse of their business and they loose a reward, but it is very difficult for an ai to link that negative reward way back into the past. Alternatively you could do sparse rewards where you only give a reward at major "checkpoints" in the general game progression, but with so many variables for the ai to get there, it might also be difficult for it to know which variables and decisions helped the most.
To me it would just find an exploit and abuse it
Bro really decided to go cinematic for this one
You should leave the AI running until it stops improving. You show the charts of time and improvement in these videos, but I want to see what the plateaus in improvement are.
You shoulda tried to play against it at the end haha. Great video as always!
Definitely want to do something like this in a future video!
I noticed the AI didn’t use combos or many special moves. Is this because it’s reacting to the CPU player’s movement and deciding the best way to counter?
Awesome stuff!
It just simply didn't learn to do them.
I could see an Ai tournament happening with this one. Nice video. 😁
Really glad you liked it! Would love to see a tournament too!
Thats a very high pun per minute rate, I love it^^
What a brilliant metric to use haha
@@aitango could u make an ai learn how to play Sifu?😮
@@aitango yo u could do an only up ai or something like that
Now you've gotta have it train against itself with the ulimate challenge being a real player
Extremely intriguing watching different platforms for your AI. AlphaStar on StarCraft was impressive but how long before launching did the AI train to be champion? Poker AI would be incredibly interesting to watch unfold. 😉
Poker AI has been done long ago! They held a big tournament around 2018 and AI crushed humans haha
It didn't even get to learn some special moves and combos, interestingly.
Oh god the AI does frame perfect dodges too? What's next, it wrong warps to the SM64 bowser fight by jumping under the bridge?
More AI, more Bloodshed! Teaching ai how to obliterate opponents sure won’t have any repercussions or anything! 👍
Of course not, don’t see how it could go wrong
9:59 LMAO
hehehe
It’s been a while I genuinely I burst out laughing like that, until this very moment
DMC: AI vs Bloody Palace. Primary Incentive: Style Points.
An AI for Spore would be pretty interesting and probably quite easy to set up the rewards for as its just a green bar at the bottom of the screen! Wonder how a bot trapped as a living creature would behave
This is why the MK team never created a Cyber Scorpion
10:00 Yooooo! Caught me off guard 😂
Would it be more efficient to pass in the latents of previous frames rather than pass in the frames themselves? You could save on feature extraction layers as the model can leverage work that has already been done from the first time it encountered the frame.
Yeah the idea of using recurrent nets it reinforcement learning has been done successfully, however has some downsides. When training, it requires you to repeat the same sequence to get the latents, which can be expensive since training is usually done from random samples, not in order
@@aitangoah, *part* of the reason Transformers took over then?
Tekken next, please, and let the ai learn jack😂
This is amazing!
What I'd _ESPECIALLY_ love to see is an AI learning to conquer the input-reading, literal-cheating, brutal AI of Mortal Kombat 2 - If it can do that, then I can imagine they'd make the perfect training partner Player 2!
Having it fight in ranked multiplayer would be crazy interesting
It would be curb stomped
I'd love to see how much this AI could learn on it's own given a bit more time! Armagedon is a game with an incredibly high skill celing, and I'd love to see how long it takes for it to learn some basics like special moves and parries, all the way to things like infinites
I never expected this, but I enjoyed it :D
i saw u had melee. fizzi the creator of slippi tried to make a competition of ia playing melee but didnt had enough entrance for it. u could maybe be one more i think. Also about melee u have so mich tool for it. U have an hardcoded "ai" ( in the sens of cpu player) that always choose the best action regardint opponent actions, it could be really usefull to train your ia once she learn the basic oof the game
Would it be possible to make an A.I. for Smash Bros?
I actually tried it a long while back but didn’t have much luck. I’m convinced I could give it a good shot now though haha
it would be cool to see two different AI learning models fight and train against each other in mortal combat or smash bros.
Should’ve taught the AI how to do air infinites and parry cancels
Super Smash Bros Brawl when?
One time a friend used cheats on MK3 for SNES to make 2 bots fight with healing and unlimited time. It went on forever. We also used ro set up Super Smash Bros bots against each other this way.
This would have been great with Killer Instinct, as it's even more combo focused than what Mortal Kombat is. If I had the patience, I would build an AI and try it myself, but alas, my skills lie in desktop and web apps, and not the more mathematical side of things.
Hey! Love the videos! Would you mind showing the full distribution instead of the expected return as well as the dueling network parts?
he is going for high ppm (pun per minute)
My new favourite metric
This is amazing! Is possible to use this to make cpu vs cpu in Mario Party 9 and others like Mario Party DS, 10, Island Tour... That would be very impressive!
would love to see a tutorial for coding deep reinforcement learning on games
I still wanna see Marvel vs. Capcom 2. . .
Great job. Do you have repo where you show any of the process?
I really would like to wacht the ai learn to play games with more complex combo systems like skullgirls, guilty gear or some game like that and then drop the ai to online matches
Next step: Rewards for successfully chaining combos together?
This AI was limited in that it could only press one key at once, meaning it couldn’t execute many combos. In the future I may looking into allowing it to perform some crazy combos!
Banger
Thanks, glad you enjoyed it!
Great video, this was very interesting
Glad you enjoyed it!
This video makes me wonder what it'd be like if you made an AI train for a more unpredictable kind of fighting game. Maybe you could train them in M.U.G.E.N.'s Survival Mode with a bunch of random character from all over the internet, and seeing how high of a score it can get, as well as if it can beat some of the toughest AI that the community has to offer (there are some that give even pro players a run for their money). Though I also wonder how they'd play at other unpredictable and random games, like Inscryption for example.
Id love to see this done with Tekken 8, it would be fun to give the ai acess to stuff like frame data and hitboxes with some sprt of mod
how do you batch the learning using noisy parameters? you have a list of states, actions and rewards. when you sample them, I assume you do batch computing by using a matrix as input instead of just a vector. but with each sample having random variances applied to the weight to produce a random action, you cant use the same weight to do matrix multiplication right? you would need to do multiple vector matrix multiplications with each weight matrix accounting for the different Gaussian * variance?
Can you get the AI to play against itself? Maybe that would make it improve even faster.
Also, just wondering, if someone wanted to get into making these AIs, what would you recommend?
Nice video, can you try letting it play something that has a linear shooter campaign like CoD or something similar ?
Is it possible to train the AI in a virtual remote gpu (such as colab) if you don’t have a machine with a gpu powerful enough?
Are the hours including the 4 environments? So like 30 hours is actually 120 hours of gameplay?
The hours is real time, so in terms of playtime it was significantly more since I use 4 environments, and I uncap the frame rate
@@aitango ah I see forgot about the framerate thing also. thanks for the info.
What software is he using to do all this AI stuff ? and How could I run it on my PC ?
He is using Dolphin Emulator to run WII games on a PC
You know, if you want to use ai learning to destroying mugen and is could be ai vs cpus since mugen had it.
Ight now interested can you do one with the game F zero gx? The course phantom road is one of the hardest even cpu use hacks to actually finish the race, in that gane is imposible to finish in one place in that track
Is there a way to see your code mr tango sir?
👏👏👏
☺️
someone here watching exurb1a
Was wondering if anyone would pick up on that hahaha, honestly one of my favourite videos on youtube
@@aitango same, guy just has a way of making you question the totality of reality without an existential crisis.
now if you excuse me, i have to get back to reaching my goal of unlimited Ricepudding
Please!! cloud you trainn an AI to play king of fighters 2002? 🙏🙏🙏🙏🙏
You should teach an ai to play super smash bros
Can you do a league of legends ai?
do smach bros mele next
I looked at doing smash bros wii a while back... didn't have much luck at the time, but maybe I'll give it another go
nice viedeo, and really funny comparisons like "with lower health than a 90 year old guy"😂
Do smash bros next!
Ai sifu?
Haha been watching JoJo?
I haven't, maybe my editor has haha
stop attempting to be humorous.
:(
it's the chatgpt script lol. he should try being more authentic and it would make the video way more watchable