Thanks for watching this video ! This is the first time i'm using NEAT algorithm, so there is obviously still room for improvement. The main problem is that my AI doesn't have a map memory, and can't anticipate "what comes next" with its current inputs. I have some ideas to improve my AI, so don't forget to subscribe if you want to see the next steps of this project ;)
Could you in the later generations select the ones that traveled the shortest distance at the last checkpoint? This would make them use the racing line more closely wouldn’t it? And then you could pick out the ones that don’t hit the walls. Not sure if that’s at all possible but it would make sense to me
in this vid i saw fast ai that started bouncing of the walls at a later point getting over taken by a slower ai. to create fast ai i needs map knowledge and learn that racing that keeps as much speed as possible and a bump on the wall slows you down. So basically all ai need to know what is faster in much smaller sections than a few checkpoints and then combine them to fastest to checkpoint. it's not going to be easy to create fast driving self learn ai.
neat optimizes the topology of the model, if i remember it correctly you don't train the weights of the model. ai.googleblog.com/2019/08/exploring-weight-agnostic-neural.html github.com/google/brain-tokyo-workshop/tree/master/WANNRelease/prettyNEAT
@@evgeny8578 this is trackmania not a simulator so basically there’s a technique called speed drift by holding brake from about half a second while turning the car enters a drift where it gains speed, I was weird game physics but it’s a part the games identity now
Try this: if the car hits the wall, then remove a point. This should make the AI learn faster because hitting the walls will let them learn to not hit walls.
@@thedoctor0892 this would be better, it has an incentive to accelerate. Please note that, if feasible, it might converge to a circular path, which would represent a very appealing local maximum
@@khantatat Then in this scenario, there should be a limited time for the AI to reach the finish line, this would then force the AI to find the fastest way, without hitting a wall to get to the finish line. Going in a circular pattern, ziz zag motion or going as slow as possible would mean failure for the AI.
One of the big differences between the human driving and the AI is information available. The human learns the layout of the track and optimizes each turn for the next. The AI is only given information about what it can see at any given moment. In other words, the AI is effectively driving the track for the first time every time.
Yeah thats exactly the point. His AI stops to progressing because it lacks of Input. He need to let the AI see further to let it predict curves and such better. Also the current speed, acceleration and other factors need to be inputs to the learning algorithm.
@@sethmath2778I think that is happening because it isn't anticipate what comes next, so even if it zigs when it is supposed to zag half the time the selection function will just take the ones that got lucky that time and put them into the next generation.
There is science that does imply, that quantum mechanics follows evolution in the form of something you've all heard before : Path of Least resistance...that this path is actually an evolution process formed by the arrangement of molecules shifting around to settle into the most efficient configuration. There is a famous experiment with the optimal configuration of bubbles...how "most" of the time they settle into the most optimal configuration...but sometimes they don't...and so it's been the study of how evolutionary processes is how nature optimizes all problems not just biology :)
I am mad that because he did such a reckless thing now his name will forever be tainted, he's no longer "one of the best players of the game", he's now stuck as "the guy who cheated". You ruined yourself and for what ?
@@Seraphim262 A record that he should have known he would have gotten caught for at some point, it's not like he played a rng heavy game and he just cheated by creating the exact odds he wanted, Trackmania is like Doom, it's very easy to access your run and replay it for scrutiny and any discrepancies can be found very easily. The only reason it took so long was because people didn't think it was necessary since he was excellent and had a good reputation and grinded so they trusted him. He himself destroyed that trust, it was certain that it would get found out someday somehow, he could have obtained the record the legit way if he kept grinding instead of taking the easy way out, now he pays the price
Genetic mutations. As we evolve ressesive traits can sprout causing the mutation. It's based of survival of the fittest but has bad traits. Over time there will be less and less until it's no more. It's literally evolving and changing it's (DNA)
Hey Josh! Your AI driving algorithm reminds me of water flowing down a tube... which is almost the opposite of how F1 drivers drive; as they hug the corners rather that rebounding off the opposite walls. Really interesting experiment! Thanks for sharing.
If a high pressure water source was blasted into a tube that was well constructed for minimum resistance, it would follow an f1 drivers route wouldn't it?
If you were to introduce a stronger penalty for hitting the wall, such as ending the run right there and not letting it progress, would a stronger rule like that ensure the 'gene' for clipping the walls was removed?
I was about to suggest this myself. It seems to me that the AI is held back by the fact that it thinks that hitting the wall is a valid strategy to make a turn, whereas we humans know that this is not the case.
I would rather add a penalty after 1 second of wall contact, depending on the vehicle it might be easier to implement a course correction via controlled crash
There’s a bunch of ways to “outsmart” genetic algorithms by teaching certain skills you previously knew were important before teaching the primary goal. For example, teaching not to hit walls or turning as little as possible. Using this you’ll have a more refined base AI to learn the track. I’d love to see you try something like this again using this method!
I wonder if a directive to simply have the maximum amount of space between the front of the car and the wall directly in front would give the best result.
Yeah, part of thinks it would have been better if the simulation was stopped as soon as the car hit the wall; although I know nothing about A.I learning. That's just a hunch.
@@sciencemanguy, obviously you also use other parameters for accelerating while the distance part is used to to determine facing, not speed. But I didn't clarify that in the original post so I guess I deserve that.
This reminds me of how they let supercomputer play a Civilisation game against normal AI. You know, the game that you can win by many possibilites - Space race, Culture race, Political race, Technology race, use diplomacy etc.... or go tryhard and conquer most of the world. The supercomputer was learned those rules, possibilites of win, conditions of technologies ... and after some calculations he entered an absolute warmonger scenario, flooded the map with his units and crushed every opposition he ever faced. It was ... disturbing to say at least and worrying to see a "normal AI" thinking of such a result and then to execute it brilliantly. Don´t know the algoritm used or any details, it just stays in my mind as a memory of an article i once read
@@siriusczech you may want to account for the artificially created environment and the limits it provides the AI to deal with, introducing a general bias towards warmongerism as the best strategy in this context. This might be a general bias in the game system (war is always stronger than any other type of progress towards victory) OR the lack of collaboration benefits. Which are always there as long as the ultimate goal is a "winner takes it all" mentality. The winning conditions of Civ are that you need to be the guy on top. The first one. So collaborating is only worth if you are still the one getting out of it on top which deminishes the whole purpose of collaboration in the long run. The game's system and winning conditions would need to be adjusted if you want the "true" AI (not the algorithms just called "AI") to come out on top without killing everyone :D
@@alexejfrohlich5869 it strongly depends on the nationality of AI - some civs have significant bonuses there to be able fo fulfill other types of victories (tech race or cultural race is one of them) and this thing doesn´t require too much giving up of any other strategy gameplay - like that you couldn´t build a strong or high tech army in the first place. The issue there was that no matter the nation, no matter those conditions, it ALWAYS flooded the map with tons of cheap units, somehow achieving the victory despite it pissed of every nation one after another and it didn´t care for diplomacy or other things neither as much as you would think it will be - just a brutal 1000 turns raid on barbarians, trading only something and with strentgh in counts it defeated even those that it (based on numbers) shouldn´t defeat. And that was interesting on it whole - that perfectly "thought-through" assault is probably the easiest and most viable strategies even in this world, no matter other facts. That the problem with humans is that even dictators cannot wage war against such computing force.
@@siriusczech might be that there are bonuses, but it still looks like waging war is just the "best" strategy in this game system. also it is most likely the easiest accessable. if the AI is beating everything by spamming units also exposes a general flaw in the game system that was there the whole time. the AI is just taking advantage of it. so it looks like "killing" is the best strategy for the AI but actually, it is the best strategy within this artificial system. the AI just makes it clearly visible.
My only issue with generations (with know experience and just watching youtube) is you see a good contender that isnt the fastest (e.g. doesn't hit a wall but comes second) and it gets scrubbed - in a generation or two might acctually take over the current wall smashing leader.
The term for what youre describing is a local minima/maxima - the algorithm effectively gets 'stuck' within the search space. Its why, as the other commenter says, you dont kill all but the winner, but its also the reason for random mutations being introduced!
@@whatelsula Could have also made 'backwards progress' on the track an elimination category too. This way they AI would have learned to not hit walls and/or go the wrong way on the track.
So the cars don't seem to be able to predict a turn. If you want them to be able to predict a turn you need to eather increase the resolution, so that AI would be able to see an oncoming turn. Or increase the neural resolution both to allow cars to process different turn radius and temporal resolution so that cars can hold and remember certain turns. Small or wide turn do look similar to the AI and AI needs a way of distinguishing between them. That's the difference between simple stimuli respondent AI and another one that can better generalize the problem.
I think an easy way to provide "memory" of track layout would be to give it list of vectors from the center vector towards the way track is going. For some tracks, the best curve through a corner depends on two following corners. If you don't provide even a rough info about those corners, the AI can never excel. With the current input, the best that AI can ever do in theory is to play at roughly level of human driving any map for the first time.
Wouldn't it be easier if you could somehow say "The AI driver hit the wall, eliminate this AI driver." And then just measure time at the checkpoints? Base your fitness level on the checkpoint times. That way the AI would have to find the fastest way to a checkpoint without hitting a wall.
@@harmenoosterhof9604 I think there might be evolutionary peaks and valleys. As in: AI have to do something unproductive for couple of generation before it can improve. As in AI has to drive close to the wall without hitting the wall, I bet that most would hit the wall and get lower score thus those genes not surviving at all. And even if you manage to train AI to hug the wall. It still needs to know what wall to hug. So you need deeper network to learn and predict corners and hug walls at the optimal time.
It's funny, coming from a language where the 'h' is silent, he puts in so mich effort to pronounce words like "how" right that he even does it with words like "hour" - where it's actually silent in english as well! :-D (no offense, just something I noticed)
I feel like the reason they're not performing as well is that they're very limited in what they can see. They can only see the walls right in front of them, so they can't think ahead for the next curve and account for it, which is why they always run into that one wall in the curve
This is where better features would help. You can see in the long straight sections the car seems to travel at a weird angle, this is because the distances to the wall aren't relevant for the straight, the next corner information is. Is there a GitHub repository for this code?
seems like the the measured input parameters (wall dists and speed) are reaching its limit regardless of the number of future iterations. Certain curves or curve combinations look "same" to the AI whilst in fact the AI should understand that they are not the same ahead of time (by measuring other/additional parameters). Because there is a limited number of curve types and thus combinations of them in TM one could try to make the AI "see" which ones they have at hand and learn accordingly. Imo this way the record of the reference driver may be broken. Also does the AI steer inputs between 0-100% or always 100%? Maybe this adds extra friction and therefore slowdown?
@AE Templates Rather than making it map specific, a better solution would be to give a line of sight that correspond to different track pieces. For instance if you use circles and curves as LoS, the AI will be able to see past the bends into the turns. The good thing is that with this kind of learning algortihm, you shouldn't need to do complicated stuff, the AI should figure it out on its own.
@AE Templates Well actually it is we can see over the walls, even without know the turns ahead of time you can see the track ahead of you. If you want the test to be fair... you'd need to recreate the track with walls a human couldn't see over... in which case you'd see humans advance similar to the AI. Slamming into the walls or going too slow... while we'd learn "faster" we'd learn basically the gap between generations. In other words, comparing AI to humans here is completely unfair as we have two different tracks.
@@abrightguy508 there is much more than that into obtaining the best track time, you need to be able to slow at a faster rate than the one given to you by the game wich is basically lack of acceleration and how fast the game brakes on its own, because the doesnt use physics, it uses numbers, so its a given break strenght number, you are able to increase that number, thats an action, so yeah, if AI cant even do that its missing quite a lot already. This assuming this is all real and not just show, but hey, its fun to watch regardless.
I love this serie of videos ! If you continue to do this type of videos, could you one day make a sort of "making off" video where you show us more about the code, the process to create these scenes with 100 cars driving on the same course or the way you create new AI to a next generation? I would love to learn more about this subject ! Ps: french team, lets go !
yes, AI resolution of data is to weak. But at 11:45 of the video, the AI makes a corrective movement to the left, because he wants to be in the center of the track, not on the fast lane. You could prioritize the ahead distance so AI dont make to much of an anticipate move.
I wonder if it remembers the next turns. If not, it will never improve as much. It just learn to drive in a track as if it is the first time it drives it. I think Trabadia drove the map some times before realizing the best time. Is it true? How many times did he try?
I feel like using shortest split times from one checkpoint to the next might give better results. Maybe there's a car that doesn't do sections 1 and 2 very well, but kills it on section 3 where every other "well performing" car is having lots of trouble. If your fitness function is only time to end point, you're going to miss out on some more targeted improvements.
Brilliant stuff. Always get excited when I see your videos pop up. As you said, supervised learning is better for new tracks, but (idk about others) for me it's more interesting to maximize a particular map using GA. As someone who's worked with GAs before, I can relate to your problems of time consumption being the biggest issue. It'd take many more iterations with different parameters (and more generations) to start finding a global maxima. I know that in tmnf people have managed to speed up the game to help with stuff like this, but I'm not sure if you could merge it with your program.
Thanks ! I'm glad you like this, I've been watching your videos for years :D Yeah time is by far the biggest problem, 100 generations already takes so much time.. I think Supervised learning is pretty cool too : it would be awesome to have an AI that can play any map, and let it discover the campaign and collect the medals for example !
That last compilation of all the runs together, looks like a marble run from overhead, the way they all bounce off corners in the same spots, and all the erratic moving back and forth, just like marbles.
One thought, could the AI be slamming into walls because they don't have brakes built in as a response? From what you mentioned, they can only turn and accelerate. Would explain the lack of learning and inability to approach human times past that certain point. Anyways, the whole video is awesome and I wanted to give you specific props on the editing and overlays on this. Really help visualize concepts. Idk how you select which runs to run together in clips, or how you got that shot sitting in the middle of the track with cars going by, but they were great visualizers.
Thanks !! You don't need brake on this specific map, Trabadia didn't use brake in his run for example. And the AI is still able to stop accelerating. But brake would be useful in more complex maps. It's easy to sort replays in folders, and to select and edit specific replays ingame. And there are tools to edit camera shots ingame.
He speaks french and we don't have H in our pronunciation. So in english class we learn to pronounce the H. We do it every time we see a H in english now
fun fact about this video : The narrator'a accent have been mentionned 26 times in the comments. 26 out of 26 are french guys making fun of it. 0 out of 26 are "international" english speaking people claiming it interfered with their viewing experience. Conclusion : french should shut the fuck up when one of them speaks english
Interesting comment ! I know my accent isn't very good. If I make another video with voice-over, maybe I'll try to make it in French with English subtitles, I'll have to think about it. If any of you have a constructive opinion on this, please let me know :)
@@yoshtm The whole point of the comment is to say that your accent is fine and there is nothing wrong with it. You're a perfectly understandable and should keep doing what you do.
@@yoshtm no, keep voice ovet with english language, not only it will improve your english (im assuming english isnt your 1st language), and that way you can train your english speaking ability, and can attract more viewers.
Along with the wall distances and car speed you should also add the car direction as an input. This should help in turns significantly. Also if you take vectors across 180 degrees for measuring wall distances, it can increase the accuracy of the algorithm.
As said by others I would add the brake output, many inputs (especially the "what's next"), then train the supervised AI and use that as a starting point for the genetic algorithm, which could be trained changing the map every few generations.
Impressive that you created this for Trackmania! Two major things you could have added are penalizing bumps, which slow down the car, and make the AI able the break, which is necessary in some tracks. Even though this isn't added, it was fun to see what your results were. Great job!
@Esteban Ariel Zegpi Hunter It's necessary if you want the AI to be able to do a perfect run though. Otherwise you shouldn't count on it learning to wiggle, or other similar things, whether known or not currently by humans, or things that may be humanly impossible to reproduce. You do need to introduce that cleverly however, so that the AI will still be able to learn how losing speed *can* be good (maybe it would work already like that though, hard to tell. Just remember the AI would take into account speed AND time. Since your last answer suggests that you overestimated how much speed would be, compared to time).
I think, beginning with supervised learning, making the ai already know the basics and then switching towards genetic algorithm could improve it's time a lot. Giving the ai more data would improve it as well. I think average speed would come in handy for it
This ai has a disadvantage vs humans its data for oncoming track is all at the lowest possible level humans can see on coming corners before they are in direct line of sight therefore the ai is having each corner blind and without being able to see if the corner tightens ahead or snakes into another corner
yes. I'm sure the AI would be able to beat even the best humans, when it is given enough information. It's very hard to give it the required information without making the evaluation of the AI too complex to be run in realtime though
On the other hand, AI has an advantage over humans, time, it could literally do this forever. If you could automate the process, and allow itself to delete down bad performances, and generate new ones, eventually it would be perfect on this track. Ultimately becoming sentient, and destroying us all.
@@Tewty11 I think It really depends on how it's programmed, the fact the AI learns slower as time goes on means it has reached a local maximum, and depending on the algorithm it could never improve beyond that. Biological evolution had millions of years and billions if not trillions of "test subjects". Once AI reaches this kind of scale then I think we could see some serious competition with humans.
You could remove its vision and reward point for every check point it would take longer to learn but the end result would be an ai that just know when to press the right button at the right time more precise then a human would ever be.
@@spaceygnat19908 this here is the right answer. One that doesn't require visual input, would yield the highest track time but comes with other problems. The issue with visual Input is that there needs to be a larger over view of the track so the ai could predict it's next move, rather then react.
A few improvments. 1. Add more inputs. Maybe dubble them. 2. More variants. 3. Let them break. 4. Let them play on moremaps to avoid overfitting. 5. Make the same variant play on like 20 diffrent maps and add the pointstogheter. This makes sure that your AI dosent reach a false best weight.
adding another fitness factor for having travelled the least amount of distance while still finishing the race would get MUCH closer to the pro driver.
@@timotheelfbv8350 Totally aware as they take the line that offers the least distance weighed against the least loss in speed when rounding turns, but this addition of "complete the map with the least distance" is only ONE fitness metric that would be balanced against the others by the algorithm. Shortest time, least distance, highest top speed etc. Just because one isn't the only factor doesn't mean it's not relevant.
@@CrawfordAutomation Sure, i agree ! It will be faster indeed, and it could be even faster with other informations but it will be to complex may be to give a car the perfect raceline !
AI trained through these processes will never be as fast as a human for the simple reason that humans are able to anticipate the track, thus using more information than the parameters given to the AI.
And it's too much data to handle at once for a neural network from what I understand, except if you could find exactly the data points we extract from what we know of the rest of the track, maybe time to next turn or position of the road 20 meters ahead, I don't know. I would not say never, I think we can make a machine better than us if we manage to give him the right data
@@vladthecon Really, I thought it was the other way around and learned to drive but didn't learn the map. I thought it just learned how to react based on certain parameters of distance to the barriers. If the map was bigger it would learn some more lessons on cornering but be none the wiser about the layout of the track.
@@AussieAmigan the algorithm tried to learn to drive but because the map was short and unchanging one new corner drove 90% of them into a wall it was a necessary step in the learning process but the tracks should have been more varied so more general lessons would be learned early(or even better we get a continuation of the project)
I wonder if the limiting factors for the AI is absence of map knowledge. The AI only "sees" the current curve but does not get any information about the next curve ahead. Would you (or anybody else) be able to choose the ideal approach to a curve without knowing what is ahead? Have you tried feeding it the cars position between the start of the map and the end of the map as another input 0.0 - 1.0? Maybe the AI can just "train/remember" the correct steering values over time depending on the input making it very map specific... Another thing helping the AI might be to provide more "refined" inputs than just the distance to the walls. Car positions, angles between car and forward direction of the road, angle to next curve, ... But i guess that clashes with you goal of having an AI that learns driving from "nothing".
Yes I think it's the limiting factor, and I really don't know how to change that ! Indeed, it's really important to know "what comes next" in Trackmania. I think it would be hard for a neural network to "understand" this x,y,z position + 0.0-1.0 input, especially on a long map. And yeah it's very map specific. But maybe I could use your 0.0 - 1.0 input in addition to distance inputs. I could also use a separate 0.0-1.0 input for each section of the map !
The main problem of this is overfitting, the ai would basically only be memorizing the map and not how to drive. Also with network of sufficient size it's very much possible that the ai could be actually guessing the position of car on the road and such (if it doesn't overfit).
@@yoshtm A pretty simple thing you could try is giving the AI a couple of past frames of data as an input. This way it may be able to get a better sense for the corner it is entering. Even better might be to give it a memory all together, but I don't think NEAT supports that
@@yoshtm If you can give it the relative track boarder position for say the next 3 blocks as input it should in theory have all the information required. It also keeps the input vector size constant if you measure the length of 3 blocks and give the edge coordinates of the track within that length. "understanding" should not be much of an issue as there is quite a direct relation between the wall and car (don't go towards them, especially the near ones). But the increased input size probably requires more training and show more odd behavior, (i am predicting u turns will be a problem initially). No idea how you would generate this input though.
@@yoshtm I've never played trackmania before. Is there a minimap? Take a screenshot of it, and use it as an input to the neural network. That should help it "look ahead" to the exit of turns and upcoming corners.
This looks great! Do you also buffer one frame? I did some similar experiments and noticed it's beneficial to have 2 or 3 timesteps as inputs for the neural network. Then your AI will be able to predict trajectories based on the frames before. (it would have a perception of speed) This is not exactly a map memory, but at least a very short term knowledge. They will then brake and accelerate better on curve entries or exits.
You should have changed the objective function into two parts, maximize the distance traveled d(x) and minimize the number of hits on the curb h(x). Then the objective function becomes something like f(x) = 1/d(x) + h(x), and minimize f(x). Then you would have seen a better convergence. You can also add importance of each criteria too, like f(x) = (w1 * 1/d(x)) + (w2 * h(x)). Even better if you could normalize d(x) and h(x) on to same range [0.0, 1.0].
Very interesting video! I’d suggest adding more factors to the fitness function. Maybe make it so that the total distance traveled has to be the shortest while still finishing. This forces the AI to take tight lines just like the pro and thus travel less distance and be faster. Right now it seems like the AI doesn’t know how to improve further because there is no incentive to take tight lines.
in general there are three things to consider: speed, distance and steering input. To be quick means to be as fast as possible while driving the shortest distance possible while steering as little as possible. To find a balance between the three factors will generally give a pretty good approximation of the racing line. I hope that helps to think about a few other factors as well, that could lead to better results if combined correctly
Thanks for watching this video !
This is the first time i'm using NEAT algorithm, so there is obviously still room for improvement. The main problem is that my AI doesn't have a map memory, and can't anticipate "what comes next" with its current inputs. I have some ideas to improve my AI, so don't forget to subscribe if you want to see the next steps of this project ;)
Could you in the later generations select the ones that traveled the shortest distance at the last checkpoint? This would make them use the racing line more closely wouldn’t it? And then you could pick out the ones that don’t hit the walls. Not sure if that’s at all possible but it would make sense to me
in this vid i saw fast ai that started bouncing of the walls at a later point getting over taken by a slower ai. to create fast ai i needs map knowledge and learn that racing that keeps as much speed as possible and a bump on the wall slows you down. So basically all ai need to know what is faster in much smaller sections than a few checkpoints and then combine them to fastest to checkpoint. it's not going to be easy to create fast driving self learn ai.
@@theracerdude also if the cars are punished for hitting a wall they will make corners without hitting the walls while also taking the shortest route
neat optimizes the topology of the model, if i remember it correctly you don't train the weights of the model.
ai.googleblog.com/2019/08/exploring-weight-agnostic-neural.html
github.com/google/brain-tokyo-workshop/tree/master/WANNRelease/prettyNEAT
neat
Trackmaina in this form is a quite sophisticated liquid simulation.
13:32 best part
@@verrtex7837 thats trippy
Totally not kidding. I bet if you layered more hough level runs at increasingly delayed start times it would appear even more accurate
Fluid simulation*
You should watch the trackmania 20k project from l4bomba
Can't wait for the implementation of the brakes in order to see the AI drift !
AI
Well drifting will be cool but in these types of races like Formula, drifting will be totally useless. But yea will look cool
@@evgeny8578 In sharp some sharp corners drifting is faster than releasing acceleration.
th-cam.com/video/lNPKKQywzEQ/w-d-xo.html
@@evgeny8578 this is trackmania not a simulator so basically there’s a technique called speed drift by holding brake from about half a second while turning the car enters a drift where it gains speed, I was weird game physics but it’s a part the games identity now
This is exactly how water flows trough pipes. Should we try to put a genetic algorithm on water drops to tech'em flow better? 🤔
Yes
Popular boy.
This is not like water flow at all. Doing a water simulation is completely different of this ai
So. We actually already do. Electrical and audible waves are used in many purification/production processes.
@@hugoantunesartwithblender I mean. It kind of is though
Try this: if the car hits the wall, then remove a point. This should make the AI learn faster because hitting the walls will let them learn to not hit walls.
Or just kill them off if stopped. A little nudge can even be better than taking a turn slower to not hit the wall.
Wrong. The GA will converge to *NOT ACCELERATE AT ALL* because you would maximise your points.
@@khantatat If it doesn't accelerate, remove a point, problem solved.
Acceleration - +1 increment
Hits a wall - -1
Doesn't accelerate - -1
@@thedoctor0892 this would be better, it has an incentive to accelerate. Please note that, if feasible, it might converge to a circular path, which would represent a very appealing local maximum
@@khantatat Then in this scenario, there should be a limited time for the AI to reach the finish line, this would then force the AI to find the fastest way, without hitting a wall to get to the finish line.
Going in a circular pattern, ziz zag motion or going as slow as possible would mean failure for the AI.
One of the big differences between the human driving and the AI is information available. The human learns the layout of the track and optimizes each turn for the next. The AI is only given information about what it can see at any given moment. In other words, the AI is effectively driving the track for the first time every time.
Also, human player sees much more further down the track (curves on the horizont), this AI is limited in its visual field.
Yeah thats exactly the point. His AI stops to progressing because it lacks of Input. He need to let the AI see further to let it predict curves and such better. Also the current speed, acceleration and other factors need to be inputs to the learning algorithm.
Each car needs its own camera for vision to see far
And the ai should take the longest visible straight-shot instead of just zig-zagging through the entire track.
@@sethmath2778I think that is happening because it isn't anticipate what comes next, so even if it zigs when it is supposed to zag half the time the selection function will just take the ones that got lucky that time and put them into the next generation.
Alternate title: Weird water learns to flow efficiently on racing track.
There is science that does imply, that quantum mechanics follows evolution in the form of something you've all heard before : Path of Least resistance...that this path is actually an evolution process formed by the arrangement of molecules shifting around to settle into the most efficient configuration.
There is a famous experiment with the optimal configuration of bubbles...how "most" of the time they settle into the most optimal configuration...but sometimes they don't...and so it's been the study of how evolutionary processes is how nature optimizes all problems not just biology :)
Of course Trabadia can help with developing a TrackMania AI. He's got lots of experience using tools in runs.
Ah a man of culture I see
I am mad that because he did such a reckless thing now his name will forever be tainted, he's no longer "one of the best players of the game", he's now stuck as "the guy who cheated". You ruined yourself and for what ?
@@sephikong8323 For the worldrecord.
@@Seraphim262 A record that he should have known he would have gotten caught for at some point, it's not like he played a rng heavy game and he just cheated by creating the exact odds he wanted, Trackmania is like Doom, it's very easy to access your run and replay it for scrutiny and any discrepancies can be found very easily. The only reason it took so long was because people didn't think it was necessary since he was excellent and had a good reputation and grinded so they trusted him. He himself destroyed that trust, it was certain that it would get found out someday somehow, he could have obtained the record the legit way if he kept grinding instead of taking the easy way out, now he pays the price
That’s a oof moment
20 generations in and I'm still the car hitting the wall at the start line.
Genetic mutations.
As we evolve ressesive traits can sprout causing the mutation.
It's based of survival of the fittest but has bad traits. Over time there will be less and less until it's no more.
It's literally evolving and changing it's (DNA)
@@dfdempire8912 what? I'm aware of how this is structured. I was making a joke.
@@Tollerah93 lol
@@dfdempire8912 Jokes. You know what jokes are, do you?
Watching this video and zoinks there I am! Amazing work!!
Thank you so much !! I loved your video series on genetic algorithms, it helped me a lot in the beginning ! Very happy you came across this video :D
@@yoshtm the
@@brad3262 th
@@gamefun2525 ok
@@yoshtm This is a lame video ai has been able to do this since the 1970s
Isn't it strange how this looks just like flowing water.
Nah, we live in a world of mathematical patterns so reality is basically just a very advanced AI.
Agreed ... it's a little bit hypnotizing as well :)
it's strange how despite his awefull pronounciation you can still make out what he says
Or like insect
which means water is AI, therefore universe is AI
The way that first car morphs at 13:33 is amazing. This was extremely visually interesting
looks like some breakcore typa visualizer
"The circle strategy" made me giggle
This just goes to show that even in a world where geniuses are all around you, some idiots decide to bash their heads on the wall instead
Enough idiots banging their heads will finally get through.
@@sulosky brute force method
It's more people born without legs
Left, Right and Floor it. The only three inputs a true racing car ever needs.
Oh and the handbrake of course, for drifting really sharp turns with style and speed.
I prefer the bang bang bang => wasted strategy =D
@@xcruell it wouldn't be as efficient so it'd probably cut all the nonsense out
@@manz7860 have you played trackmania before?
You can get rid of one of those inputs in Nascar. Makes it simpler... Lol
Trabadia is very well suited to assist a tool, as he has received so much help from a tool assistant himself. :p
Was looking for a comment on Trabadia lol
Yeah trabadia is really good for real trabadia is amazing
En 0.0005 secondes, j'ai compris que j'avais affaire à un français haha
La mm mdr 😂
Pareil
+1
Mais tellement 😂
C'est marrant parce qu'aucun coms anglais mentionne son accent, les seuls qui en parlent c'est nous même x)
Damn every episode is better than the other, this project is just too cool man, keep it up! :D
The last clips are basically a fluids simulation 😂
The first bit of the track looks like a sink trap/u-bend 😂
I thought the same
Hey Josh! Your AI driving algorithm reminds me of water flowing down a tube... which is almost the opposite of how F1 drivers drive; as they hug the corners rather that rebounding off the opposite walls. Really interesting experiment! Thanks for sharing.
If a high pressure water source was blasted into a tube that was well constructed for minimum resistance, it would follow an f1 drivers route wouldn't it?
If you were to introduce a stronger penalty for hitting the wall, such as ending the run right there and not letting it progress, would a stronger rule like that ensure the 'gene' for clipping the walls was removed?
I was thinking the same thing, if you use a combination of factors for the fitness function it can learn better behavior.
Exactly what I was thinking. A sort of ‘punishment’ for the AI for either hitting the wall or not reaching a specific checkpoint by a certain time
I was about to suggest this myself. It seems to me that the AI is held back by the fact that it thinks that hitting the wall is a valid strategy to make a turn, whereas we humans know that this is not the case.
A really straightforward solution is just to add a time penalty if it hits the wall, so total time is duration + penalties and optimise total time.
I would rather add a penalty after 1 second of wall contact, depending on the vehicle it might be easier to implement a course correction via controlled crash
There’s a bunch of ways to “outsmart” genetic algorithms by teaching certain skills you previously knew were important before teaching the primary goal. For example, teaching not to hit walls or turning as little as possible. Using this you’ll have a more refined base AI to learn the track. I’d love to see you try something like this again using this method!
I wonder if a directive to simply have the maximum amount of space between the front of the car and the wall directly in front would give the best result.
@@TheRealMeatwad That would be very inefficient....it isn't how apex of a corner works
Yeah, part of thinks it would have been better if the simulation was stopped as soon as the car hit the wall; although I know nothing about A.I learning. That's just a hunch.
@@TheRealMeatwad Congratulations, the car will now stay still and not move!
@@sciencemanguy, obviously you also use other parameters for accelerating while the distance part is used to to determine facing, not speed. But I didn't clarify that in the original post so I guess I deserve that.
"-Hey Terminator, how did Skynet ever become so powerful?"
"-Gaming."
This reminds me of how they let supercomputer play a Civilisation game against normal AI. You know, the game that you can win by many possibilites - Space race, Culture race, Political race, Technology race, use diplomacy etc.... or go tryhard and conquer most of the world.
The supercomputer was learned those rules, possibilites of win, conditions of technologies ... and after some calculations he entered an absolute warmonger scenario, flooded the map with his units and crushed every opposition he ever faced. It was ... disturbing to say at least and worrying to see a "normal AI" thinking of such a result and then to execute it brilliantly.
Don´t know the algoritm used or any details, it just stays in my mind as a memory of an article i once read
_WarGames_ the 1983 movie. To a computer, reality is a simulation.
@@siriusczech you may want to account for the artificially created environment and the limits it provides the AI to deal with, introducing a general bias towards warmongerism as the best strategy in this context. This might be a general bias in the game system (war is always stronger than any other type of progress towards victory) OR the lack of collaboration benefits. Which are always there as long as the ultimate goal is a "winner takes it all" mentality. The winning conditions of Civ are that you need to be the guy on top. The first one. So collaborating is only worth if you are still the one getting out of it on top which deminishes the whole purpose of collaboration in the long run. The game's system and winning conditions would need to be adjusted if you want the "true" AI (not the algorithms just called "AI") to come out on top without killing everyone :D
@@alexejfrohlich5869 it strongly depends on the nationality of AI - some civs have significant bonuses there to be able fo fulfill other types of victories (tech race or cultural race is one of them) and this thing doesn´t require too much giving up of any other strategy gameplay - like that you couldn´t build a strong or high tech army in the first place.
The issue there was that no matter the nation, no matter those conditions, it ALWAYS flooded the map with tons of cheap units, somehow achieving the victory despite it pissed of every nation one after another and it didn´t care for diplomacy or other things neither as much as you would think it will be - just a brutal 1000 turns raid on barbarians, trading only something and with strentgh in counts it defeated even those that it (based on numbers) shouldn´t defeat. And that was interesting on it whole - that perfectly "thought-through" assault is probably the easiest and most viable strategies even in this world, no matter other facts.
That the problem with humans is that even dictators cannot wage war against such computing force.
@@siriusczech might be that there are bonuses, but it still looks like waging war is just the "best" strategy in this game system. also it is most likely the easiest accessable. if the AI is beating everything by spamming units also exposes a general flaw in the game system that was there the whole time. the AI is just taking advantage of it. so it looks like "killing" is the best strategy for the AI but actually, it is the best strategy within this artificial system. the AI just makes it clearly visible.
My only issue with generations (with know experience and just watching youtube) is you see a good contender that isnt the fastest (e.g. doesn't hit a wall but comes second) and it gets scrubbed - in a generation or two might acctually take over the current wall smashing leader.
That's why you usually only cull the worst 50% instead of the 99% that didn't win.
The term for what youre describing is a local minima/maxima - the algorithm effectively gets 'stuck' within the search space. Its why, as the other commenter says, you dont kill all but the winner, but its also the reason for random mutations being introduced!
@@TheSmiddy Oh, is he culling the worst 99%?
@@diabl2master
One assumes not as that's basically never done.
I feel like not using wall hits as an elimination category was an oops moment
I don't know. AI who learn to follow the wall will progress, a contrario of AI doing circles.
@@whatelsula Could have also made 'backwards progress' on the track an elimination category too. This way they AI would have learned to not hit walls and/or go the wrong way on the track.
@@ziero1986 Yes, but maybe not for the first generations, I think.
Yep, even the fastest in generation 100 still slammed into that wall before the chicane
Didn't you wonder wether or not it was a good strategy? Maybe it makes you faster overall even if you lose some speed the moment you hit the wall.
So the cars don't seem to be able to predict a turn. If you want them to be able to predict a turn you need to eather increase the resolution, so that AI would be able to see an oncoming turn. Or increase the neural resolution both to allow cars to process different turn radius and temporal resolution so that cars can hold and remember certain turns.
Small or wide turn do look similar to the AI and AI needs a way of distinguishing between them.
That's the difference between simple stimuli respondent AI and another one that can better generalize the problem.
plus in a track you may have to tackle a corner differently based on what corner immediately follows it, not sure how to do that here though
@@Obi-WanKannabis I'm sure that with bigger network AI could simply learn the map. Or learn how to learn the map.
I think an easy way to provide "memory" of track layout would be to give it list of vectors from the center vector towards the way track is going. For some tracks, the best curve through a corner depends on two following corners. If you don't provide even a rough info about those corners, the AI can never excel.
With the current input, the best that AI can ever do in theory is to play at roughly level of human driving any map for the first time.
Wouldn't it be easier if you could somehow say "The AI driver hit the wall, eliminate this AI driver." And then just measure time at the checkpoints? Base your fitness level on the checkpoint times. That way the AI would have to find the fastest way to a checkpoint without hitting a wall.
@@harmenoosterhof9604 I think there might be evolutionary peaks and valleys.
As in: AI have to do something unproductive for couple of generation before it can improve. As in AI has to drive close to the wall without hitting the wall, I bet that most would hit the wall and get lower score thus those genes not surviving at all.
And even if you manage to train AI to hug the wall. It still needs to know what wall to hug.
So you need deeper network to learn and predict corners and hug walls at the optimal time.
Je regrette pas de m'être abonné avec la première vidéo, super hâte de voir la suite !
There needs to be more content like this. I would literally watch videos of every single popular game even though it's the same algorithm
The French is strong with this one.
Algorizzum
It's funny, coming from a language where the 'h' is silent, he puts in so mich effort to pronounce words like "how" right that he even does it with words like "hour" - where it's actually silent in english as well! :-D
(no offense, just something I noticed)
He sounds like he had 1 hour to learn english pronounciation and then had to read the script
Oui oui
@@Brabldibrablmann trust me, even after years of speaking english, my prononciation isn't much better
I feel like the reason they're not performing as well is that they're very limited in what they can see. They can only see the walls right in front of them, so they can't think ahead for the next curve and account for it, which is why they always run into that one wall in the curve
This is where better features would help. You can see in the long straight sections the car seems to travel at a weird angle, this is because the distances to the wall aren't relevant for the straight, the next corner information is.
Is there a GitHub repository for this code?
seems like the the measured input parameters (wall dists and speed) are reaching its limit regardless of the number of future iterations. Certain curves or curve combinations look "same" to the AI whilst in fact the AI should understand that they are not the same ahead of time (by measuring other/additional parameters). Because there is a limited number of curve types and thus combinations of them in TM one could try to make the AI "see" which ones they have at hand and learn accordingly. Imo this way the record of the reference driver may be broken. Also does the AI steer inputs between 0-100% or always 100%? Maybe this adds extra friction and therefore slowdown?
@AE Templates Rather than making it map specific, a better solution would be to give a line of sight that correspond to different track pieces. For instance if you use circles and curves as LoS, the AI will be able to see past the bends into the turns. The good thing is that with this kind of learning algortihm, you shouldn't need to do complicated stuff, the AI should figure it out on its own.
@AE Templates Well actually it is we can see over the walls, even without know the turns ahead of time you can see the track ahead of you. If you want the test to be fair... you'd need to recreate the track with walls a human couldn't see over... in which case you'd see humans advance similar to the AI. Slamming into the walls or going too slow... while we'd learn "faster" we'd learn basically the gap between generations.
In other words, comparing AI to humans here is completely unfair as we have two different tracks.
hi taco.
"Turn left, turn right, and accelerate"
Robots don't need brakes nor breaks.
Acceleration means the rate of change of speed which also includes slowing down
@@abrightguy508 there is much more than that into obtaining the best track time, you need to be able to slow at a faster rate than the one given to you by the game wich is basically lack of acceleration and how fast the game brakes on its own, because the doesnt use physics, it uses numbers, so its a given break strenght number, you are able to increase that number, thats an action, so yeah, if AI cant even do that its missing quite a lot already. This assuming this is all real and not just show, but hey, its fun to watch regardless.
I love how you explained all the science behind it, it allows me to understand how these AI work on a conceptual level
Le bon accent de Vendée on le sent on l'entend il fait chanter nos tympans :x
je me disais aussi que yavais un ptit goût de brioche 😅
@@Daneri42 Ah la c'est la gâche carrément :v
Mdr x)
@@ShiroGojo miam
Plutôt bocage où littoral ?
I love this serie of videos !
If you continue to do this type of videos, could you one day make a sort of "making off" video where you show us more about the code, the process to create these scenes with 100 cars driving on the same course or the way you create new AI to a next generation? I would love to learn more about this subject !
Ps: french team, lets go !
Watch the videos of code Bullet
Is it a mod for TrackMania ? How can he control so many cars in TrackMania?
13:33 this just makes me think about how long this must have taken to render
Long enough 😂
En tant que français, j'ai tout de suite reconnu d'où tu venais ptdrr
Superbe vidéo sinon, et très beau travail !
13:32 was so trippy!
I love how at the end following Trabadia's car its like he's escaping the tidal wave of AI like its an action movie or something XD
I thought the same thing. And the ones that turn around always make me laugh. 🙂
Would make for an intense game, especially if you had a mini turret on the back.
It almost looks like he's playing it in slow motion.
It's like his car is outrunning a zombie horde.
The AI could probably do better if there are more “sight” lines clustered toward the front allowing it to make more precise movements.
I was about to say this as well. As it is now, I feel like it's handicapped compared to a human.
yes, AI resolution of data is to weak. But at 11:45 of the video, the AI makes a corrective movement to the left, because he wants to be in the center of the track, not on the fast lane. You could prioritize the ahead distance so AI dont make to much of an anticipate move.
@@camilohurtadoacero7233 That's because the AI was trained to just finish the track, not get the fastest time.
I wonder if it remembers the next turns. If not, it will never improve as much. It just learn to drive in a track as if it is the first time it drives it. I think Trabadia drove the map some times before realizing the best time. Is it true? How many times did he try?
th-cam.com/video/yZFY5ZJtgyM/w-d-xo.html
cool video mon gars
Always love seeing videos like this! Well explained concepts and presentation. I'm sure a lot of work has been put into this one.
Thanks ! and thank you also for your help with Openplanet :)
I feel like using shortest split times from one checkpoint to the next might give better results. Maybe there's a car that doesn't do sections 1 and 2 very well, but kills it on section 3 where every other "well performing" car is having lots of trouble. If your fitness function is only time to end point, you're going to miss out on some more targeted improvements.
13:32 That's a great shot!
corona be like...
Bro, this guy. I work in IT and still don't understand how you mad mad mad genius code this stuff! Great video!
Bro I can just sense the French in his English
you have to be deaf not to hear that
@@metalvideos1961 Im a Brit Naniiii I hate france
@@JakSpate bonjour
this guy is not an indian?
@@Antiork lol no, listen at his pronunciation of r and you'll see
13:00 the girls in your dms after you get rid of that yee yee haircut
Lol
229 likes but 1 reply ! How
@@koo9ol Actually three
@@tvojejidlo8143 you're wrong it's four
@@knoert7977 no its actually five
Brilliant stuff. Always get excited when I see your videos pop up. As you said, supervised learning is better for new tracks, but (idk about others) for me it's more interesting to maximize a particular map using GA. As someone who's worked with GAs before, I can relate to your problems of time consumption being the biggest issue. It'd take many more iterations with different parameters (and more generations) to start finding a global maxima. I know that in tmnf people have managed to speed up the game to help with stuff like this, but I'm not sure if you could merge it with your program.
Thanks ! I'm glad you like this, I've been watching your videos for years :D
Yeah time is by far the biggest problem, 100 generations already takes so much time..
I think Supervised learning is pretty cool too : it would be awesome to have an AI that can play any map, and let it discover the campaign and collect the medals for example !
th-cam.com/video/U9hq2keQgY0/w-d-xo.html
Super-interesting! I found you by accident and subscribed immediately. Thank you! Very smart stuff. I love it.
That last compilation of all the runs together, looks like a marble run from overhead, the way they all bounce off corners in the same spots, and all the erratic moving back and forth, just like marbles.
AI: Jelle's Simulated Marble Runs
Martin: I'm still making the Wintergatan MMX physical
One thought, could the AI be slamming into walls because they don't have brakes built in as a response? From what you mentioned, they can only turn and accelerate. Would explain the lack of learning and inability to approach human times past that certain point.
Anyways, the whole video is awesome and I wanted to give you specific props on the editing and overlays on this. Really help visualize concepts. Idk how you select which runs to run together in clips, or how you got that shot sitting in the middle of the track with cars going by, but they were great visualizers.
Thanks !!
You don't need brake on this specific map, Trabadia didn't use brake in his run for example. And the AI is still able to stop accelerating. But brake would be useful in more complex maps.
It's easy to sort replays in folders, and to select and edit specific replays ingame. And there are tools to edit camera shots ingame.
Bro I love the fact you pronounce the H in hours. My wife cringes every time I do it.
He speaks french and we don't have H in our pronunciation. So in english class we learn to pronounce the H. We do it every time we see a H in english now
I stand with your wife.
That "hitting the wall" they all have is the machine equivalent to a human sneeze or stuck eyelash...
I watch code bullet and riolu a lot and you’re like a mix of them both. Love these types of videos :)
It is amazing how close it looks to a fluid passing through a tube.
fun fact about this video :
The narrator'a accent have been mentionned 26 times in the comments.
26 out of 26 are french guys making fun of it.
0 out of 26 are "international" english speaking people claiming it interfered with their viewing experience.
Conclusion : french should shut the fuck up when one of them speaks english
Interesting comment !
I know my accent isn't very good. If I make another video with voice-over, maybe I'll try to make it in French with English subtitles, I'll have to think about it.
If any of you have a constructive opinion on this, please let me know :)
@@yoshtm The whole point of the comment is to say that your accent is fine and there is nothing wrong with it. You're a perfectly understandable and should keep doing what you do.
@@yoshtm no, keep voice ovet with english language, not only it will improve your english (im assuming english isnt your 1st language), and that way you can train your english speaking ability, and can attract more viewers.
@....... Tu feras gaffe, ton anglais est plus propre que ton français...
Conclusion: everybody hates french people
Along with the wall distances and car speed you should also add the car direction as an input. This should help in turns significantly. Also if you take vectors across 180 degrees for measuring wall distances, it can increase the accuracy of the algorithm.
Nobody:
A chocolate bar in my pocket: 13:32
Running from zombies in real life: There's a lot of them, better walk fast
Running from zombies in movies: 14:27
running from zombies in real life? whaa
@@mebe6474 you what
The red one is Neo.
The blue ones are agents Smith.
13:17 "Guys, go ahead, do not wait for me! I forgot the mask, need to get back home and get it."
ahaha nice comment
Super video et super l’accent! :)
As said by others I would add the brake output, many inputs (especially the "what's next"), then train the supervised AI and use that as a starting point for the genetic algorithm, which could be trained changing the map every few generations.
Brake is useless on this map. Changing the map every few generations would be a good idea for generalisation
Trabadia? This hasn't aged well
If reality is a big simulation, probably I'm one of those cars that did go backwards.
Reality has not to be a simulation, this simulation is based on reality.
Impressive that you created this for Trackmania! Two major things you could have added are penalizing bumps, which slow down the car, and make the AI able the break, which is necessary in some tracks. Even though this isn't added, it was fun to see what your results were. Great job!
It is so satisfying watching the hundreds of cars drive all over track like a wave
Yes, like Satan
Maybe add speedometer as a variable for “reward”. Such that it learns that completion and go fast are both “good”.
@Esteban Ariel Zegpi Hunter Yes but speed can be tracked throughout the map.
So the difference of a big good, fast race, and small good, individual instance of speed
@Oussema Nijewi Reinforcement learning
The point of any race is to get the lowest time. It doesn't make sense to make the AI do top speed runs.
@Esteban Ariel Zegpi Hunter
It's necessary if you want the AI to be able to do a perfect run though. Otherwise you shouldn't count on it learning to wiggle, or other similar things, whether known or not currently by humans, or things that may be humanly impossible to reproduce. You do need to introduce that cleverly however, so that the AI will still be able to learn how losing speed *can* be good (maybe it would work already like that though, hard to tell. Just remember the AI would take into account speed AND time. Since your last answer suggests that you overestimated how much speed would be, compared to time).
Imagine an AI who knows how to find shortcuts.
that is the inevitable outcome of a well developed self learning AI. Shortcuts and exploits....
Interesting video and amazing effort! Many hours spent, nice work man!
Why would anyone dislike this? I can't even imagine the amount of work put into that video.
Maybe because he mentioned someone who got exposed as a cheater?
Code bullet watching this be like: *sweats*
13:31 just saying if you don't watch while video click this spot for a mind melting car melt
great video Yosh thank you very much for your great effort! subbed :)
I think, beginning with supervised learning, making the ai already know the basics and then switching towards genetic algorithm could improve it's time a lot. Giving the ai more data would improve it as well. I think average speed would come in handy for it
All cars : finding best path to drive
That one car at the start line : I Quit!
amazing dedication, keep it up! Also, I think that having closer checkpoints would train the AI a lot faster.
Amazing video. Inspirational 😎
13:17 that one it's evolving, just backwards :D
13:32 When I drop my can of Coke and say "It's just a little spill"
This ai has a disadvantage vs humans its data for oncoming track is all at the lowest possible level humans can see on coming corners before they are in direct line of sight therefore the ai is having each corner blind and without being able to see if the corner tightens ahead or snakes into another corner
yes. I'm sure the AI would be able to beat even the best humans, when it is given enough information. It's very hard to give it the required information without making the evaluation of the AI too complex to be run in realtime though
On the other hand, AI has an advantage over humans, time, it could literally do this forever. If you could automate the process, and allow itself to delete down bad performances, and generate new ones, eventually it would be perfect on this track.
Ultimately becoming sentient, and destroying us all.
@@Tewty11 I think It really depends on how it's programmed, the fact the AI learns slower as time goes on means it has reached a local maximum, and depending on the algorithm it could never improve beyond that. Biological evolution had millions of years and billions if not trillions of "test subjects". Once AI reaches this kind of scale then I think we could see some serious competition with humans.
You could remove its vision and reward point for every check point it would take longer to learn but the end result would be an ai that just know when to press the right button at the right time more precise then a human would ever be.
@@spaceygnat19908 this here is the right answer. One that doesn't require visual input, would yield the highest track time but comes with other problems.
The issue with visual Input is that there needs to be a larger over view of the track so the ai could predict it's next move, rather then react.
A few improvments.
1. Add more inputs. Maybe dubble them.
2. More variants.
3. Let them break.
4. Let them play on moremaps to avoid overfitting.
5. Make the same variant play on like 20 diffrent maps and add the pointstogheter. This makes sure that your AI dosent reach a false best weight.
adding another fitness factor for having travelled the least amount of distance while still finishing the race would get MUCH closer to the pro driver.
Pro drivers dont use the raceline with the least distance
@@timotheelfbv8350 Totally aware as they take the line that offers the least distance weighed against the least loss in speed when rounding turns, but this addition of "complete the map with the least distance" is only ONE fitness metric that would be balanced against the others by the algorithm.
Shortest time, least distance, highest top speed etc. Just because one isn't the only factor doesn't mean it's not relevant.
@@CrawfordAutomation Sure, i agree ! It will be faster indeed, and it could be even faster with other informations but it will be to complex may be to give a car the perfect raceline !
@@timotheelfbv8350 I would argue complexity is a requirement to reach a perfect line, as there as a great number of factors in the concept.
14:00 You Simulated Turbulent Flow... That's Really Useful in science.
AI trained through these processes will never be as fast as a human for the simple reason that humans are able to anticipate the track, thus using more information than the parameters given to the AI.
And it's too much data to handle at once for a neural network from what I understand, except if you could find exactly the data points we extract from what we know of the rest of the track, maybe time to next turn or position of the road 20 meters ahead, I don't know. I would not say never, I think we can make a machine better than us if we manage to give him the right data
one thing i didn't like was that the map was so small and was not changed so the ai learned the map instead of how to drive.
@@vladthecon Really, I thought it was the other way around and learned to drive but didn't learn the map. I thought it just learned how to react based on certain parameters of distance to the barriers. If the map was bigger it would learn some more lessons on cornering but be none the wiser about the layout of the track.
@@AussieAmigan the algorithm tried to learn to drive but because the map was short and unchanging one new corner drove 90% of them into a wall it was a necessary step in the learning process but the tracks should have been more varied so more general lessons would be learned early(or even better we get a continuation of the project)
13:32 THE routes taken by the cars look like a mycelium, beginning to explore a surface looking for food. Damn trippy.
This man is underrated
I like the A.I that stops immediately and is like "yeah, nah, I'm good"
I wonder if the limiting factors for the AI is absence of map knowledge.
The AI only "sees" the current curve but does not get any information about the next curve ahead.
Would you (or anybody else) be able to choose the ideal approach to a curve without knowing what is ahead?
Have you tried feeding it the cars position between the start of the map and the end of the map as another input 0.0 - 1.0?
Maybe the AI can just "train/remember" the correct steering values over time depending on the input making it very map specific...
Another thing helping the AI might be to provide more "refined" inputs than just the distance to the walls.
Car positions, angles between car and forward direction of the road, angle to next curve, ...
But i guess that clashes with you goal of having an AI that learns driving from "nothing".
Yes I think it's the limiting factor, and I really don't know how to change that ! Indeed, it's really important to know "what comes next" in Trackmania.
I think it would be hard for a neural network to "understand" this x,y,z position + 0.0-1.0 input, especially on a long map. And yeah it's very map specific. But maybe I could use your 0.0 - 1.0 input in addition to distance inputs. I could also use a separate 0.0-1.0 input for each section of the map !
The main problem of this is overfitting, the ai would basically only be memorizing the map and not how to drive.
Also with network of sufficient size it's very much possible that the ai could be actually guessing the position of car on the road and such (if it doesn't overfit).
@@yoshtm A pretty simple thing you could try is giving the AI a couple of past frames of data as an input. This way it may be able to get a better sense for the corner it is entering. Even better might be to give it a memory all together, but I don't think NEAT supports that
@@yoshtm If you can give it the relative track boarder position for say the next 3 blocks as input it should in theory have all the information required. It also keeps the input vector size constant if you measure the length of 3 blocks and give the edge coordinates of the track within that length. "understanding" should not be much of an issue as there is quite a direct relation between the wall and car (don't go towards them, especially the near ones). But the increased input size probably requires more training and show more odd behavior, (i am predicting u turns will be a problem initially). No idea how you would generate this input though.
@@yoshtm I've never played trackmania before. Is there a minimap? Take a screenshot of it, and use it as an input to the neural network. That should help it "look ahead" to the exit of turns and upcoming corners.
This is a great video. Very informative, and delivered amazingly. FUCK YEAH!
13:32 most satisfying moment of the video
The best humans simulator. There are those who learn from mistakes and there are also those who do not learn
Not really. From what Ive understood AIs which arent efficient enough just get deleted. They dont learn from mistakes.
That's just like humanity until the past few hundred years. Suboptimal humans would die and optimal ones would survive to pass on their genes.
@@tydal6516 you're not wrong...
@@identitymatrix you could consider all of them being the same ai just trying new things
@@homailot2378 Yes but like other instances of the same AI.
This looks great! Do you also buffer one frame?
I did some similar experiments and noticed it's beneficial to have 2 or 3 timesteps as inputs for the neural network. Then your AI will be able to predict trajectories based on the frames before. (it would have a perception of speed) This is not exactly a map memory, but at least a very short term knowledge. They will then brake and accelerate better on curve entries or exits.
Great video, structure and explanation!
as an actual fluid dynamicist I was excited to point out how it looks like fluid flow, then i realised everyone else in the world spotted it.
Years of academy training wasted
@@HaXD1209 What'd you spend your years of academy training on?
If you wanna collab, I can drive some future maps ;)
YOU will drive?
All your videos are of other people driving...
@@XDRosenheim no they are not actually.
It’s heating up in here
You have a very good accent français 😂, sinon c'est super cool
Tous les français reconnaissent instantanément notre magnifique accent en anglais
You should have changed the objective function into two parts, maximize the distance traveled d(x) and minimize the number of hits on the curb h(x). Then the objective function becomes something like f(x) = 1/d(x) + h(x), and minimize f(x). Then you would have seen a better convergence. You can also add importance of each criteria too, like f(x) = (w1 * 1/d(x)) + (w2 * h(x)). Even better if you could normalize d(x) and h(x) on to same range [0.0, 1.0].
Very interesting video! I’d suggest adding more factors to the fitness function. Maybe make it so that the total distance traveled has to be the shortest while still finishing. This forces the AI to take tight lines just like the pro and thus travel less distance and be faster. Right now it seems like the AI doesn’t know how to improve further because there is no incentive to take tight lines.
Thanks ! It's a good idea, I should try that on a short map :)
in general there are three things to consider: speed, distance and steering input.
To be quick means to be as fast as possible while driving the shortest distance possible while steering as little as possible. To find a balance between the three factors will generally give a pretty good approximation of the racing line.
I hope that helps to think about a few other factors as well, that could lead to better results if combined correctly
The end of this video is how the TV show Lost came up with the Smoke Monster's movement
your combined run clips remind me of how a light pulse spreads out in a fiber optic line! a pretty good representation if ya ask me :D
Engineers on the Titanic: "Hey do you guys hear th-"
Water: 13:32