I love how they seem to move so organically even though it seems like a relatively simple model. I bet there's some really interesting optimisation problems and extra restrictions you could throw at this.
I think that the need to center themselves perfectly with the sphere is what makes them not become speed machines. Because when they reach the target they always gotta somehow "dock". And that requires their inertia to be 0 when they reach that point so they have to slow down. If somehow this was changed by making the drones to just need to touch the point at any part and maybe making the orb bigger I would certainly expect that there would be more speedy manoeuvres to just arrive at the target and pass through it. Perhaps even in an elliptical patrolling. Would be certainly interesting to see.
Exactly my thoughts. It looks like the target requires pixel perfect precision to count as a success. Careful approach is the only way when the targeting criteria are so unnecessarily strict.
@@Wock__ I believe you are right. On one of their videos, there is an actual clock face that counts down on top of the target, like a circular loading bar.
The goal is to dock, not to touch the target. Changing the goals to achieve a better outcome does not mean that your model improved. Making them just have to touch the target so they could go really fast does not mean that they are suddenly better. Your thinking is flawed.
I'd love to see a game where your enemies are all neural network trained AI, and the higher the difficulty, the more trained AI variant you will have to face
Or even the player being an AI - I can totally see a 2D game with your cursor being the target point, and the more you play/the more enemies you defeat/etc. the smarter your character gets
@@originalbillyspeed1 i guess for different difficulty levels game designer can use agents (enemies) from different generations, for example "easy" = generation 400, medium = generation 500, hard=generation 1000.
@@AB-bp9fi I don't think that would work for most applications. When you want to make enemy AI easier or harder, you always have to think of it in relation to the player - for instance, in a stealth game, harder AI could mean it detects you faster - which pushes the player to improve and be more careful. That won't happen if you just made the enemies drunk (which is basically what would happen if you pick bad neural networks) - it just adds randomness which can be annoying to deal with. Maybe it could work better in things like racing games though.
I'm now imagining a game cloud coordinating through the internet. The AI uses background CPU while the game is running to simulate and evolve against itself, spits its best results against the player to see how they fare, and takes those results as more data to go back to the cloud with to keep working. The bots will start laughably bad at first, but they'll learn how players act, and make players devise new tactics... You might even get good teammate and wingman AI out of it if you put those AIs on the player's side.
@@commenturthegreat2915 What about training AI to match the certain level of intelligence? Like if AI detects a player too fast, then it failed the test.
Impressive Stuff! Had my hands on GAs too for my Bachelor Thesis but with a 6 DOF 3D acting robotic arm. Kinda addicting when you dive deep down in ML :)!
Hello, very interesting work ! Did you think about testing scenarios with obstacles ? It would be also interesting to compare the last trajectories and controls with optimal control algorithms solutions. Cheers.
Adding a fuel allowance would probably add a more varied result, possibly get those burn hard drones quicker. Also maybe increase your destination bubble a fraction ? This increase the prize rate and hopefully the drones would tighten up the homecoming naturally like the ants do for food routes
I tryed the mouse controlled vesion what you uploaded on github. And i saw that it's easy to confuse the A.I. in that way to lose controll and fall off the map. I think if you crate a small Trainer A.I. for the target control what best interest to confuse the drone and make it fall off the map, it can train the drone to not fall off no matter how the target moves.
You could turn the target tracking into a game, try to get the drone to lose control as quickly as possible, using your mouse as the target! Or, just play with it. It looks fun.
I’d love to see an algorithm where you simply add the direction from the current target point to the next, and see if it, with only that information learns to steer ahead of time.
This video felt like it's 30 minutes because I somehow kept falling asleep every ten seconds or so. And it's not boring and no I am not high, idk I guess I just got tired or something
I suggest to add more then just time to the fitness equation. Fe. Energy use, pressicion, stability of flight and adding external forces like wind. with these factors the movement would become smooth like silk. But nice project anyway
The current fitness evaluation takes speed, precision and stability into account. I tried to add wind after the training was done and it worked quite well :)
@@DeepRafterGaming Yes you're right and I don't really know why they do this. My assumption is that it is a way to reduce power, as if they couldn't go very close to 0 power so it is easier to add angle. This could be avoided by taking energy into account in the fitness function. If I increase gravity, they don't angle the thrusters to gain more power. Here is a windows demo with a config file if you want to try it out github.com/johnBuffer/AutoDrone/releases/tag/v1
@@PezzzasWork Yeah it's hard to tell why. The fitness function is the most complicated part of any neural network. I would allway advocate for implementing energy use in any neural network because, if you think about it, if the network doesn't have to bother with the used energy it will always come up with unnecessary movement patterns that look jenky. It's more important than speed I'd say ^^
@@PezzzasWork if you watch the way generation 5500 flys sideways, it ends up with one thruster almost horizontal and the other almost vertical. They might like tilting the thrusters because its kind of an inbetween state between flying right and left. So when it gets a new target, it can start flying towards the target sooner. That might be part of the reason anyway.
somewhat smaller models and policy gradient following might have increased convergence speed. MLPs are differentiable, so you could just backpropagate through them, sampling distance to the target at every frame and accumulating rewards over the trajectory for an unbiased estimate of a policy’s optimality. you could even use a decay term to incentivize the robots to move faster by downweighting rewards acquired later in the trajectory: distance to the target is ideally the same in the end, but according to the gradient of this reward function, faster would be better. the only thing left would be running the simulations in parallel or faster than real-time by simply not fully rendering the state of the environment at every training step
I'm kind of upset that you didn't publish the thing at the end on itch. Its so satisfying to see the drone follow your mouse and I want to play around with it. Great video!
Ok, now make these drones fight in groups of 5, they can kill other drones in 2 ways one is to ram into enemy drones (killing both of them instantaneously), or shooting them with miniguns (only killing the target if it is hit X amount of times). But every time when they die they respawn, smarter, faster, more accurate, etc.
This would be a great premise for a game where the character tracks the mouse so instead of controlling the character you're directing it and it gets better as you play through AI learning
And you did it with two hidden layers, nice! Also, you have to give it a gun now I mean come on. Let's see the the level 5500 drones beat a human being.
you should make a game where you control a small ship like asteroids and your goal is to juke out the drones and cause them to crash or see how long you can survive before they hit you or something
7:24 Loved how the Gen-400's legs synced with the music... Btw, How do we decide the size of the hidden layers? Is there some rule or formula for the best size approximation?
you should place the targets randomly and not in a specific order. And for more challenge, they only have a specific time to reach the target. After the time the target disappears. And finally, the targets are fuel. If they miss too often they run out of fuel. Edit: maybe even add obstacles.
Now make it 3d and hve the gen 5500 implemented in there, have them master the flight in 3d, then make the difficulties higher. Then after all that, put the best gen into a irl drone and have it fly around
its so funny to me how people in comment sections always say "now do [INSERT UNREALISTIC EXPECTATION HERE]" like there's so much difference between simulating drones, and making a drone in real life
While it was nice for the visual of all the different generations together, I feel like it would have been better to randomize the dot locations so that they have to learn to adapt to a new path every time
I would love to see what happens if you give them a finite amount of fuel to manage. Have the fuel decrease quickly/slowly depending on how hard they burn thrusters. Extra bonus for fuel remaining when the task is complete. Death if you run out of fuel.
Acceleration (gravity, mass an inertia) is probably the simplest physics properties to program. Literally just adding or subtracting numbers. He does not require your compliments on the physics.
Great video! I've been trying to make a similar recreation of this project in Python but while I get some decent results, I'm struggling with local minima trapping and have failed to get the kind of 'brutal' drones you got at the end of training. Tried having a look at the source code but I'm not too familiar with C++. Just wanna know, what did you use for your fitness function and how did you mutate your networks? A reply would be very much appreciated!
Programmers and scientists are going to have a lot to study on neural networks, maybe thats what new AI will provide. Just more information for humans to expand their minds
Hi Pezzza, I really liked the video and the way you trained it. Can you tell me how can I learn to code to train a model like this ?? I really want to learn how to do this level of coding. pls reply
Would it be possible to have the drones compete? For example, by simulating the entire population of drones at once, and only rewarding the first drone to reach a target.
![The moment we stopped understanding AI [AlexNet]](http://i.ytimg.com/vi/UZDiGooFs54/mqdefault.jpg)
Good idea, and I like your smoke!
Thanks! I think smoke is where I spent the most time :D
@@PezzzasWork Why do we get hung up on those small sidequests?
@@mendelovitch it’s an easy way to procrastinate the main problem
How or where you stimulate this in unity or special software
I love how they seem to move so organically even though it seems like a relatively simple model. I bet there's some really interesting optimisation problems and extra restrictions you could throw at this.
Also thanks for uploading the demo and source code, very fun to play around with!
I think that the need to center themselves perfectly with the sphere is what makes them not become speed machines. Because when they reach the target they always gotta somehow "dock". And that requires their inertia to be 0 when they reach that point so they have to slow down. If somehow this was changed by making the drones to just need to touch the point at any part and maybe making the orb bigger I would certainly expect that there would be more speedy manoeuvres to just arrive at the target and pass through it. Perhaps even in an elliptical patrolling. Would be certainly interesting to see.
im currently working on the same thing but with more inputs;
I will try ours too;
@@00swinter21 Don't forget to post the result on your TH-cam channel !
Exactly my thoughts. It looks like the target requires pixel perfect precision to count as a success. Careful approach is the only way when the targeting criteria are so unnecessarily strict.
@@Wock__ I believe you are right. On one of their videos, there is an actual clock face that counts down on top of the target, like a circular loading bar.
The goal is to dock, not to touch the target. Changing the goals to achieve a better outcome does not mean that your model improved. Making them just have to touch the target so they could go really fast does not mean that they are suddenly better. Your thinking is flawed.
5:28, gen 900: Ok, you guys are too good and I'm tired now. Bye!!!
true
"I have to go now, my planet needs me"
I'd love to see a game where your enemies are all neural network trained AI, and the higher the difficulty, the more trained AI variant you will have to face
Give it 10 years
imagine if the AI is being trained while you play. The better you play the less hard the ai is, but if you slow down the difficulty increases
@@ChunkyWaterisReal it's already possible now lol
@@marfitrblx AI has been shit since the 64 hush yourself.
Or even the player being an AI - I can totally see a 2D game with your cursor being the target point, and the more you play/the more enemies you defeat/etc. the smarter your character gets
This channel is a gem
Would be really interesting to add fuel consumption to the mix and watch them optimize their fuel economy
and give them more fuel for every target they reach as more reward for doing that
Very cool stuff, well done!
Now make them go through an obstacle course 😁
I am working on it ;)
a combination of the ants finding the optimal path and then the drones following that? :)
@@PezzzasWork where's the video 🗿
That end result with the live-tracking is so good! I wonder how viable it is to train simple neural networks like this for game enemy AI
Depends on the game, but on games with a clear goal, it is fairly trivial and will quickly surpass humans.
@@originalbillyspeed1 i guess for different difficulty levels game designer can use agents (enemies) from different generations, for example "easy" = generation 400, medium = generation 500, hard=generation 1000.
@@AB-bp9fi I don't think that would work for most applications. When you want to make enemy AI easier or harder, you always have to think of it in relation to the player - for instance, in a stealth game, harder AI could mean it detects you faster - which pushes the player to improve and be more careful. That won't happen if you just made the enemies drunk (which is basically what would happen if you pick bad neural networks) - it just adds randomness which can be annoying to deal with. Maybe it could work better in things like racing games though.
I'm now imagining a game cloud coordinating through the internet. The AI uses background CPU while the game is running to simulate and evolve against itself, spits its best results against the player to see how they fare, and takes those results as more data to go back to the cloud with to keep working. The bots will start laughably bad at first, but they'll learn how players act, and make players devise new tactics... You might even get good teammate and wingman AI out of it if you put those AIs on the player's side.
@@commenturthegreat2915 What about training AI to match the certain level of intelligence? Like if AI detects a player too fast, then it failed the test.
Great work. I think many people would appreciate seeing background of the work.
love this channel. what separates this guy from others is his consistent ability to make his sims look cool.
This is one of the coolest implementations i've seen. Nj!
I wrote my autopilot cargo drone for space engineers and still i am impressed by the work
Props to Gen 300 and 400 for beings underdogs and yet surviving for so long
This is one of the coolest projects I've ever seen. Would be awesome to extend to add walls and an environment! Great work.
Dude I love it when they get sooo roofless! So fun to watch!
The end of play lineup was a cute touch. Nice work!
very nice! I'd love to see the same tests, but with added random disturbances like wind gusts from the side, to see how well they can adapt to that!
OMG This is so cool, your video actually change my attitude toward neural network from hate to love.
Impressive Stuff! Had my hands on GAs too for my Bachelor Thesis but with a 6 DOF 3D acting robotic arm. Kinda addicting when you dive deep down in ML :)!
Hello,
very interesting work !
Did you think about testing scenarios with obstacles ?
It would be also interesting to compare the last trajectories and controls with optimal control algorithms solutions.
Cheers.
5:56 The drone in the left down corner synchronized with the beat in the music. Perfection.
You are amazing. Thank you for sharing your fascinating work.
Getting some strong Factorio vibes at 4:57
give a consolation prize to generation 300!
It deserves it all
Have you ever tried using a neural network on a hardware platform?
1:58 that faint Vader "noooooo" put me on the floor for some reason
gen 400 is like that one kid in your class that cant stand still when waiting in a queue
Very nice! Please make more such content, with neural network and drones! :)
Adding a fuel allowance would probably add a more varied result, possibly get those burn hard drones quicker. Also maybe increase your destination bubble a fraction ? This increase the prize rate and hopefully the drones would tighten up the homecoming naturally like the ants do for food routes
It would be great to have a remake of this one
I am actually working on a follow up :)
@@PezzzasWork noice! I will certainly watch it
I tryed the mouse controlled vesion what you uploaded on github. And i saw that it's easy to confuse the A.I. in that way to lose controll and fall off the map. I think if you crate a small Trainer A.I. for the target control what best interest to confuse the drone and make it fall off the map, it can train the drone to not fall off no matter how the target moves.
Yes I did a more robust version that I can upload as well
You could turn the target tracking into a game, try to get the drone to lose control as quickly as possible, using your mouse as the target! Or, just play with it. It looks fun.
Give the target to another network that tries to learn how to get the drones to crash while the drones learn how not to crash
@@DogsRNice oh no the ai wars
I’d love to see an algorithm where you simply add the direction from the current target point to the next, and see if it, with only that information learns to steer ahead of time.
Other than giving us almost 20 seconds to read 6 words at 4:39 this was very enjoyable to watch :p
I like how it learned to turn off its thrusters to arrest upward motion and to speed up descent.
Imagine spending hours and hours trying to get to something and then when you finally get there you just have to go to another one
xDDD the "ok..." almost kills me
The memes are fun on this vid
This video felt like it's 30 minutes because I somehow kept falling asleep every ten seconds or so.
And it's not boring and no I am not high, idk I guess I just got tired or something
I suggest to add more then just time to the fitness equation. Fe. Energy use, pressicion, stability of flight and adding external forces like wind. with these factors the movement would become smooth like silk. But nice project anyway
The current fitness evaluation takes speed, precision and stability into account. I tried to add wind after the training was done and it worked quite well :)
@@PezzzasWorkahh I see, but the angled engines while hovering still seem very inefficient to me :)
@@DeepRafterGaming Yes you're right and I don't really know why they do this. My assumption is that it is a way to reduce power, as if they couldn't go very close to 0 power so it is easier to add angle. This could be avoided by taking energy into account in the fitness function. If I increase gravity, they don't angle the thrusters to gain more power. Here is a windows demo with a config file if you want to try it out github.com/johnBuffer/AutoDrone/releases/tag/v1
@@PezzzasWork Yeah it's hard to tell why. The fitness function is the most complicated part of any neural network.
I would allway advocate for implementing energy use in any neural network because, if you think about it, if the network doesn't have to bother with the used energy it will always come up with unnecessary movement patterns that look jenky. It's more important than speed I'd say ^^
@@PezzzasWork if you watch the way generation 5500 flys sideways, it ends up with one thruster almost horizontal and the other almost vertical. They might like tilting the thrusters because its kind of an inbetween state between flying right and left. So when it gets a new target, it can start flying towards the target sooner. That might be part of the reason anyway.
somewhat smaller models and policy gradient following might have increased convergence speed. MLPs are differentiable, so you could just backpropagate through them, sampling distance to the target at every frame and accumulating rewards over the trajectory for an unbiased estimate of a policy’s optimality. you could even use a decay term to incentivize the robots to move faster by downweighting rewards acquired later in the trajectory: distance to the target is ideally the same in the end, but according to the gradient of this reward function, faster would be better.
the only thing left would be running the simulations in parallel or faster than real-time by simply not fully rendering the state of the environment at every training step
Thanks for the video! It's really inspiring.
i have no idea about how you did it ..but it seems like something fun to learn
Machine learning is extremely fun and addictive :)
@@PezzzasWork can confirm
I'm kind of upset that you didn't publish the thing at the end on itch. Its so satisfying to see the drone follow your mouse and I want to play around with it. Great video!
You can download the control demo here github.com/johnBuffer/AutoDrone/releases/tag/v1
@@PezzzasWork thank you! :)
Ok, now make these drones fight in groups of 5, they can kill other drones in 2 ways one is to ram into enemy drones (killing both of them instantaneously), or shooting them with miniguns (only killing the target if it is hit X amount of times). But every time when they die they respawn, smarter, faster, more accurate, etc.
oooh idea. Space Invaders: Drones Addition. Different levels use different generations of drones as enemies.
This would be a great premise for a game where the character tracks the mouse so instead of controlling the character you're directing it and it gets better as you play through AI learning
400 was such a trooper
"300! 400! you're embarrassing everyone!"
Beginning of the video: LOL!! those squeaks as they fall are really funny
End of the video: let's run to buy some food cans before they come for me!!!
im more impressed by the smoke, great project though!
"Im a Hovercraft like my Father before me and his before him!"
i want to see how chaotic it will be if the drones had collision
I will try this, that’s a good idea ;)
The target tracking would be cool for a background
After a few tweaks, I have a feeling this could have real-world use.
And you did it with two hidden layers, nice! Also, you have to give it a gun now I mean come on. Let's see the the level 5500 drones beat a human being.
Pls make more vids like this I love them
Micheal Reeves breaking out in a cold sweat in the corner
These drones are adorable
Would be interesting to have a drone sumo where they can collide and try to shove each other out of a ring.
You should make a game out of this, it looks very funny!!
Gen 5500 appears to display knowing how to fall rather than turning the thrusters to push itself down.
That drone that got yeeted at 5:30 had me dieing 😂
it's cool to see your using dropout, so it learns better
you should make a game where you control a small ship like asteroids and your goal is to juke out the drones and cause them to crash or see how long you can survive before they hit you or something
the target tracking drone would be a really cool and distracting extension, it follows your cursor around where ever you put it lol
so, it was you who programmed my dog to run after the laser pointer.... 🤔
Which parameters give the drone positive or negative feedback?
Is flying time a positive or a negative parameter? An acceleration to the target?
Now create an additional network which positions the orange dot (target) to navigate around obstacles on its own.
7:24 Loved how the Gen-400's legs synced with the music...
Btw, How do we decide the size of the hidden layers? Is there some rule or formula for the best size approximation?
7:25 the music moves to your left and right ear as the drone in the top right moves it's power to it's left and right thruster.
I like these projects !
you should place the targets randomly and not in a specific order. And for more challenge, they only have a specific time to reach the target. After the time the target disappears. And finally, the targets are fuel. If they miss too often they run out of fuel.
Edit: maybe even add obstacles.
In the video the targets are in a specific order to be able to benchmark the different generations, for the training I used random sequences
@@PezzzasWork Ok, that makes sense
Imagine one of these things chasing you irl.
Now make it 3d and hve the gen 5500 implemented in there, have them master the flight in 3d, then make the difficulties higher. Then after all that, put the best gen into a irl drone and have it fly around
But what should be the target
its so funny to me how people in comment sections always say "now do [INSERT UNREALISTIC EXPECTATION HERE]" like there's so much difference between simulating drones, and making a drone in real life
That was really cool.
Very nice result!
I would love for you to make an eco system like the bibites using those drones
Totally amazing!!!
It’s like they’re scared to collect it, knowing that once they do they’re likely to die xD
Nobody:
Generation 200: SPEEN
You may have to select more aggressively for speed. They seem a bit slower than what the optimal handmade algorithm could do
While it was nice for the visual of all the different generations together, I feel like it would have been better to randomize the dot locations so that they have to learn to adapt to a new path every time
Gen 2600 was a big leap in speed and control.
I would love to see what happens if you give them a finite amount of fuel to manage. Have the fuel decrease quickly/slowly depending on how hard they burn thrusters. Extra bonus for fuel remaining when the task is complete. Death if you run out of fuel.
i think just counting how much fuel is used based on thrusterpower and then rewarding low numbers is better then limiting fuel
Dude the physics look so polished. This is amazing!
Acceleration (gravity, mass an inertia) is probably the simplest physics properties to program. Literally just adding or subtracting numbers. He does not require your compliments on the physics.
@@UnitSe7en ?
@@UnitSe7en shut the fuck up, he’s giving him a compliment
Nice work! Can you propose me material so that I can understand in practice how to build a neural network? Something with examples.
That's a good tutorial idea, I will think about it :)
Great video! I've been trying to make a similar recreation of this project in Python but while I get some decent results, I'm struggling with local minima trapping and have failed to get the kind of 'brutal' drones you got at the end of training. Tried having a look at the source code but I'm not too familiar with C++. Just wanna know, what did you use for your fitness function and how did you mutate your networks? A reply would be very much appreciated!
Love that video
DARPA wants to know your location
Now you can make game with mouse controled drones
Programmers and scientists are going to have a lot to study on neural networks, maybe thats what new AI will provide. Just more information for humans to expand their minds
Great video! How long have you been training them? Greetings from Uruguay!
What about creating new variables? Like saving fuel or energy consumption, or giving priorities like speed over energy/fuel consumption
if you had an body orientation/angle input they would have been able to recover from a spin out or even fly upsidedown
Wonderful!
1:41 Lmaoo. This looks like flappy bird in terms of difficulty.
Hi Pezzza, I really liked the video and the way you trained it. Can you tell me how can I learn to code to train a model like this ?? I really want to learn how to do this level of coding. pls reply
ask chat gpt.
it knows a lot about it
i used unity for the physics and did my recreation there and it was even better then the original
In life, I am gen 400, btw it would be interesting if you add a fuel and time constraints and experiment with those
Would it be possible to have the drones compete? For example, by simulating the entire population of drones at once, and only rewarding the first drone to reach a target.
That's exactly what genetic algorithms (GA) means
1:05: this one looks like Los Angeles Battle drones