could you add some way of one time donation? because I rly want to thank you for these tutorials, but my financial situation is not good. also could you please try to make agents fly?
Hey, can you give us the link to this course? because you went over some things that you did off screen really fast, and the links you are including are to your website that has all of your courses, and I can't find this one on there.
@@keyhaven8151 No, if I remember correctly. I had to sort of just find it out on my own. Its been a year, and I can't find anything on my computer on it. I am so sorry!
Absolutely legendary. I had no idea that Imitation Learning could be slapped on top of Reinforcement Learning so easily now. Your explanations are clear and concise not just to repeat exactly what you're doing, but also to comprehend enough to come up with novel ideas. Thank you so much.
Having watched your previous video, I've had an "ok, done that" with reinforcement learning. Sure it can be handy for some scenarios but the training time can be insane. But this...? THIS I CAN USE! THANKS! I had no idea this was a thing. :D
I wanted to ask what is the ram you have in your pc. I have 4gb ram and as I downloaded pytorch my pc crashed due to immense lag. Any idea to reduce lag
@@TechnicalBurhan I'm running 16 gigs of RAM and running the Neural Network with an i7 9700k running at 4.5 ghz. Generally, with NN you want as beefy of a computer as possible, including a very generous amount of RAM so it can store the model in. 8 gigs is the minimum for NN I'd say. And if you're really serious you should get an NVidia GPU with 8 gigs+ of VRAM so it train the model much faster. Long story short: Neural Networks require a lot of horsepower to both run and train, if this one gives your computer an embolism, I'm afraid the only option is for you to find an alternative NN that requires less specs, somehow tweak this one to be smaller or code one that is lighter. (Though be warned that simpler NN's may be significantly dumber.) That or get a new computer. Up to you. I however suggest you look into other types of game development, I myself have trouble running them and doing so with your machine sounds like a miserable experience.
Thank you for these ML-Agents videos. For me the topic of board game AI training would be interesting too. For example, teaching AI to play Go on unusual boards. It would probably be based on imitation learning and self play, I guess...
Yup, adversarial agents is one of the next topics I want to research. They have a functioning Soccer game in the examples which seems to work very well.
Cool tool! It would be interesting to make very smart "enemies"for the game. Of course if I have enough brains and time to use it in a real project )))
When I use Imitation Learning with behavioral_cloning parameter, Unity has like 0.001 fps. Any tips how to fix it? Should I left only GAIL parameter? With GAIL only, it works fine though, but learning process looks weird
Hi, I was watching your video and I saw till 4 rth section and had a thought that if the agent's position is reducing from the button then give the agent half (0.5f) reward, and after it pushes the button give it full reward (1f). So the training will be faster, here is the code for checking if the agent is moving closer to the button or the food: public float dist; float distTemp = Vector3.Distance(destination.position, transform.position); if (distTemp < dist) { dist = distTemp; AddReward(0.5f); } else if (distTemp > dist) { dist = distTemp; AddReward(-0.5f); } Might Be helpful :)
Yup you can add a reward if the agent is moving closer to the goal. However it should be a much smaller reward than that. With that value the agent gets more of a reward by moving closer for 2 frames than by pressing the button. So over time it might learn to just move closer and further away and repeat that for infinity.
@@CodeMonkeyUnity I learned this the hard way back when I made a similar project in the early beta version of AgentML. Something more like AddReward(10f/(distance + 10f)) is a lot safer
I have always had a question about mlagents: they randomly select actions at the beginning of training. Can we incorporate human intervention into the training process of mlagents to make them train faster? Is there a corresponding method in mlagents? Looking forward to your answer.
Great video, I have the reinforcement learning running but I am confused by the heuristic function, when does this get called is it any time RequestDecision is called when we run in Heuristic behaviour mode? I have it translating key presses into actions so I can record a demo for imitation learning but when I record my key presses aren't registered and consequently no actions spawned.
Dose anybody know why my model stops learning no matter how long I let it run. in some cases, the Mean Reward goes all the way from something like 0 to 400 then back down to around 0. I was wondering if somehow it grows too big and runs out of something?
Hey CodeMonkey. Is it possible for Imitating Agent to get observations from mlagent's raycast or grid sensors? My agent's navigation is based hugely on m baske's Spatial Grid Sensor. and its been quite a while now its not showing any improvement during learning.
Why did you change the script to use only discrete actions? Movement was a continuous action in the previous video. As far as I know you should be able to use both discrete and continuous actions in your ML agents.
hey code monkey.. can i ask something about learning method? can you add more difficulty to already trained onnx? let say you add wall in the middle after the agent push the button or when button pressed agent and ball will randomly placed
Hi! When does the bot get rewarded when using the gail? Is he trying to repeat the sequence and time between rewards or is he just repeating the recording?
Great Work! I'm not realizing how the architecture of project is build :( I need check the prefabs and scripts to undestand all the picture so could you please provide a link to github to this project?
I have a question, I'm not overly familiar with ML but after watching this video it seems very useful. Now I'm just wondering if something like this can be used on something much more complex (Ex: Training a platformer AI or a 2d shooter AI). What do you think? Do you think something like that would even be possible? or would it introduce too many unknown variables and cause the learning to fail?
It can be used on anything. As long as you give it enough data and enough time to train it can learn to do anything. Check my latest video on using it to play Flappy Bird th-cam.com/video/fz8D0OZkQGQ/w-d-xo.html Stay tuned for a video on driving a car and more complex examples
is it possible to exclusively use GAIL? Im trying to compare RL vs IL and using GAIL as my representative for IL, but since it also uses extrinsic rewards it doesnt seem like a fair comparison.
After about 20 seconds into training my agent from behavioral_cloning/gail, pytorch crashes and I get the error 'Class values must be smaller than num_classes'. There's no error in Unity. This doesn't happen if I remove behavioral_cloning and gail from the config file. Anyone know what's happening? Edit: I have narrowed it down to behavioral cloning, not gail
How do you create machine learning for for a grid based movement with turns between the player and the AI? Can you make 3d tutorial based on turn based grid game with the player having turns against the AI
Hi codemonkey, I recreated the same as you did for immetation learning. But I am facing 2 issues. 1. OnEpisodeBegin is executing continuously on the loop. I can;t find why its happening so I put a flag to stop that actions. there I noticed the 2nd issue. 2. If I play the game in editor before starting the mlagents-learn on cmd the agent movement based on my input is working fine. if I did the same after entering the cmd mlagent-learn on cmd means. the Agent movement speed in unity is 200% higher. can you please help me to solve the issue? Only script I did on my own was FoodSpawner script and Food button script. FoodSpawner script was not showed in the video and foodButton is not completely displayed that's the reason. Note: if I test the script in unity editor before starting the cmd means its working as expected. the issue is only occurring after starting the mlagent to learn
After I adjust everything in the config file for the next run how do I "update" the brain? I'm not sure about the commands to use for Python etc and what to do in the inspector in Unity.
Hi, so to enable GAIL and Behavioural Cloning, we simply just change the Config file. Right? I heard you mentioned about the Discriminator for GAIL. Do we need to write the code for it?
Every time my agent has to make a decision, unity just doen't respond. After 10 seconds it continues until it has to make the next decision. But when I'm not using a recording, it runs smoothly...
I noticed that I have way less episodes in way more steps (several thousand steps but only like 30 episodes) in my recording. I had to put max steps to 0, or else I wouldn't be able to record a single good episode. 1000 Steps is less then a second for me. Does anyone know why?
@@CodeMonkeyUnity Funnily enough, I have strong photophobia (kind of the exact opposite to what you experience) and I get horrible eye strain (or just don't see at all) when the screen is too bright...
because i by need to make bots for my fps game i have my qustion is what method do you the better ? my other qustion is passable to use both of this methods
Depends a lot on the game, sometimes classic AI is the better approach and sometimes ML is better. Also depends on your familiarity with the tools, how familiar are you with classic AI? What about ML? Yes you can use both methods at once, like for example using ML to decide what to do and using classic AI to do that action.
@@CodeMonkeyUnity ok form i need to make the game play wean i make the game play and the multiple same time i will check haw to make i bot system with help or not
I'm sorry im new in this but i wanted to see how you relation every item with the script or how do you do the button to function and i don't see it in this video so i don't really understand what is happening do you have a step by step or the project?
@@CodeMonkeyUnity How do you relate the prefab ball to the button? I need it hidden in the unity environment and link it from there or how do you make that link?
@@laurasofiapadillacabrera4912 Can be a simple SerializeField Sounds like you need to learn the basics of Unity and C# before attempting Machine Learning which is a pretty complex topic th-cam.com/video/E6A4WvsDeLE/w-d-xo.html th-cam.com/video/IFayQioG71A/w-d-xo.html
I have a question If I start with bolt and then slowly move to c# will it be better or to direct learn c# And if I pick bolt will I have to start over for c# ?
Start with whichever one you feel most comfortable. Bolt is perfectly fine to start with, just choose programmer naming and later on it will be easy to transition into C# unitycodemonkey.com/courses.php?c=visualscripting
I almost copied the same code form the video, my AI looks like it's learning at first, but when reaching around an average of 1 for the reward in the learning curve, the AI be like, you know what, I think dancing around the button seems more fun, and the curve gets worst from there on, anyone faced such a thing ?
EDIT: I think I got it now the problem is that the food Prefab is Spawning like every second :( Hello, can anybody help, I'm having trouble with the following method GetLastFoodTransform(), it´s giving me the following error. I must say that I'm pretty new at coding so forgive me if I´m doing something pretty stupid . NullReferenceException: Object reference not set to an instance of an object FoodSpawner.GetLastFoodTransform () (at Assets/Scripts/Food Collect/FoodSpawner.cs:29) FoodAgent.CollectObservations (Unity.MLAgents.Sensors.VectorSensor sensor) (at Assets/Scripts/Food Collect/FoodAgent.cs:42) Unity.MLAgents.Agent.SendInfoToBrain () (at Library/PackageCache/com.unity.ml-agents@1.7.2-preview/Runtime/Agent.cs:1048) Unity.MLAgents.Agent.SendInfo () (at Library/PackageCache/com.unity.ml-agents@1.7.2-preview/Runtime/Agent.cs:1320) Unity.MLAgents.Academy.EnvironmentStep () (at Library/PackageCache/com.unity.ml-agents@1.7.2-preview/Runtime/Academy.cs:557) Unity.MLAgents.AcademyFixedUpdateStepper.FixedUpdate () (at Library/PackageCache/com.unity.ml-agents@1.7.2-preview/Runtime/Academy.cs:43). And this is the code I put in that method, public Transform GetLastFoodTransform() { lastFoodTransform.localPosition = food.transform.localPosition; if(lastFoodTransform) { return lastFoodTransform; } else { lastFoodTransform.localPosition = new Vector3(Random.Range(-2f, +2f), 0, Random.Range(-2f, +2f)); return lastFoodTransform; } }
actually suddenly out of no where all kinds of training make unity run at 3fps, was quick only an hour ago, whats going on lol now i have to give up since its so slow
I'm having a lot of trouble with the food spawner. I love the course idea, but without the project files, you waste a lot of time in the environment, and end up losing focus on the central topic that is ml agents. Anyway, if you can guide better about the food spawner, I would be very grateful.
My food prefab always instantiates in the same environment, like a clone, all food objects instantiate in the same environment simultaneously, I have 15 environments in massive training, but all foods instantiate only in the main one, I need help to solve this, could you help me?
Would it be possible to make "fair" Ai. The job of the AI would be to be an NPC. Could i make it fair so it does not hit like every shot or uses some moves that are just unbeatable. I read about this in the unity forum and they said it would not be possible.
You can train an AI to the point where its perfect, then instead of using the very last brain, you use one of the previous brains. So it will be good but not perfect.
💬 Here's how you can teach your agents how to complete more complex tasks. Stay tuned for more ML videos like Driving a Car and playing Flappy Bird! 📦 Complete Unity Machine Learning Playlist: th-cam.com/play/PLzDRvYVwl53vehwiN_odYJkPBzcqFw110.html
Hey Mr. Code Monkey I have a request to you to make a video about some checkpoints system. To be clear I am making a game and for efficiency and lag compression I want to disable rooms. I have some rooms and I only want some room to be rendered. I had an idea of a box collider 2d set to trigger but I don't know how to find the way the player is moving. For eg if the player is moving to the right I want to disable te room in the left. I don't know how to check which part of a collider is hit. Apart from that it is a box so I have no idea how to find where the player is going and reduce the lag as possible. Hopefully you get my point. incase any doubts please reply me. Hopefully I see a video on it. Btw Love your videos. Love from India
Wherever you are moving the player you have the new player position and the previous player position, with those two you can get the movement vector and see which direction the player is moving in. Then for disabling rooms that is highly dependent on your specific use case. It can be as simple as going through all the rooms and disabling/enabling them one by one.
Hello @Code Monkey! I know this field is pretty to new to gamedev, and, requires lots of experience. In up coming videos, Can y make some masterclass about creating "custom tools" for ML.? For instance, my interest is in "character joint physics tools" for ML. To be clear, I want to create custom Brain in future for my research. But y'know, the editor and environment interactions take too much time .In this time, I can easy forget about working with ML brain and pay close attention to automation Physics engine tools for the sake of future. So I think, we should also pay close attention to common logics that saves time to interaction between ML brain and environment logics. Then, it will be easier to make efficient ML technology to use everyday game development time.
You dont need tools specific for ML, everything that Unity already has already works with ML. You can create a Physics Joint then connect it to ML Actions and Observations. Check out the official example project, they have a Crawler example that uses physics joints to move a spider-like creature.
this code doesn't even work, tons of errors after manually typing it all after pausing a million times since you didn't post code. other videos were helpful but man, how frustrating...
What errors? What version are you using? ML-Agents is constantly being updated so if you're not using the same version as me you need to look in the changelog to see what changed.
@@CodeMonkeyUnity i dont know, im giving up on it, wasting too much time. im now trying to record videos of me playing but that lags, tried so often to get some simple replay system be it ai or human but its too difficult.
Hello, I have currently problems with a bug. As I was finished with saving the new .yaml, I run the ml agents and it sais: yaml.scanner.ScannerError: while scanning for the next Token found Character „\t“ that cannot Start any Token in „config/MTG.yaml“, line 24, column. But there is nothing… behaviors: MoveTo: trainer_type: ppo hyperparameters: batch_size: 10 buffer_size: 100 learning_rate: 3.0e-4 beta: 5.0e-4 epsilon: 0.2 lambd: 0.99 num_epoch: 3 learning_rate_schedule: linear beta_schedule: constant epsilon_schedule: linear network_settings: normalize: false hidden_units: 128 num_layers: 2 reward_signals: extrinsic: gamma: 0.99 strength: 1.0 gail: strength: 0.5 demo_path: D:\Unity\Games\ML_AI\Demos behavioral_cloning: strength: 0.5 demo_path: D:\Unity\Games\ML_AI\Demos max_steps: 500000 time_horizon: 64 summary_freq: 10000
🌐 Have you found the videos Helpful and Valuable?
❤️ Get my Courses unitycodemonkey.com/courses or Support on Patreon www.patreon.com/unitycodemonkey
could you add some way of one time donation? because I rly want to thank you for these tutorials, but my financial situation is not good. also could you please try to make agents fly?
Yes!
Hey, can you give us the link to this course? because you went over some things that you did off screen really fast, and the links you are including are to your website that has all of your courses, and I can't find this one on there.
@@BenVhekith I haven't found it either. Have you found it now?
@@keyhaven8151 No, if I remember correctly. I had to sort of just find it out on my own. Its been a year, and I can't find anything on my computer on it. I am so sorry!
You've been killing it lately! Great work - thanks for making these and the simulation videos! You rock!
Absolutely legendary. I had no idea that Imitation Learning could be slapped on top of Reinforcement Learning so easily now. Your explanations are clear and concise not just to repeat exactly what you're doing, but also to comprehend enough to come up with novel ideas. Thank you so much.
Thanks! Glad you liked the video!
@@CodeMonkeyUnity can you show us the foodSpawner code? I can't figure out what to put in there.
Having watched your previous video, I've had an "ok, done that" with reinforcement learning. Sure it can be handy for some scenarios but the training time can be insane.
But this...? THIS I CAN USE!
THANKS! I had no idea this was a thing. :D
I wanted to ask what is the ram you have in your pc. I have 4gb ram and as I downloaded pytorch my pc crashed due to immense lag. Any idea to reduce lag
@@TechnicalBurhan I'm running 16 gigs of RAM and running the Neural Network with an i7 9700k running at 4.5 ghz. Generally, with NN you want as beefy of a computer as possible, including a very generous amount of RAM so it can store the model in. 8 gigs is the minimum for NN I'd say.
And if you're really serious you should get an NVidia GPU with 8 gigs+ of VRAM so it train the model much faster.
Long story short: Neural Networks require a lot of horsepower to both run and train, if this one gives your computer an embolism, I'm afraid the only option is for you to find an alternative NN that requires less specs, somehow tweak this one to be smaller or code one that is lighter. (Though be warned that simpler NN's may be significantly dumber.)
That or get a new computer. Up to you.
I however suggest you look into other types of game development, I myself have trouble running them and doing so with your machine sounds like a miserable experience.
@@ArkyonVeil thanks for a detailed explanation. I have 4gb Ram and it doesn't work with me np
Imitation learning seems to give a huge headstart!
Amazing videos lately! Hope we see that Car AI with ML-Agents soon 😉
I love your videos buddy. Thanks for uploading so much.
Thanks!
Incredible. Thanks for this great tutorial, you covered all the information I needed and more.
Great work, thank you very much.
This is seriously awesome
Is there anywhere we can download the source code for this video? Or even clone the repo?
Yessss ! Keep like that !
hey man did you get the foodspawner and foodspawnerenv files?
It's fun to see how the ia exploits bugs. Like at 15:02 it goes in diagonal because the input vector has not been normalized.
Oh wow I never noticed that! Yup it apparently learned diagonals are faster, awesome!
Thank you for these ML-Agents videos.
For me the topic of board game AI training would be interesting too. For example, teaching AI to play Go on unusual boards.
It would probably be based on imitation learning and self play, I guess...
Manual change of velocity can work randomly. Even you don't change code for velocity game can run different.
Awesome! Could you try replicating the hide-and-seek game from OpenAI? Or just any other adversarial multi-agent game?
Yup, adversarial agents is one of the next topics I want to research. They have a functioning Soccer game in the examples which seems to work very well.
@@CodeMonkeyUnity nice, I'm looking forward to it
I wish you get 2 million subs by the end of this year
That would be quite the achievement!
@@CodeMonkeyUnity indeed it is
Ai is amazing ♥
amazing video!!
Cool tool! It would be interesting to make very smart "enemies"for the game. Of course if I have enough brains and time to use it in a real project )))
Yea, especially AI in platformers. It's the one AI that I never understood how to implement, and it would be neat to see how it's done.
When I use Imitation Learning with behavioral_cloning parameter, Unity has like 0.001 fps. Any tips how to fix it? Should I left only GAIL parameter? With GAIL only, it works fine though, but learning process looks weird
Hi, I was watching your video and I saw till 4 rth section and had a thought that if the agent's position is reducing from the button then give the agent half (0.5f) reward, and after it pushes the button give it full reward (1f). So the training will be faster, here is the code for checking if the agent is moving closer to the button or the food: public float dist; float distTemp = Vector3.Distance(destination.position, transform.position);
if (distTemp < dist) {
dist = distTemp;
AddReward(0.5f);
} else if (distTemp > dist) {
dist = distTemp;
AddReward(-0.5f);
} Might Be helpful :)
Yup you can add a reward if the agent is moving closer to the goal.
However it should be a much smaller reward than that. With that value the agent gets more of a reward by moving closer for 2 frames than by pressing the button. So over time it might learn to just move closer and further away and repeat that for infinity.
@@CodeMonkeyUnity I learned this the hard way back when I made a similar project in the early beta version of AgentML.
Something more like AddReward(10f/(distance + 10f)) is a lot safer
saw the ad , proceeded to completely watch the ad
thanks for your video
can add more data in the demonstration file after pause the record?
These tutorials are incredibly helpful, I fail to see how not including a link to a repo to clone is logical at all though
I do provide the project files for videos where I can. For these complex machine learning videos there are too many dependencies to share the project.
I have always had a question about mlagents: they randomly select actions at the beginning of training. Can we incorporate human intervention into the training process of mlagents to make them train faster? Is there a corresponding method in mlagents? Looking forward to your answer.
I follow your step to set yaml,but I find this error:
'behavioral_cloning' is not a valid RewardSignalType'
Gotta make sure its indented correctly. Right now you have it underRewardSignalType but it should be under file name
@@revilorere i didnt notice it and it could have take hours to fix, thank you
Hello, I would like to ask which link is available for courses on reinforcement learning and imitation learning? I haven't found it yet
Great video, I have the reinforcement learning running but I am confused by the heuristic function, when does this get called is it any time RequestDecision is called when we run in Heuristic behaviour mode? I have it translating key presses into actions so I can record a demo for imitation learning but when I record my key presses aren't registered and consequently no actions spawned.
Dose anybody know why my model stops learning no matter how long I let it run. in some cases, the Mean Reward goes all the way from something like 0 to 400 then back down to around 0. I was wondering if somehow it grows too big and runs out of something?
I don't get it, you never reset the foodbutton. Been stuck on this for over an hour. OnEpisodeBeginEvent will not trigger the OnResetButton.
Hey CodeMonkey. Is it possible for Imitating Agent to get observations from mlagent's raycast or grid sensors? My agent's navigation is based hugely on m baske's Spatial Grid Sensor. and its been quite a while now its not showing any improvement during learning.
Why did you change the script to use only discrete actions? Movement was a continuous action in the previous video. As far as I know you should be able to use both discrete and continuous actions in your ML agents.
hey code monkey.. can i ask something about learning method?
can you add more difficulty to already trained onnx?
let say you add wall in the middle after the agent push the button
or when button pressed agent and ball will randomly placed
Hi!
When does the bot get rewarded when using the gail? Is he trying to repeat the sequence and time between rewards or is he just repeating the recording?
Monke do thing
Great Work! I'm not realizing how the architecture of project is build :( I need check the prefabs and scripts to undestand all the picture so could you please provide a link to github to this project?
Please create a big series on game development from scratch
I've made Flappy Bird and Snake from scratch.
And all my courses are also complete step by step guides from start to finish
I have a question, I'm not overly familiar with ML but after watching this video it seems very useful. Now I'm just wondering if something like this can be used on something much more complex (Ex: Training a platformer AI or a 2d shooter AI). What do you think? Do you think something like that would even be possible? or would it introduce too many unknown variables and cause the learning to fail?
It can be used on anything. As long as you give it enough data and enough time to train it can learn to do anything.
Check my latest video on using it to play Flappy Bird th-cam.com/video/fz8D0OZkQGQ/w-d-xo.html
Stay tuned for a video on driving a car and more complex examples
is it possible to exclusively use GAIL? Im trying to compare RL vs IL and using GAIL as my representative for IL, but since it also uses extrinsic rewards it doesnt seem like a fair comparison.
After about 20 seconds into training my agent from behavioral_cloning/gail, pytorch crashes and I get the error 'Class values must be smaller than num_classes'. There's no error in Unity. This doesn't happen if I remove behavioral_cloning and gail from the config file. Anyone know what's happening?
Edit: I have narrowed it down to behavioral cloning, not gail
I am having the same issue
How could you handle a game that was difficult to play manually, such as a game of tennis where the bat angle had to be adjusted?
please make a video on "How to implement multiagent from scratch?" with mlagents toolkit.
How do you create machine learning for for a grid based movement with turns between the player and the AI? Can you make 3d tutorial based on turn based grid game with the player having turns against the AI
Hello sir, huge fan
Can I ask something
Pls make "Journey of the Priarie king" with unity,pls pls pls
@Code Monkey Can you show us the foodSpawner code? I can't figure out what to put in it.
Just Instantiate the prefab and make sure the Agent knows where it is
Ohh what happens when it's a number like 21 and they either keep the cards or discard but the can't go over or be short
Please make a video about inheritance
Hi codemonkey, I recreated the same as you did for immetation learning. But I am facing 2 issues.
1. OnEpisodeBegin is executing continuously on the loop. I can;t find why its happening so I put a flag to stop that actions. there I noticed the 2nd issue.
2. If I play the game in editor before starting the mlagents-learn on cmd the agent movement based on my input is working fine. if I did the same after entering the cmd mlagent-learn on cmd means. the Agent movement speed in unity is 200% higher.
can you please help me to solve the issue?
Only script I did on my own was FoodSpawner script and Food button script. FoodSpawner script was not showed in the video and foodButton is not completely displayed that's the reason.
Note: if I test the script in unity editor before starting the cmd means its working as expected. the issue is only occurring after starting the mlagent to learn
Do you think we can do everything with machine learning or some IA can only work with state machine ?
pls put some video on Android Scripts and tools 😊
Pretty much everything I cover in every video works the same for Mobile. The only difference is the input system.
After I adjust everything in the config file for the next run how do I "update" the brain? I'm not sure about the commands to use for Python etc and what to do in the inspector in Unity.
You can use --initialize-from=RUN_ID or --resume
I covered that in the getting started video th-cam.com/video/zPFU30tbyKs/w-d-xo.html
Hello!Thank you very much for your video.
Could you please show me the "foodSpawner" script and the foodSpawnEnvironment script?
Can these all be done, while having the curiosity model also?
Yes
Hi, so to enable GAIL and Behavioural Cloning, we simply just change the Config file. Right? I heard you mentioned about the Discriminator for GAIL. Do we need to write the code for it?
Yes you just change the config, all the Agent code is the same.
@@CodeMonkeyUnity Thanks a lot!
Hi, can i get the source code of this project to practice imitation learning?
Every time my agent has to make a decision, unity just doen't respond. After 10 seconds it continues until it has to make the next decision. But when I'm not using a recording, it runs smoothly...
I noticed that I have way less episodes in way more steps (several thousand steps but only like 30 episodes) in my recording. I had to put max steps to 0, or else I wouldn't be able to record a single good episode. 1000 Steps is less then a second for me. Does anyone know why?
root.DecideAction takes 197289ms, that's why Unity is not responding. Then it runs smoothly. Then again...
GPU and CPU are
Great videos man but please, for the love of god, use dark mode
Sorry I can't, dark mode burns my eyes
@@CodeMonkeyUnity Funnily enough, I have strong photophobia (kind of the exact opposite to what you experience) and I get horrible eye strain (or just don't see at all) when the screen is too bright...
because i by need to make bots for my fps game i have
my qustion is what method do you the better ? my other qustion is passable to use both of this methods
Depends a lot on the game, sometimes classic AI is the better approach and sometimes ML is better.
Also depends on your familiarity with the tools, how familiar are you with classic AI? What about ML?
Yes you can use both methods at once, like for example using ML to decide what to do and using classic AI to do that action.
@@CodeMonkeyUnity ok form i need to make the game play wean i make the game play and the multiple same time i will check haw to make i bot system with help or not
may i know what kind of ML agents version your download , is it can be done with 1.0.2 version?
ML agents version package manager at your Unity3d
I think I used the same version I installed in the first ML Agents video unitycodemonkey.com/video.php?v=zPFU30tbyKs
I'm sorry im new in this but i wanted to see how you relation every item with the script or how do you do the button to function and i don't see it in this video so i don't really understand what is happening do you have a step by step or the project?
It's just a simple physics OnTriggerEnter th-cam.com/video/Bc9lmHjqLZc/w-d-xo.html
@@CodeMonkeyUnity How do you relate the prefab ball to the button? I need it hidden in the unity environment and link it from there or how do you make that link?
@@laurasofiapadillacabrera4912 Can be a simple SerializeField
Sounds like you need to learn the basics of Unity and C# before attempting Machine Learning which is a pretty complex topic
th-cam.com/video/E6A4WvsDeLE/w-d-xo.html
th-cam.com/video/IFayQioG71A/w-d-xo.html
I have a question
If I start with bolt and then slowly move to c# will it be better or to direct learn c#
And if I pick bolt will I have to start over for c# ?
Start with whichever one you feel most comfortable. Bolt is perfectly fine to start with, just choose programmer naming and later on it will be easy to transition into C# unitycodemonkey.com/courses.php?c=visualscripting
Tnx bro
@@CodeMonkeyUnity your course is very nice
Where to get these config yaml files for machine learning?
From their GitHub and then I modified them
It is possible to download the project in the video ?
It's pretty simple so I didn't make a package for this one. You can grab the visuals from the official Unity ML-Agents examples.
@@CodeMonkeyUnity Please share the FoodSpawner & FoodSpawnerEnvironment Scripts for let us learn
Sir easy road v3 pro has some bugs i think please make a tutorial on it 🙏🥺
I almost copied the same code form the video, my AI looks like it's learning at first, but when reaching around an average of 1 for the reward in the learning curve, the AI be like, you know what, I think dancing around the button seems more fun, and the curve gets worst from there on, anyone faced such a thing ?
Hi, how can I access project files?
EDIT:
I think I got it now the problem is that the food Prefab is Spawning like every second :(
Hello, can anybody help, I'm having trouble with the following method GetLastFoodTransform(), it´s giving me the following error. I must say that I'm pretty new at coding so forgive me if I´m doing something pretty stupid .
NullReferenceException: Object reference not set to an instance of an object
FoodSpawner.GetLastFoodTransform () (at Assets/Scripts/Food Collect/FoodSpawner.cs:29)
FoodAgent.CollectObservations (Unity.MLAgents.Sensors.VectorSensor sensor) (at Assets/Scripts/Food Collect/FoodAgent.cs:42)
Unity.MLAgents.Agent.SendInfoToBrain () (at Library/PackageCache/com.unity.ml-agents@1.7.2-preview/Runtime/Agent.cs:1048)
Unity.MLAgents.Agent.SendInfo () (at Library/PackageCache/com.unity.ml-agents@1.7.2-preview/Runtime/Agent.cs:1320)
Unity.MLAgents.Academy.EnvironmentStep () (at Library/PackageCache/com.unity.ml-agents@1.7.2-preview/Runtime/Academy.cs:557)
Unity.MLAgents.AcademyFixedUpdateStepper.FixedUpdate () (at Library/PackageCache/com.unity.ml-agents@1.7.2-preview/Runtime/Academy.cs:43).
And this is the code I put in that method,
public Transform GetLastFoodTransform()
{
lastFoodTransform.localPosition = food.transform.localPosition;
if(lastFoodTransform)
{
return lastFoodTransform;
}
else
{
lastFoodTransform.localPosition = new Vector3(Random.Range(-2f, +2f), 0, Random.Range(-2f, +2f));
return lastFoodTransform;
}
}
Hello, can you please show us FoodSpawner script?
It instantiates the prefab, that's pretty much it
@@CodeMonkeyUnity Thank you, you're amazing. Keep up the good work!
where do i found the course for mlagents?
I don't have any ML course, just a bunch of free tutorials th-cam.com/play/PLzDRvYVwl53vehwiN_odYJkPBzcqFw110.html
So... Why didn't you use the GPU?
Where did you get that character from?
It's from the Unity MLAgents samples, or you can just build it yourself, it's really just a cube
@@CodeMonkeyUnity still looks better than just a cube
my game runs incredibly slow when i try using gail and bc, any ideas why?
actually suddenly out of no where all kinds of training make unity run at 3fps, was quick only an hour ago, whats going on lol now i have to give up since its so slow
and it didnt follow my demo AT ALL, i think ML is still a bit poor unfortunately, its now pausing unity editor every 10 seconds.
It should be slow while training, should be fast while just running a trained model.
Maybe look in the profiler to see exactly what is slow?
@@CodeMonkeyUnity yes thanks, it went quick again not sure what was happening :) I almost got a little guy playing my game thanks to you!
I'm having a lot of trouble with the food spawner. I love the course idea, but without the project files, you waste a lot of time in the environment, and end up losing focus on the central topic that is ml agents. Anyway, if you can guide better about the food spawner, I would be very grateful.
My food prefab always instantiates in the same environment, like a clone, all food objects instantiate in the same environment simultaneously, I have 15 environments in massive training, but all foods instantiate only in the main one, I need help to solve this, could you help me?
Would it be possible to make "fair" Ai.
The job of the AI would be to be an NPC.
Could i make it fair so it does not hit like every shot or uses some moves that are just unbeatable. I read about this in the unity forum and they said it would not be possible.
You can train an AI to the point where its perfect, then instead of using the very last brain, you use one of the previous brains. So it will be good but not perfect.
@@CodeMonkeyUnity thank you
Can you make a game for play store and app store
Hi, are there any source code ?
please provide the scripts, it would be of great help...Thanks
💬 Here's how you can teach your agents how to complete more complex tasks.
Stay tuned for more ML videos like Driving a Car and playing Flappy Bird!
📦 Complete Unity Machine Learning Playlist: th-cam.com/play/PLzDRvYVwl53vehwiN_odYJkPBzcqFw110.html
Hey Mr. Code Monkey I have a request to you to make a video about some checkpoints system. To be clear I am making a game and for efficiency and lag compression I want to disable rooms. I have some rooms and I only want some room to be rendered. I had an idea of a box collider 2d set to trigger but I don't know how to find the way the player is moving. For eg if the player is moving to the right I want to disable te room in the left. I don't know how to check which part of a collider is hit. Apart from that it is a box so I have no idea how to find where the player is going and reduce the lag as possible. Hopefully you get my point. incase any doubts please reply me. Hopefully I see a video on it. Btw Love your videos. Love from India
Wherever you are moving the player you have the new player position and the previous player position, with those two you can get the movement vector and see which direction the player is moving in.
Then for disabling rooms that is highly dependent on your specific use case. It can be as simple as going through all the rooms and disabling/enabling them one by one.
@@CodeMonkeyUnity yeah sir they are actually in some game objects so I will just do setactive(false);
where is foodspawner script
There isn’t any, he wants that we do that on our own
switch to dark theme code editor please!
Sorry but my eyes can't handle the dark theme
@@CodeMonkeyUnity Blasphemy!
:O did you just assume my species? 1:44
Problem is, if I suck at my game, the AI would imitate that.
It would, but then you would enable Reinforcement Learning and it would quickly surpass your own skills
@@CodeMonkeyUnity So it's okay to potentially fail at playing my own games?
How many iterations did it take to model be trained?
Hello @Code Monkey!
I know this field is pretty to new to gamedev, and, requires lots of experience.
In up coming videos, Can y make some masterclass about creating "custom tools" for ML.?
For instance, my interest is in "character joint physics tools" for ML. To be clear, I want to create custom Brain in future for my research. But y'know, the editor and environment interactions take too much time .In this time, I can easy forget about working with ML brain and pay close attention to automation Physics engine tools for the sake of future.
So I think, we should also pay close attention to common logics that saves time to interaction between ML brain and environment logics. Then, it will be easier to make efficient ML technology to use everyday game development time.
You dont need tools specific for ML, everything that Unity already has already works with ML.
You can create a Physics Joint then connect it to ML Actions and Observations.
Check out the official example project, they have a Crawler example that uses physics joints to move a spider-like creature.
this code doesn't even work, tons of errors after manually typing it all after pausing a million times since you didn't post code. other videos were helpful but man, how frustrating...
What errors? What version are you using? ML-Agents is constantly being updated so if you're not using the same version as me you need to look in the changelog to see what changed.
@@CodeMonkeyUnity Posting a link to the code would be helpful though.
Hi
U copied serial apeist monkey see monkey kill from bbt right lol
lets be fair, it would be amazing if it worked. but it doesnt ;-/
How does it not work? I haven't touched MLAgents in ages but it worked when I made this video
@@CodeMonkeyUnity i dont know, im giving up on it, wasting too much time. im now trying to record videos of me playing but that lags, tried so often to get some simple replay system be it ai or human but its too difficult.
did you just assume Im a human being?
Hello, I have currently problems with a bug. As I was finished with saving the new .yaml, I run the ml agents and it sais:
yaml.scanner.ScannerError: while scanning for the next Token found Character „\t“ that cannot Start any Token in „config/MTG.yaml“, line 24, column.
But there is nothing…
behaviors:
MoveTo:
trainer_type: ppo
hyperparameters:
batch_size: 10
buffer_size: 100
learning_rate: 3.0e-4
beta: 5.0e-4
epsilon: 0.2
lambd: 0.99
num_epoch: 3
learning_rate_schedule: linear
beta_schedule: constant
epsilon_schedule: linear
network_settings:
normalize: false
hidden_units: 128
num_layers: 2
reward_signals:
extrinsic:
gamma: 0.99
strength: 1.0
gail:
strength: 0.5
demo_path: D:\Unity\Games\ML_AI\Demos
behavioral_cloning:
strength: 0.5
demo_path: D:\Unity\Games\ML_AI\Demos
max_steps: 500000
time_horizon: 64
summary_freq: 10000
Column 7.
delete the whitespace and make sure all the characters are in the right columns
For anyone else facing this, yaml does not allow tabs, only spaces for indentation
LOVE your videos man!!!
Thanks!
Where can I download the code in the video screen