Thanks! I've made the github repo public, so hopefully you will be able to find it all here: github.com/ThomasLynn/Stickman-Walker Good luck on your project!
That's a good question. I was planning on making this video into a full video explaining it all, but I don't yet have the script writing skills to make it good. The way to find a good strength is: first get a good starting value (normally 0.01), then do a little trail and error I trained with 0.01 curiosity (which is in the middle of the typical range 0.001-0.1 from the docs) and got ~4 extrinsic reward (~40 metres / 10) and ~1 intrinsic reward, so ~20% of the total reward is from the curiosity. Normally 20% is pretty high, but I'm using curiosity to change how to stickman walks, instead of using it to explore an area passively, so it needs the high % for this to work. I tried a few values (in the order I tested: 0.01, 0.1, 0.05, 0.02, 0.03, kind of like a binary search) At strength 0.01, the curiosity was too small to affect training. Strength 0.02 sometimes did, sometimes didn't affect training. Strength 0.03 and 0.05 always led to walking instead of galloping. When I tried strength 0.1, it did learn to walk, but it learned so slowly that it ran out of training time before getting to the second obstacle. Basically, any strength 0.03 or above always gave walking, but the higher I went over 0.03, the slower it trained (about 80% training speed at strength 0.05, and ~40% training speed at 0.1). So the reason I use 0.03 in the video is because it worked, and it trained the fastest out of the 3 that did, giving it more time to perfect its walking. docs link: github.com/Unity-Technologies/ml-agents/blob/release_14_docs/docs/Training-Configuration-File.md#curiosity-intrinsic-reward also, I got the intrinsic strength information using Tensorboard. It's pretty useful, I would recommend it.
Well, me too =/ still can't figure out how to script this one. Thought I'd upload something though, as it was getting to a point of me just staring at a crappy script for days on end.
The Chad Curiosity vs Virgin No Curiosity
nice job man!
When will you come back and make more machine learning videos?
is it balancing as well?
Very cool. Is this project on github? I'm trying to do something like that but I'm not getting it.
Thanks! I've made the github repo public, so hopefully you will be able to find it all here: github.com/ThomasLynn/Stickman-Walker
Good luck on your project!
@@zoelynn126 Thanks!
So you used curiosity strength at 0.03? How did you settle on this number?
That's a good question. I was planning on making this video into a full video explaining it all, but I don't yet have the script writing skills to make it good.
The way to find a good strength is: first get a good starting value (normally 0.01), then do a little trail and error
I trained with 0.01 curiosity (which is in the middle of the typical range 0.001-0.1 from the docs) and got ~4 extrinsic reward (~40 metres / 10) and ~1 intrinsic reward, so ~20% of the total reward is from the curiosity.
Normally 20% is pretty high, but I'm using curiosity to change how to stickman walks, instead of using it to explore an area passively, so it needs the high % for this to work.
I tried a few values (in the order I tested: 0.01, 0.1, 0.05, 0.02, 0.03, kind of like a binary search)
At strength 0.01, the curiosity was too small to affect training. Strength 0.02 sometimes did, sometimes didn't affect training. Strength 0.03 and 0.05 always led to walking instead of galloping. When I tried strength 0.1, it did learn to walk, but it learned so slowly that it ran out of training time before getting to the second obstacle.
Basically, any strength 0.03 or above always gave walking, but the higher I went over 0.03, the slower it trained (about 80% training speed at strength 0.05, and ~40% training speed at 0.1).
So the reason I use 0.03 in the video is because it worked, and it trained the fastest out of the 3 that did, giving it more time to perfect its walking.
docs link: github.com/Unity-Technologies/ml-agents/blob/release_14_docs/docs/Training-Configuration-File.md#curiosity-intrinsic-reward
also, I got the intrinsic strength information using Tensorboard. It's pretty useful, I would recommend it.
I’m disappointed by the lack of talking
Well, me too =/ still can't figure out how to script this one.
Thought I'd upload something though, as it was getting to a point of me just staring at a crappy script for days on end.