ฝัง
- เผยแพร่เมื่อ 3 ธ.ค. 2024
- In this video, and AI is trained with reinforcement learning to accumulate speed and finish a map as fast as possible.
The AI learned a behavior where it turns around right before the finish line. This is not a one-off mistake, the AI repeatedly did similar things in back-to-back runs. Can you guess why ?
Because it was rewarded for speed and not for finishing the track?
It was rewarded for both, but not enough for finishing. You are right 🙂
thats so similar when i play the game myself
Great work tho! I saw the best run and the Wirtual video too. I would like to ask some questions. The AI's job is probably finishing the track as fast as it can, but can it actually "see" turns and the track, or you have to train it again and again if you put it on a new map? I am curious if you can train an AI which can finish any non complicated map at a decent time first time or no.
it can actually "see" turns and the track, as it "sees"/uses 20 fps black and white "screenshots" (idk if they actually go and take screenshots or just have it play on 20 fps colorless or something)
It depends. Usually it works best with a top down camera cause 3d shit makes it a lot harder for AI to know what they're looking at. But some do and some don't. Depends on the model
Do you think this AI could play ZRT Trackmania cup circuits ? i'm wondering if it would be possible, but don't really know how to use it and have the time to try on my side.
Are those circuits available on Trackmania Nations Forever ?
@@linesight-rl I don't think so... but does the AI could work on TM² Stadium ?
@@ArKantiK well well well
What's the best way to contribute changes to the code? I've got a fork with some changes so far.
Thank you for your interest. This is a rapidly evolving 2-person project. At least in the short term, we prefer to keep it small.
I'd be happy to have a look at changes you make, but will likely not merge your branch in our private repo.
We will probably have a more open approach in the future and welcome contributions.
How can we inform you when we decide to open our private repo and welcome contributions?
@@linesight-rl I've got notifications turned on for the channel so if a video is uploaded I'll see it. I'd suggest putting any open source notification there. As far as changes go I've done the following:
Moved to logging instead of print
Optimised variable access
Made resuming last training easier
Moved to automatic timestamp based filenames for saved data
My to do is currently:
Look into using mss instead of dxcam to see if it's quicker, from profiling it seems to screenshot creation is the longest part of the script.
Introduce the surface and wall contact into the reward function to stop it trending towards ramming into the walls.
Upon going onto the grass outside of the track ending the run and heavily punishing reware
Remember me when your viral🙏