Monte Carlo Tree Search
ฝัง
- เผยแพร่เมื่อ 13 พ.ค. 2024
- This is a video I made for my class "CS310: Foundations of Artificial Intelligence" at the University of Strathclyde. The video has a brief description of the Monte Carlo Tree Search algorithm and includes a worked example.
This is a ridiculously good explanation for MCTS. Keep teaching, you're really good at it
Best explanation for MCTS
I watched this vid like 10 times to make my monte carlo agent. thnx man
You're very welcome!
The visualization make this algorithm so easy to understand! Thanks for your great work, professor!
Honestly, the best explanation for MCTS so far. This helped greatly, thank you ☺
Oh my god, this video is so great, the explanation is so clear, all with flowchart , pseudocode, visualization. Thank you for providing this wonderful learning material
After reading about this method, the rollout was confusing. The verbosity of your example completely eliminates that confusion. Thank you.
I came here after watching many vids on MCTS and none of the videos were able to make me understand the concept but this video is the best
Your videos are so helpful. Thank you for posting these online!
This is the best step by step explanation of MTCS! You save my day! Thanks!
The best explanation. Thank you!
Thank you for your explanation. Your video was the only source where I could understand this algorithm. It really helps writing my seminar work
You're very welcome! Glad it was of use.
Great video. It was well explained and easy to follow. I'd love to see other videos related to deep learning if you have them.
Thank you Prof! The worked example makes the absolute difference!
This is pure gold! Thank you infinitely
Great explanation of the algorithm, showing all the elements which are needed to get a basic understanding of MCTS. Thank you.
Very clear explanation of the algorithm and the video is also well made! Thank you!
What you did is admirable sir, great thanks for the explanation. Saved my day.
What a great explanation! I've heard other explanations, but this is the condensed one I'll refer to in the future!
Sir, thank you so much for the explanations. Even dumb like me now feels like I will now apply MCTS to my tictactoe game! If teachers like you taught me when I was a kid my life would definitely be different. Just can't thank enough for this lecture!
Incredibly clear, after just one watch i now understand MCTS
Thank you so much for the motivation this gives me to not give up, its a lot easier to believe in myself after understanding these concepts!
Very clear lecture. Great explanation! Thank you!
a monument to clarity. thank you very much.
By far the best explanation I've seen. Thank you.
Thank you so much! I'm working on my FYP and this is the best explanation of MCTS out there!
Thank you for the great lecture.
By far, the simplest explanation of MCTS. Thank you so much.
Very good presentation. Thank you! I really needed clarification on whether to increment visit count BEFORE or AFTER backpropagation. Thanks for this :)
Great video. Explaining things clearly is hard. You make it look easy John.
Great video! Helped me a lot, thanks.
Truly incredible explanation.
Thank you so much, because this is the first source I find that gives a true example.
So once again, thank you so much. I think I'll now be able to implement it in C for my connect 4 :)
Thank you for such a clear explanation of MCTS.
One of the best explanations of MCTS
Beautifully explained
Great lecture and vivid example! Thanks a lot! ^_^
life saver , great run through
Extremely helpful! Thank you.
Really great lecture. Continue the good work :)
At the beginning, your root node's visit number is 0, but you expand it instead of rollout. I know it is the right way but then you should alter your algorithm. At the left board you said "rollout if the ni value is 0"
You can do this by saying the root node has the visit value 1 from the start.
Don't you expand and then rollout just like he did, meaning that the visits would be 0.
Your explanations are much better than the Wikipedia page. Congrats! Thought that MCTS was only applicable to games in which you have an opponent, glad to see it is not the case and will apply it to my pb. Thanks!
Thanks a lot for the lecture, its was really helpful :)
You're very welcome!
Thank you for the video, you explained very well the single steps. That helps me a lot, most of all the expansion step was the one I didn't understand until now.
You're very welcome! It took me a while to work out how the expansion step works, since the standard explanation offered is a bit unclear on this point.
Amazing explanation!!
Great Explanation!! Thank you alot!! Keep the good work going!
The best explanation I can find.
5:25 - It's not the smartest idea to take them in numerical order it will somewhat bias the algo, I think a better policy would be a random uniform one or to put some priors over the actions using some heuristic or better yet an ML model (maybe a policy dnn like in the case of AlphaGo) in case you know some specifics of the problem you're trying to solve with MCTS.
It's also maybe worth mentioning that the expansion threshold doesn't necessarily have to be 1, it's the hyperparam of MCTS. So maybe you'll have to visit the node 40 times before expanding it.
Finally in order to pick the best action at the end sometimes you don't want to choose it looking at the average reward of the child states but simply take the visit count - but I guess this is problem specific.
Loved the walk-through, thank you John! Great pedagogical methods you've got.
Thanks for the insights!
Thank you! Explained it better than my professor ever did
Jeez. You are the best!
amazing walkthrough, thank you
Wow, this is liquid gold. Thanks John.
Nice to hear from a fellow brit and not MIT OCW! Big up America too tho..
Bro u rocked, alot respect for u. Thanks
Thank you for the explanation. It was clear!
Thanks a lot for sharing this video Sir, It was really very help full.
Great explanation, thanks
really helpful, thank you!
The content is so exciting even the camera shares the feeling.
Clean explanation!
Right on time for end of term, great video
Thanks! Glad you found it useful.
Great explanation of MCTS
awesome explanation. thank you
Thank you, this is very helpful 👍
Best possible explanation...
Great explanation.
thanks for sharing the lecture 😊
Thanks the best explanation of MCTS. I wish to understand how it would work with a game with multiple players.
Super clear explanation!!! 🌟🌟🌟🌟🌟
Very nice explanation, thank you
Perfect video
First of all, thank you for the excellent lecture and explanation! The algorithm is very clear and understandable from the way it's presented.
My question is: In the end you mention that what this gives us, is that we should take action a2 over a1. Why can't we take it a step further and also infer that, for example a5 is better than a6 (assuming we have visited them both more times than what is presented in the example). Why do we only use a whole search tree for just one decision, and then scrap it and rebuild it anew, instead of making more sequential decisions using the same tree?
So clear! Thank you!
thank u for helping me in my AI studies... ure great.
fantastic!
Brilliant.
Thank you very much.. You are great! ❤️
Very good and easy to follow explanation!
Thank you! Glad you liked it.
Thanks a lot, it's really clear!
thank you very much, it helped me a lot!
Great Video!
Thank you very much! My game of othello is fully working now! :)
I really appreciate the algorithm trace.
really great video!
Perfect!
Very nice explanation! Thank you!
I have a question: what should I do if I am in the leaf node and I should extend it, but the state of this node is terminal?
Super clear thank you.
Thank you so much ! The explanations are very clear, you make it sound like it's easy! What's with the human calculator by the way ?
Straight forward explanation
John levine and josh starmer are carrying me so hard my feet aren't even touching the ground
Thank you sooooo much!!!!
really helpful, like it
Nice lecture!
Thank you!
Thank you so much for this video, really helped me a lot.
I just have one question : when expanding, why do you only roll out from one of the children? Wouldn't it give a more accurate score for the parent if we simulate all its children?
Good question. In general, MCTS is trying to make the best decision it can from the limited number of rollouts that it can make in the time available. Therefore, every time it does a rollout, it updates the whole tree and then uses this updated tree too decide where to sample next - it could be that sampling only a single child will tell us that the node above is a poor choice, and so we will start sampling elsewhere in the tree.
Great video
Dear Prof. John, thank you for the amazing video on MCTS. Where ca we find the pseudocode for the MCTS? Which materials do you suggest for this?
Didn't understand why after backpropagation on S5, S0 became 44 if it was 30 and S2 = 24. I thought we should add S0 = 30 plus S2 = 24 so S0 would be 54!
By the way, best explanation EVER! Thanks a lot for sharing.
Pedro Coelho I believe we just backpropagate the value we obtained after the rollout. So if we got 14, we just add that 4 to each of the parents all the way to the root.
This will go in my bachelor thesis about alpha go zero :) Thanks so much for the explanation! Do you have some sources for citation on MCTS maybe?
Really useful.
Amazing explanation. Thank you. But i'm not sure how to get terminal value of leaf state. Is it average score you get after you do random actions from this point?