I must say that in more than 6 months, this is by far the best lecture/ material I have come across that was able to make me understand what policy gradient method actually is. I really praise this work. :) Thank you.
Very good lecture about policy gradient method. I have looked through a lot of articles and was understanding almost everything, but your derivation explanation is really the best. It just opened my eyes and showed the whole picture. Thank you very much!!
I might be missing a simple concept here but how are we increasing/decreasing the grad log probability of the actions using the gradient of U(theta)? I get that positive return for a trajectory will make the gradient of U positive and so theta will be increased in favour of those trajectories but how is it increasing grad log prob?
1:11 why are DQNs and friends Dynamic Programming Methods? I mean the neural network works as functions approximator to satisfy Bellmans eqn. , but still Backprop is the workhorse. In my opinion DQNs are much more similar to PG methods than to Bellman Updates??! And another issue with RL Landscape slide is where the heck are model based RL algos?? This slide should be renamed to model free RL landscape.
could you train a robot for 2 weeks in the real world then use those trained parameters to optimize a virtual environment? You know.. making the virtual environment very close to the real world?
Real world is very complicated with model uncertainties, friction, wear and tear and what have you... Simulators can come close , but we cannot expect them to fully mimic real world phenomena.
After 5 years still, the lecture is a great video, thank you a lot
This is the best lecture I have seen on policy gradient methods. Thanks a lot.
Every time I forget how policy gradients work exactly, I just come back here and watch starting at 9:30
And every time you visited this video, you forgot where to start watching. Thats why you posted this comment. Smart guy!
I must say that in more than 6 months, this is by far the best lecture/ material I have come across that was able to make me understand what policy gradient method actually is. I really praise this work. :) Thank you.
Amazing lecture! Love how Pieter explains the math. super easy to understand.
Great Lecture!!!! Pieter's explanations are just a gem!
Best lecture on Policy Gradients hands down. Has covered some worth noting superficial details of many papers as well.
It's such a good lecture, I'm stopping to ask myself why it was so easy to cover such significant information with full understanding
Very good lecture about policy gradient method. I have looked through a lot of articles and was understanding almost everything, but your derivation explanation is really the best. It just opened my eyes and showed the whole picture. Thank you very much!!
Honestly I can’t keep up without seeing what he’s pointing at. Gotta pause and search around the screen each time he says “this over here”
Exactly. "This over here" has got to be the most uttered phrase in this lecture. So frustrating.
amazing work. super understandable, concise and information dense.
The explanation of the derivation of policy gradient is really nice and understandable here
I got a lot out of this lecture in particular. Thank you.
best lecturer in series
Wow, this is beautiful!
why a good "R" would increase the probability of path??? Please help me
Awesome instructor.
Instead of having a baseline, why not make your reward function be negative for undesired scenarios and positive for good ones? Great lecture!
wow damn this is so well explained and the last video is very entertaining.
I might be missing a simple concept here but how are we increasing/decreasing the grad log probability of the actions using the gradient of U(theta)? I get that positive return for a trajectory will make the gradient of U positive and so theta will be increased in favour of those trajectories but how is it increasing grad log prob?
23:20- Gradient of expectation is expectation of gradient "under mild assumptions". What are those assumptions?
math.stackexchange.com/questions/12909/will-moving-differentiation-from-inside-to-outside-an-integral-change-the-resu
nice but hard to follow without knowing what "this" refers to. I hope my guesses were right :)
The guy in the background at 51:30
excellent lecture. Thank you for sharing.
Where can i buy his T-shirt?
It would be nice to see what laboratories they had (what exercises)
dont know if ur still interested, but the labs are on the bootcamp website
Great lecture, thanks
1:11 why are DQNs and friends Dynamic Programming Methods? I mean the neural network works as functions approximator to satisfy Bellmans eqn. , but still Backprop is the workhorse. In my opinion DQNs are much more similar to PG methods than to Bellman Updates??! And another issue with RL Landscape slide is where the heck are model based RL algos?? This slide should be renamed to model free RL landscape.
PG is awesome!!!
Doesn't depend on environment Dynamics really?? Wow
All the pain and stress just goes away when we see our algorithms working😇😇
great talk!
Awesome!! Thanks
If you can't explain it like Pieter Abbeel or Andrew NG then you don't understand it well enough.
could you train a robot for 2 weeks in the real world then use those trained parameters to optimize a virtual environment? You know.. making the virtual environment very close to the real world?
Of course you could. It's the opposite of what OpenAI does when they train a model in a virtual environment and deploy it in reality.
Real world is very complicated with model uncertainties, friction, wear and tear and what have you...
Simulators can come close , but we cannot expect them to fully mimic real world phenomena.
very nice :)
Can you share ppt??
You can find all the slides and the other lectures here:
sites.google.com/view/deep-rl-bootcamp/lectures
... you can use "a", and the math will be the same. :)