Levenberg-Marquardt Algorithm
ฝัง
- เผยแพร่เมื่อ 24 ก.ย. 2024
- Details of the Levenberg-Marquardt Algorithm and comparison between this method and the Gradient Descent and Newton-Raphson method are provided in this video.
Gradient Descent Problems: 1:50
Newton-Raphson for finding a function's extrema: 5:58
Hessian Matrix: 8:59
Newton-Raphson Problems: 18:31
Levenberg-Marquardt Algorithm: 25:00
MATLAB demo of applying all 3 algorithms to 2 multi-dimensional functions: 43:05
The code for this video can be downloaded from here:
drive.google.c...
I usually don't look at videos longer than 30 minutes but WOW.. I saw it whole and it was amazing. Many thanks to you!
Thank you soo much for uploading this. It means A LOT to every engeenering student in different parts of the world who is struggling to understand this algorithm.
You are welcome. Happy that you like the video. Please share this Channel with your friends.
Fantastic. You made a complex subject seem easier to understand by your way of explaining it in a clear, intuitive, illustrative and easy language. Thank you very much.
Excellent explanation! Your English is very good and easy to understand! Thank you very much!
Incredibly intuitive and helpful. Easily the best way out there to spend an hour to better understand this topic
Very clear explanation for the Levenberg-Marquardt algorithm. Thank you so much!
Thanks for your explanation!! The Levenberg-Marquardt method that balances the converge-speed(Newton method) and converge-robustness(GD)
You are welcome. Happy to hear that you found the video useful. Please share this channel with your friends.
@@mehran1384 Yeah I will😊, Thanks!
Thank you for that video. Excellent explanation!
Great and very clear explanation! Thank you so much for your work
Thanks for the explanation. I will add that this is not LM though, this is a trust region method using GD and NR. While LM is a trust region-based method using GD and gauss-newton (GN). They look similar, but you would end up with x_(n+1) = x_n - (J^T*J + kI)J^T*E_n, where k is lambda, J is the jacobian matrix and E_n is error vector (see GN). But other than that, the explanation on how the weights etc is used is very descriptive.
Hi, where could I look an explanation this clear about the real LM method?
You are right. Strictly speaking, LM method is a trust region based method that solves the nonlinear least square problem. And in which Hessian uses JTJ instead of the conventional second order derivative. And gradient descent is replaced by the error vector.
Great video. Explained with utmost clarity!
thanks. happy you liked it.
Many thanks to you , it was very clear and simple explanation from a professional person. My understanding of this algorithm was stuck in some points (as GD😊😊 ) until this video.
Great, easy to understand explanation. Thank you.
Happy that you found the video easy to follow. Please share this channel with your friends.
Thank you for the amazing video! It helped me a lot!
Thank you so much for this video. Very clear information
I have been binge watching you videos about non-linear equation and their solvers and optimisers. By, every video I am getting more clarity. Your background in teaching students at different levels really helps you explaining very clearly. I question thought, do you think we( as in viewers) get the material from your videos?
Thanks. I am not sure if I understood your question about getting the material? Could you elaborate?
@@mehran1384 The one note notes are what I meant.
Amazing explanations!
Nice Videos with excellent demonstration
Happy to hear that you liked the video. Please share this channel with your friends.
Fantastic. Thank you!
Excellent video
Thank you.
Excellent presentation :) :)
Thank you. Please share this channel with your friends.
great talk and heavily informative.
can you provide the sheet that you are presenting?
Great lecture
Thank you. Please share this channel with your friends.
Awesome video! easy to follow along. One question, is there a way to choose the initial value of lambda? or any value would work?
Sorry for the late response. Since lambda changes by an order of magnitude each time, the initial value of it is not so critical. An imperfect lambda just slows downs the entire convergence by only a few iterations.
That's great!
Thanks for the video! From my understanding the most common heuristic for lambda is to having the increase factor be smaller than the decrease factor. However, I'm not sure that I understand the rational since we expect the algorithm to have more decreasing steps. At some point lambda will reach zero, or at least zero in the numerical sense - can you elaborate a bit more on this point?
Thank you for this great video ,but I'm just wondering,in the matlab code for the gradient descent method, why did you divide by norm(temp)? what's the purpose of it?
You are welcome. Diving by norm gives a unit vector (direction only) of the notion and magnitude of it is determined by alpha.
@@mehran1384 im a bit weak in linear algebra so I'm not sure what is alpha? Also norm(temp) is taking the norm of 2×2 matrix correct? Does dividing by the norm of a matrix also gives us the unit vector like when dividing by the norm of a vector? Because I thought taking the norm of a matrix gives us info about how big the elements are
Is this least squares and levenberg-marquardt algorithm? I see things like Jacobian matrix in other resources...
This is the standard LM algorithm. It has least squares as a part of it.
Great Work! Thank you for the good explanation. Can i get your OneNote Lecture Notes that you showed to us in this lecture?
Nice Video!!!
Thank you.
thank you so much
thank you
You are welcome. Please share this channel with your friends.