First of all, how do we know that j(theta) curve is of that kind? The curve will be obtained only after the gradient descent worked upon or run random 1000 or so theta values...
Overfitting is not really related to the *method* of doing the optimization (online=stochastic GD, versus batch GD, or second order methods like BFGS, etc.) but rather to the complexity of the model, and the *degree* to which the optimization process is allowed to fit the data. So, early stopping (incomplete optimization) can reduce overfitting, for example. Changing optimization methods can appear to change overfitting simply because of stopping rules interacting with optimization efficiency, but they don't really change the fundamental issue.
+Umesh Morankar Then it might diverge or offshoot from the minima..... for eg. if the minima is 2 ,the latest value of Θ = 4 and α = 8 (suppose) , then it would diverge to 4 - 8 = -4 which is too far away from 2 whereas if α = 1(too small) then it would reach the local minima in the next 2 iterations. I havent applied the algorithm. This is just for explanation purpose.
because it is reference to theta0 therefore derivative theta0*x is 1*theta0^(1-0)*x=1*1*x=x (btw, this is partial derivative, so all other terms before derivative are constant expect theta0*x0)
so, how many of you came here from Andrew NG's ML Course?
i'm here
Me
I'm here.
mee too
yea!
Simple , Clear , and Gentle Explanation Thanks More Prof.
I'm very visual so I particularly loved the visualisations showing the progressive improvement of the hypothesis as the parameters were refined.
Very interesting. Thanks for the clear and vizual explanation that provided me quite good intuition of different version of Gradient descent.
This was the simplest explaination as most beautiful and precise one
Exactly what I was looking for, thx!
Thanks! Making an essay on machine learning, and this really helped me out!
Thanks for such kinda helpful lecture
This helped a lot, thanks
gracias!
thx a lot
excellent
First of all, how do we know that j(theta) curve is of that kind? The curve will be obtained only after the gradient descent worked upon or run random 1000 or so theta values...
Wouldn't online gradient decent cause the problem of overfitting?
Overfitting is not really related to the *method* of doing the optimization (online=stochastic GD, versus batch GD, or second order methods like BFGS, etc.) but rather to the complexity of the model, and the *degree* to which the optimization process is allowed to fit the data. So, early stopping (incomplete optimization) can reduce overfitting, for example. Changing optimization methods can appear to change overfitting simply because of stopping rules interacting with optimization efficiency, but they don't really change the fundamental issue.
nice
What modifications can we make to use l2 norm instead?
what if we set the
learning rate α to too large a value.
+Umesh Morankar Then it might diverge or offshoot from the minima..... for eg. if the minima is 2 ,the latest value of Θ = 4 and α = 8 (suppose) , then it would diverge to 4 - 8 = -4 which is too far away from 2 whereas if α = 1(too small) then it would reach the local minima in the next 2 iterations.
I havent applied the algorithm. This is just for explanation purpose.
Can someone explain how the derivative of ⍬X(t) wrt ⍬ becomes X and not X(t)?
because it is reference to theta0 therefore derivative theta0*x is 1*theta0^(1-0)*x=1*1*x=x (btw, this is partial derivative, so all other terms before derivative are constant expect theta0*x0)
Can i get this slide please