Tutorial 27- Ridge and Lasso Regression Indepth Intuition- Data Science
ฝัง
- เผยแพร่เมื่อ 14 ต.ค. 2024
- Please join as a member in my channel to get additional benefits like materials in Data Science, live streaming for Members and many more
/ @krishnaik06
#Regularization
⭐ Kite is a free AI-powered coding assistant that will help you code faster and smarter. The Kite plugin integrates with all the top editors and IDEs to give you smart completions and documentation while you’re typing. I've been using Kite for a few months and I love it! www.kite.com/g...
Please do subscribe my other channel too
/ @krishnaikhindi
Connect with me here:
Twitter: / krishnaik06
Facebook: / krishnaik06
instagram: / krishnaik06
![โรงเรียนดัดสันดาน นักเรียนเถื่อน..!! [โรงเรียนไดเม่]](http://i.ytimg.com/vi/vRKPeXn4Fl4/mqdefault.jpg)
Lucid explanation at free of cost . Your passion to make the concept crystal clear is very much evident in your eyes...Hats Off!!!
Only person who is providing this level of knowledge at free of cost. Really appreciate it .
I never watched your videos but after watching this video I regret for ignoring your channel. You are a worthy teacher and a data scientist.
I'm in crisis to learn this topic and all I know is y=mx+c. I think this is the clearest one I've watched on youtube. Thank you sooooo much and love your enthusiasm when you tried to explain the confusing parts
100 years of blessing for you.
You just gained a subscriber!
You are helping many of the ML enthusiasts free of cost... Thank you
Brilliant. I searched all over the net but couldn't find such an easy yet detailed explanation of Regularization. Thank you very much! Very much considering joining the membership
Thank You bhaiya. It feels like every mroning when I watch your videos my career slope will increase. Thank you for this explaination.
This guy is legit !!
Hat's off for the explanation!!
Loved it sir, Thanks
if you pause the video and just watch his facial movements and body movements, he looks like hes trying his best to convince you to stay with him during a break up. Then you turn on the audio and its like hes yelling at you to get you to understand something.
Clearly, this man is passionate about teaching Ridge regression and knows a lot. I think its easier to follow when hes like checking up on you by saying, you need to understand this, and repeats words and uses his voice to emphasize concepts. I wish he could explain other things to me besides data science.
he will explain u the meaning of life too
my next project is counting head shakes in a youtube video .
@@MrBemnet1 Ngl that actually sounds interesting.
i have tried 4 very reputed organizations for doing courses all claim faculty from IIT and xyz high profile name. My feedback is if you are from IIT then that doesnt mean you are a good teacher, for teaching they should have passion like you had.
When i see your lectures i enjoy learnings. Thank you
Thank you soo much Krish, Linear regression + Ridge + Lasso cleared my concepts with your videos.
Thanks Krish for explaining the intuition behind Ridge and Lasso regression. Very helpful.
Now I finally got about key L2 and L3 difference. Thanks a lot!
Great video! I appreciate how hard his effort is to help us really understand the material!
Keep up the good work, blessing from italy My friend :)
so the basic idea is:
1)steeper slope leads to overfitting @8:16 (what he basically means is that the overfitting line we have has a steeper slope which does not justify his statement on the contrary)
2)adding lambda*(slope)^2 will increase the value of cost function for the overfitted line, which will lead to reduction of slopes or 'thetas' or m's (there are all the same things) @10:03
3)now that the value of cost function for overfitted line is not minimum, another best line is selected by reducing the slopes or 'thetas' or m's which will also reflect in addition of lambda*(slope)^2 ,just this time slope added will be less. @13:45
4)doing this will overcome overfitting as the new best fit line will have less variance(more successful for training data) and less bias than our previous line @14:10 , the bias maybe more because it was 0 for overfitted line ,then it will be a bit more for the new line
5)lambda can be also called as scaling factor or inflation rate to manipulate the regularization. as for the question ,what happens if we have overfitted line with less steeper slope?, then i think we'll find the best fit line with even less steep slope(maybe close to slope~0 but !=0) @16:30
and tadaa!!!!
we have reduced overfititng successfully!!
please correct me if anything's wrong
I've same queries bro.
for 1st point. What if test data has the same slope value as that of train data? in such case there won't be overfitting correct ?
just remember the 4 th point that the main goal of regularization we just wanted to generalize better for the test dateset while having some errors in the test dateset
First of all, no linear regression can be built with just two data points. He is not aware of degree of freedom.
the basic idea is to use lambda (i.e. also known as the regularization parameter) to reduce the product term of Lambda*(slope). Here slope implies various values of m, such as if y = m1x1+m2x2 and so on... we have many values of m(i). So here, we try to adjust the value of lambda such that, the existence of those extra m(i) doesn't matter. And hence we are then able to remove them, i.e. remove the extra features from the model. And we are doing this as one of the major causes of overfitting is due to the addition of extra features. Hence by getting rid of these features, we can curb the problem of overfitting. Hope this helps.
Krish,but equation of a line with best generalized fit,eg y=Mx+c,can be possible with M being high-->AS ACTUAL DATAPOINT CAN BE closer and crowded near Y axis.So,steep slope can't be criteria.
18:35 How the features get removed where |slope| is very very less? Hats off for this fantastic clarity on the topic.
The regression works in such a way that, the coefficients of the inputs which are of less importance will keep on decreasing in every iteration.
In case of ridge regression, the unimportant coefficients exponentially decrease (or asymptotically decrease). In which it will come very close to zero, but will never become zero. (Look the exponential graph for refrence)
This is not the case with lasso regression.
Hope this helps.
Sir, you have mentioned in your lecture this concept is complicated but never I felt it is so. you have explained very excellent.👌👌👌👌👌👌👌👌👌👌👌👌👌👌👌👌👌👌👌👌
Hats offf...grateful for valuable content at 0 cost
i found your free videos better than some other paid tutorials...thanx for ur work
Krish Sir you are saving my masters literally,up to date explanation,and the efforts you are putting to help us understand,Thank You so Much Sir.😇🥰
Concept well explained. I've watch a lot of videos on Ridge regression but most well explained has shown mathematically the effect of lambda on slope.
Thank you so much for this wonderful explanation,
truly appreciate your efforts in helping the data science community.
At 8:15 you say "a steep slope will always lead to overfitting case, why? I will just tell you now..." But I couldn't find where later on you have explained this.
This is where I have found the best explanation of ridge regression after searching a lot of videos and documentations. thank you sir
Thanks is there articles i can refer any blogs you recommend thanks again great content
Hats off... What a way... What a way to explain man... Clear...all doubts
Thanks, Krish,
I want to correct one thing here, the motivation behind the penalty is not to change the slop; it is to reduce the model's complexity.
For example, consider the flowing tow models:
f1: x + y + z + 2*x^2 + 5y^2 + z^2 =10
f2: 2*x^2 + 5y^2 + z^2 =15
f1 is more complicated than f2. Clearly, a complicated model has a higher chance of overfitting.
By increasing lambda (the complexity factor), it is more likely to have a simpler model.
Another example:
f1: x + 2y + 10z + 5h + 30g = 100
f2: 10z + 30g = 120
f2 is simpler than f1. If both models have the same performance on the training data, we would like to use f2 as our model. Because it is a simpler model and a simpler model has less chance for overfitting.
i think you need to emphasize on the high variance toward the test data, and low variance toward the training data, the problem with overfitting is that this low variance on the training data comes at the expense of high variance on the test data. When the model is exposed to new, unseen data (the test data), it struggles to generalize because it has essentially memorized the noise and intricacies of the training data. This results in a significant difference between the model's predictions and the true values on the test data, indicating high variance on the test data.
Thanks, the enthusiasm with which you teach also carries over to us. 🥰
good explanation without touching any complex maths derivations.
Loved your energy sir and your conviction to explain and make it clear to your students. I know it is hard to look at the camera and talk - you nailed it. This video really helped me to understand the overall concept. My two cents, 1) Keep the camera focus on the white board I think it is autofocussing between you and the white board and maybe that is why you get that change in brightness also.
Very good video to understand the intuition behind L1, L2
Best explanation on lasso n ridge regression ever on TH-cam... Thanks krish... You nailed it...
Hi Krish,
Your are making our confidence more on data scince with the clear explanations
Extraordinary talented Sir
You're the best trainer!! Thank you!
I wish I could like his videos multiple times. You are a great teacher, Kind Sir.
Very informative lecture dear. You explained with maximum detail. thanks
This naik is excellent. He is solving every ones problem.
Very nice explanation. I have started watching your videos and your teaching style is very nice .
Very nice you tube channel for understanding data science-Hats Off!!
I have watched more than 8 videos and 2-3 articles but didn't get how lambda value effect the slope ur video explain it best. Thanks
Small correction - For Lasso regression, it is sum of mod of coefficients multiplied by the regularization parameter. You wrote it as mod of sum of coefficients multiplied by the regularization
It is lambda*(|m1| + |m2| + ..) and not lambda*(|m1+ m2 + ...)
True
This is the most clearest and best explanation about this topic on youtube. I can't express how thankful I am for this video for finally understanding the concept
Hello from México. Thank you for your tutorials, they are as if one of my class mates was explaining concepts to me in simple words. A suggestion, please include a short tutorial on ablation of Deep Learning Models.
Best best best bestttttt class...hats off maaan
Regularization was a very abstruse and knotty topic. However, after watching this video; it is a piece of cake
Thank you, Krish
Thank you! I thought this was a great explanation (as someone who has listened to a bunch of different ones trying to nail my understanding of this)
Brilliantly done! Thanks Krish
Muchas gracias por compartir. Se agradece mucho el esfuerzo por aclarar los conceptos que es la base de partida para la resolución. Saludos desde España!
sí, es un tipo genial ... también me encanta ver sus videos!
best tutorials for machine learning with indepth intuition...i think there is no tutorial on utube like this...Thankuu sir..
Your explanations are sooo clear!
Thank you. Now I got a clear idea on both these regression techniques.
Thanks a lot for your Support
just wanted to say that I absolutely loved the video
This man has a gift.
after going through tons of videos, finally found the best one, thnx!!
Very nicely explained, thank God I found your posts on TH-cam while searching the stuff👍
I have a mémoire master 2 about lasso régression i need informations more informations about régression de lasso but in frensh can you help me
Thank you, for the crystal clear explanation. Now I will remember Ridge and Lasso.
well explained krish. thank you for creating . great work
Here is where I understood that damn issue. I’m appreciated too much, thanks my dear friend :)
May be you could use different colour for highlighting the text before or after editing. That would be helpful and easy to grasp
Good video for beginners, a couple of pointers though:
1. The lasso regression would lead to |m1|+|m2|+|m3| +.... and not |m1+m2+m3+m4....|
2. The explanation why coefficients in L1 regularization would go to zero and not for L2 is missing. Probably can expand upon it theoretically.
You made it so simple! Thank you.
Krish An excellent explanation. Thank you so much for this wonderful in-depth intuition.
perfect explanationnnnnnnn. WAs wandering around for whole day. And at the end of the day, found this one.
Brilliantly explained !! thankyou !!
After watching this lecture is not complicated... good teaching sir
What a great video thank you so much for posting. Can anyone tell me what happens if you’re overfitting with a smaller slope? I.e. you need to increase the slope to generalise the model more.
Is this when you need to make the lambda parameter negative?
The kind of explanation is truly inspirational. I am truly overfitted by knowledge after seeing your video.❤
Reduce the rate of learning, this may help you as per Ridge regression :P
Your explanations are superbbb!
Great explanation Krish.I think I a understanding a little bit about L1 andL2 regression.Thanks
you are awesome, better than our professors in explaining such complex topics.
Hi Krish here we are adding the regularization term to our loss function and this regularization will always positive as lambda value is greater then 1 and slope value we will do square or taking mode then how we can say that it is penalizing . Infact here it is adding some positive value in the loss .
Amazing explanation, thank you!
@Krish Naik
These penalties making the slope smooth not zero..these slopes are nothing but coefficients of the features(x1,x2,x3...) and we are trying to reduce the impact of a features which have high value of coefficient with respect to other coefficients(in case of multicollinearity some feature which are highly correlated will tend to confuse the gradient decent while it minimize the error function and which results in high value of coefficients of correlated features and eventually the higher values of coefficients tells the models that these are the most important deciding features for the target column, but it is a wrong prediction ...there are many reasons for the high values of coefficients of some features with respect to the other coefficients )
So by adding the new parameter(lambda×sum of squares of coefficients) to the error function (which will eventually minimized by gradient decent) we are telling the gradient decent to take care of the coefficients which are very high even if we have to loose some training accuracy (we want generalized model not model which shows best training accuracy but not a good predictor for new unseen data ??)
statisticsbyjim.com/regression/multicollinearity-in-regression-analysis/
Just one word, Fantastic.
thanks for explaining Buddy!
Absolutely excellent explanation!
Krish it is a great video to understand Ridge and Lasso. You have shown here the example of test data which is below the best fit line. In this case we need to reduce the slope of the best fit line. It is fine. But if the test data is populated on the upper side (above) of the best fit line, then what we will do. Will we reduce the slope or increase the slope of the best fit line to overcome the overfitting and underfitting?
Same question.. Can you explain me if you know then
Thanks krish! Just a question, why are we penalising high slope functions? Are there any situations where the model would benefit from a higher slope (as compared to the linear regression slope) ?
ah hello there mate, the penalty that's being incurred on the high slopes is because when a steep slope occurs the model is trying to overfit the dataset, it tries to make a line that should pass through every point in the dataset, but when you provide the model with test set, the point may or may not lie close to your best fit passing line, so it will affect the performance, Hence you got to generalize more, you have to observe general patterns not to get 100% results by just overfitting training sets
thank you for this! finally understood the topic
Excellent video!!! Thanks for a very good explanation.
you make it look so easy, kudos to you Krish!!!
thanks sir,is when i am learnning , hopping i will be good in 3-6 months to come , 2 hours a day
Too good and short for ppl clear with basic modeling concepts
You did a great job sir.... It helped me a lot in understanding this concept. In 20min I understood the basic of this concept. Thank you💯💯
Thanks. Very well explained (I needed this). I have a question though. For ridge regression, we're trying to penalize steep slopes. Ok, but why is that ? Why don't we try to penalize flat slopes instead (imagine the flat slope is flat, and the real data points are above the best fit line, so that there is still overfitting)? I mean: don't we want to penalize any type of difference between the actual data points and the best fit line? Why only penalize the difference in one direction?
Hello Sir,
As I understand, we apply Regularization [Ridge and Lasso] to reduce features or reduce contribution of features.
As you also shared we took sum of squared residuals and then added the regularization constant to it.
Thus, we get cost value which is equal to sum of squared residuals and regularization penalty and we get a numerical result as the output of the computation.
At this point, How do we perform the feature selection or control contribution of each input variable in feature space?
Thank you
Hi Krish, Just curious, are there any general guidelines when to use which technique? Should we use Lasso regression in favour of Feature selection objective? or Should we experiment the best of the both?
Thanks for a crystal clear explanation of this topic, you made it so easy to visualize this concept of regularisation.
Hi Krish,
Great video.. Thanks a lot. However, I have one small concern. through out the duration of 20 mins I was interrupted with close to 10 advert breaks and that is disturbing the flow. Please see is you can control the no of adverts. It is annoying. But the content and your way of explanation is what has kept me hooked.
Krish,Thanks for the clear explanation. I have a doubt : if the test data points are below the best fit line we have selected earlier, this is fine. What if the test data points are above the best fit lines ? in that case , applying ridge regression , would still try to reduce the slope and we may not end up getting the best fit line. can i get a help in this scenario. Thanks in Advance!
Same doubt I also have. I hope he explains this
Hi Krish, here you explained overfitting when the slope is steep, but what if the slope is less steep for regression line.
Lasso regression does feature selection which is an extra thing done when compared to ridge regression
Excellent explanation sir...thanks
Superbly explained
Amazing explanations.
11:34 there is a symbolic error. It is > instead of
Yes, that's right. Even I was wondering the same