Regularization Part 2: Lasso (L1) Regression
ฝัง
- เผยแพร่เมื่อ 27 ก.ค. 2024
- Lasso Regression is super similar to Ridge Regression, but there is one big, huge difference between the two. In this video, I start by talking about all of the similarities, and then show you the cool thing that Lasso Regression can do that Ridge Regression can't.
NOTE: This StatQuest follows up on the the StatQuest on Ridge Regression:
• Regularization Part 1:...
For a complete index of all the StatQuest videos, check out:
statquest.org/video-index/
If you'd like to support StatQuest, please consider...
Buying The StatQuest Illustrated Guide to Machine Learning!!!
PDF - statquest.gumroad.com/l/wvtmc
Paperback - www.amazon.com/dp/B09ZCKR4H6
Kindle eBook - www.amazon.com/dp/B09ZG79HXC
Patreon: / statquest
...or...
TH-cam Membership: / @statquest
...a cool StatQuest t-shirt or sweatshirt:
shop.spreadshirt.com/statques...
...buying one or two of my songs (or go large and get a whole album!)
joshuastarmer.bandcamp.com/
...or just donating to StatQuest!
www.paypal.me/statquest
Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
/ joshuastarmer
#statquest #regularization
![[TH] 2024 PMWC x EWC Main Tournament Day 2 | PUBG MOBILE WORLD CUP x ESPORTS WORLD CUP](http://i.ytimg.com/vi/TgEFw285vXA/mqdefault.jpg)
If you want to see why Lasso can set parameters to 0 and Ridge can not, check out: th-cam.com/video/Xm2C_gTAl8c/w-d-xo.html
Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/
Love how you keep these videos introductory and don't go into the heavy math right away to confuse;
Love the series!
Thank you!
One of the best explanation of Ridge and Lasso regression I have seen till date... Keep up the good work....Kudos !!!
Thanks! :)
I am eternally grateful to you and those videos!! Really saves me time in preparing for exams!!
Happy to help!
That "Bam???" cracks me up. Thanks for your work!
:)
Every time I think your video subject is going to be daunting, I find you explanation dispel that thought pretty quickly. Nice job!
Hi man, really LOVE your videos. Right now I'm studying Data Science and Machine Learning and more often than not your videos are the light at the end of the tunnel, sot thanks!
My teacher is 75 years old, explained us Lasso during one hour , without explaining it. But this is a war I can win :), thanks to your efforts.
I love it!!! Glad my video is helpful! :) p.s. I got the joke too. Nice! ;)
Why is this scenario many times the reality? Also, I check StatQuest's vids very often to really understand the things. Thanks @StatQuest
Hi, I can't thank you enough for explaining the core concepts in such short amount of time. Your videos help a lot! My appreciations are beyond words.
Thank you!
Good video, but didn't really explain how LASSO gets to make a variable zero. What's the difference between squaring a term and using the absolute value for that?
Intuitively, the closer slope gets to zero, the square of that number becomes insignificant compared to the increase in the sum of the squared error. In other words, the smaller you slope, the square gets asymptotically close to 0 because it can't outweigh the increase in the sum of squared error. In contrast, the absolute value adds a fixed amount to the regularization penalty and can overcome the increase in the sum of squared error.
@@theethatanuraksoontorn2517 Maybe this discussion on stack-exchange will clear things up for you: stats.stackexchange.com/questions/151954/sparsity-in-lasso-and-advantage-over-ridge-statistical-learning
@@statquest Thanks for reading the comments and responding!
@@programminginterviewprep1808 I'm glad to help. :)
@@statquest I didn't reply before, but the answer really helped me a lot, with basic machine learning and now artificial neural networks, thank you very much for the videos and the replies :D
Don't think your Monty Python reference went unnoticed
(Terrific and very helpful video, as always)
Thanks so much!!! :)
Oh it absolutely did. And it was much loved!
Thank you, Josh, for this exciting and educational video! It was really insightful to learn both the superficial difference (i.e. how the coefficients of the predictors are penalized) and the significant difference in terms of application (i.e. some useless predictors may be excluded through Lasso regression)!
Double BAM! :)
The difference between BAM??? and BAM!!! is hilarious!!
:)
@@statquestCan you please explain how the irrelevant parameters "shrink"? How does Lasso go to zero when Ridge doesn't?
@@SaiSrikarDabbukottu I show how it all works in this video: th-cam.com/video/Xm2C_gTAl8c/w-d-xo.html
Some video ideas to better explain the following topics:
1. Monte Carlo experiments
2. Bootstrapping
3. Kernel functions in ML
4. Why ML is black box
OK. I'll add those to the to-do list. The more people that ask for them, the more I'll priority they will get.
@@statquest That is great! keep up the great work!
@@statquest yes we need it please do plsssssssssssssssssssssssssssssssss
plsssssssssssssssssssssssssssssssssssssssssssssss
Bootstrapping is explained well in Random Forest video.
Do it for us... thanks good stuff
Josh - as always your videos are brilliant in their simplicity! Please keep up your good work!
Thanks, will do!
I am so happy to easily understand these methods after only a few minutes (after spending so many hours studying without really understanding what it was about). Thank you so much, your videos are increadibly helpful! 💯☺
Great to hear!
This channel is pure gold. This would have saved me hours of internet search... Keep up the good work!
Thank you! :)
Your intro songs reminds me of Pheobe from the TV show "Friends", and the songs are amazing for starting the videos on a good note, cheers!
You should really check out the intro song for this StatQuest: th-cam.com/video/D0efHEJsfHo/w-d-xo.html
Bam! I appreciate the pace of the videos. Thanks for doing this.
Thanks! :)
This is brilliant. Thanks for making it publicly available
You're welcome! :)
Came here because I didn't understand it at all when my professor lectured about LASSO in my university course... I have a much better understanding now thank you so much!
Awesome!! I'm glad the video was helpful. :)
I am eternally grateful to you. You've helped immensely with my last assessment in uni to finish my bachelors
Congratulations!!! I'm glad my videos were helpful! BAM! :)
Explained in a very simple yet very effective way! Thank you for your contribution Sir
Hooray! I'm glad you like my video. :)
(Possible) Fact: 78% of people who understand statistics and machine learning attribute their comprehension to StatQuest.
bam! :)
Yeahhhh!!! I was the first to express Gratitude to Josh for this awesome video!! Thanks Josh for posting this and man! your channel is growing.. last time, 4 months ago it was 12k. You have the better stats ;)
Hooray! Yes, the channel is growing and that is very exciting. It makes me want to work harder to make more videos as quickly as I can. :)
@@statquest please keep on going... You are our saviour
Thx very much. Clear explanation for these similar models. Great video I will conserve forever
The beginning songs are always amazing hahaha!!
Awesome! :)
The other day, I had homework to write about Lasso and I struggled.. wish I had seen this video a few days earlier.. Thank you as always!
So easy to understand. And I like the double BAM!!!
Thanks!
NOBODY IS GOING TO TALK ABOUT THE EUROPEAN / AFRICAN SWALLOW REFERENCE ????are you all dummies or something ? It made my day. Plus, video on top, congratulation. BAMM !
bam!
I really appreciated the inclusion of swallow airspeed as a variable above and beyond the clear-cut explanation. Thanks Josh. ;-)
:)
Me too!
Your videos make it so easy to understand. Thank you!
Thank you! :)
Airspeed of swallow lol. These videos are really helping me a ton, very simply explained and entertaining as well!
Glad you like them!
Incredible great explanations of regularization methods, thanks a lot.
Thanks! :)
Thanks for posting, my new favourite youtube channel absolutely !!!!
Wow, thanks!
Thank you so much for the video !
I have watched several your videos and I prefer to watch your video first then see the real math formula. When I did that, the formula became so easier and understandable!
For instance, I don't even know what does 'norm' is, but after watching your video then it would be very easy to understand!
Awesome! I'm glad the videos are helpful. :)
Thank you so much for these videos you are a literal godsend. You should do a video on weighted least squares!!
Dude you are an absolute lifesaver! keep it up!!!
Hooray! I'm glad I could help. :)
I came for the quality content, fell in love with the songs and bam.
BAM! :)
this is awesome thank you so much for this u explained it so well . I will recommend this video to every one I know who is interested . I also watched your lasso video and it was just as good thank you
Thank you very much! :)
Great video, clear explanation, loved the Swallows reference! Keep it up! :)
Awesome, thank you!
Just love the way you say 'BAM?'.....a feeling of hope mixed with optimism, anxiety and doubt 😅
:)
Thanks a lot! Amazing explanation! Please, continue the great work and add more on statistics, probability in general and machine learning in particular. Sinse Data Science suppose to have a great future, I am certain that your channel also will prosper a great deal!
Thank you! :)
Hooray!!!! excellent video as always
Thank you!
Hooray, indeed!!!! Glad you like this one! :)
Me and my friend are studying. When the first BAM came, we fell for laught for about 5min. Then the DOUBLE BAM would cause a catrastofic laughter if we didn't stop it . I want you to be my professor please!
BAM! :)
Statquest is like Marshall Eriksen from HIMYM teaching us stats. BAM? Awesome work Josh.
Thanks!
You have a gift for teaching! Excellent videos!
Thanks! :)
Great people know subtle differences which is not visible to common eyes
love you sir
Thanks!
Amazing video, explanation is fantastic. I like the song along with the concept :)
Bam! :)
Thanks! I finally understand how they shrink parameters!
a man of his word...very clearly explained!
Thank you! :)
Excelent video Josh! Amazing way to explain Statistics Thank you so much! Regards from Querétaro, México
Muchas gracias! :)
Brilliant explanation
didnt need to check out any other video
Thank you!
Ah! A triple THANKSSSS!!!!. I finally got what they are really doing.
Hooray! :)
A StatQuest a day, keeps Stat fear away!
I love it! :)
Thank you so much for making these videos! Had to hold a presentation about LASSO in university.
I hope the presentation went well! :)
@@statquest Thx. It did :)
Keep it up man. Awesome content.
Thanks! :)
Hi Josh, Thanks for clear explanation on regularization techniques. very exciting. God bless for efforts.
Glad you enjoyed it!
Both the Ridge and Lasso videos made me want to cry. (Know you aren't alone if anyone else feels the same.) Also noteworthy: Ridge Regression avoids problems introduced by having more observations than predictor variables, or when multicollinearity is an issue. This example avoids either condition. Triple Bam. (Obviously, I am taking the definition too literally. It's a relative statement, re: the vars to observations ratio. )...Nevertheless, there's no end to my confusion. I was approaching "understanding", using the ISLR book....but you can actually get two different perspectives on the same topic, and then be worse off due to variance in how the concepts are presented. That said, you're still awesome, StatsQuest, and you are invited to play guitar at my funeral when I end things from trying to learn ML.
(gonna check the StatsExchange link down below that you provided. Thank you, sir!!!)
Seriously the best videos ever!!
Thanks!
Hahaha.. That moment you said BAM??? I laughed out loud 🤣🤣🤣
:)
The best explanation ever.
love the work, i remember reading books about linear regresion, when they spent like 5 pages for these 2 topics but i still have no clue what they really do =))
Glad it was helpful!
Love the fact that you reply to every single comment here in YT haha
Fantastic videos - very well explained!
Thank you! :)
Hi Josh! I am a big fan of your videos and it is clearly the best way to learn machine learning. I would like to ask you if you will be uploading videos relating to deep learning and NLP as well. If so, that will be awesome. BAM!!!
Right now I'm finishing up Support Vector Machines (one more video), then I'll do a series of videos on XGBoost and after that I'll do neural networks and deep learning.
StatQuest with Josh Starmer Thanks Josh for the updates. I’ll send you request at Linkedin.
Always amazing videos.
Thank you!
Great video! The topic is really well explained
Thank you!
Million BAM for this channel 🎉🎉🎉
Thank you!
That Monty Python reference though... good video btw :)
Ha! I'm glad you like the video. ;)
Great video! I finally understand!
Thank you once again Josh!
bam!
awesome your explanation just simplifies everything
request to make videos on rest of the algorithms as well
thank you
I'm working on them :)
Thanks for the Video. They make difficult concepts seem really easy..
Thank you! :)
@@statquest Can u make a similar video for LSTM?
Great....please continue to learn other models...thank you so much.
Thanks!
I love your style of explaining! You leave enough time for anyone to take in all information while talking. Sometimes it feels like you are trying to teach little kids, but it actually just works. I often watch other teaching videos and can't remember most of it afterwards, but I can remember almost everything that you are saying after the first time watching. Amazing job!
I have one question though. You were saying that the regression model in the beginning had low bias and high variance. Does it not have high bias? As far as I know bias represents the expected generalization (or test) error, if we were to fit a very large training set. If we fit that simple model to a lot of data, the generalization error would be rather high, because it could not capture the true patterns in the data.
I'm glad you like the videos! In ML, there are specific meanings for bias and variance that are a little bit different from what you are using and I explain in this StatQuest: th-cam.com/video/EuBBz3bI-aA/w-d-xo.html
Amazing explanation. Loved the Monty Python reference :D
:)
so incredible, so well explained
Thanks!
Thankyou Sir ! Great Help.
Wow! so easy to understand this! Thanks very much!
Thanks!
Hi. Your videos are so helpful. I really appreciate you spend time doing them.
I have one question related to this video: Is the result of Lasso Regression sensitive to the unit of variables?
For example in the model: size of mice = B0 + B1*weight + B2*High Fat Diet + B3*Sign + B4*AirSpeed + epsilon
Suppose the original unit of weight in the data is gram. If we divide the weight by 1,000 to get unit in kilogram, is the Lasso Regression different?
As I understand, the least square estimated B1-kilogram should be 1,000 times higher than the B1-gram. Therefore, B1-kilogram is more likely to be vanished in Lasso, isn't?
I enjoy the content and your jam so much! '~Stat Quest~~'
Thanks!
I prefer the intro where is firmly claimed that StatQuest is bad to the bone. And yes I think this is fundamental.
That’s one of my favorite intros too! :)
But I think my all time favorite is the one for LDA.
Yes I agree! Together these two could be the StatQuest manifesto summarising what people think about stats!
So true!
Harvard should hire you. Your videos never fail me!
Thank you for such great content!
Thank you very much!!!
Thank you for uploading this video. Can you upload a video explaining the difference between Lasso and Group Lasso? Thanks again.
One more use case of Ridge/Lasso regression is 1) When data points are less 2) High Multicollinearity between variables
My favourite youtuber!
Thank you! :)
Best youtube channel
Thank you! :)
Amazing! Thank you so much for this!
Thanks!
love your videos.... extremely helpful and cristal clear explained.... but your songs..... let's say you have a very promising career as a statistician... no question
;)
wonderfully explained
Thank you! :)
Thank you for clarifying that the Swallow can be African or European
bam! :)
You are the best! I understand it now!
Thanks!
Magnificent video
Thanks!
Love this song bro!
Thanks!
This was gold!
Thank you!
me: wathcing these videos in full panic
video: plays calming music
me: :)
bam! Good luck! :)
Thanks a lot for the explanation !!!
You are welcome!
This is dope fam!
:)
why can't ridge reduce weight/parameter to 0 like lasso?
Hi Josh, could you please make video(s) on Correspondence Analysis, Chi-Square distance and Multiple contingency table?
Learning from your videos are much efficient than several books combined :) !
I'm working on those, but the bad news is that they will not be ready for a long time.... :(