StatQuest: Random Forests Part 1 - Building, Using and Evaluating
ฝัง
- เผยแพร่เมื่อ 25 มิ.ย. 2024
- Random Forests make a simple, yet effective, machine learning method. They are made out of decision trees, but don't have the same problems with accuracy. In this video, I walk you through the steps to build, use and evaluate a random forest.
NOTE: Random Forests are made from Decision Trees, so if you don't know about those, here's the Quest: • Decision and Classific...
ALSO NOTE: This StatQuest is based on Leo Breiman's (one of the creators of Random Forests) website: www.stat.berkeley.edu/~breima...
For a complete index of all the StatQuest videos, check out:
statquest.org/video-index/
If you'd like to support StatQuest, please consider...
Buy The StatQuest Illustrated Guide to Machine Learning!!!
PDF - statquest.gumroad.com/l/wvtmc
Paperback - www.amazon.com/dp/B09ZCKR4H6
Kindle eBook - www.amazon.com/dp/B09ZG79HXC
Patreon: / statquest
...or...
TH-cam Membership: / @statquest
...a cool StatQuest t-shirt or sweatshirt:
shop.spreadshirt.com/statques...
...buying one or two of my songs (or go large and get a whole album!)
joshuastarmer.bandcamp.com/
...or just donating to StatQuest!
www.paypal.me/statquest
Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
/ joshuastarmer
0:00 Awesome song and introduction
0:31 Motivation for using Random Forests
1:17 Step 1, create a bootstrapped dataset
2:23 Step 2, create a decision tree a random subset of variables at each step
4:00 Step 3, repeat steps 1 and 2 a bunch of times
4:40 Classifying a new sample with a Random Forest
5:41 Definition of Bagging
6:03 Evaluating a Random Forest
8:34 Optimizing the Random Forest
Corrections:
3:18 I should have said the same feature (or variable) can be selected multiple times in a tree. Every time we select a subset of features to choose from, we choose from the full list of features, even if we have already used some of those features. Thus, a single feature can appear multiple times in a tree.
9:28 I say "square" when I meant to say "square root".
#statquest #randomforest #ML
Corrections:
3:18 I should have said the same feature (or variable) can be selected multiple times in a tree. Every time we select a subset of features to choose from, we choose from the full list of features, even if we have already used some of those features. Thus, a single feature can appear multiple times in a tree.
9:28 I say "square" when I meant to say "square root".
Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/
StatQuest with Josh Starmer - Can you explain more on this? I mean how does it work when same variable is selected more than once... Let’s say for example., we have our root node as Good blood circulation - if it is true then it goes to left node and if is false it goes to right node
Then let’s say our internal node on right side is chest pain and then next internal node below it is weight
Then as you said we have again selected good blood circulation randomly ... how does it work because right side nodes have bad blood circulation already
Selecting variables multiple times makes sense when it is continuous variable because it can used again with different threshold values but how come it works with categorical variable
For the same bootstrap data set we can make different trees with randomly picked variables?
No, you only make one tree per boostrapped dataset. What I was trying to say was that all columns/variables/features in that bootstrapped dataset are considered at each node.
It’s true. There’s no use in visiting a boolean variable more than once. However, as a general rule, all variables, at least in theory, can be visited more than once in a single tree. The actual implementation may optimize this by omitting boolean variables if they have already been visited.
@@statquest so here randomly 2 is picked at each node without taking into account gini ? Is that right
Please continue what you're doing. TH-cam users are blessed to have you here. Your contents are not long, right to the point, clear and your way of teaching is amazing. If there will be a heaven you will be right in the middle of it.
Thank you very much! :)
Beats the Wikipedia articles on these subjects by miles! Perhaps if I was more versed in math, I could just read it like the daily newspaper, but as it is, having this clear visualization and examples is a waay superior learning method!
This video is so well made and so well explained that I comprehend the subject matter even while sitting here drunk AF at my keyboard... *hic*
Dang! :)
Well said ! Understanding a subject is more relying on the teacher's skills to explain clearly than to the student's efforts to trying to understand.
Your "BAM!!" always bring me back to the reality when, eventually, my brain takes a nap during the class haha. It makes me go back and rewatch the issue. I really appreciate your art, sir.
Awesome! :)
Very true. He is so good at teaching!
I have no background in Machine Learning and was attempting to understand random forests. Watched a bunch of videos but this was the one that actually made it clear to me. Thanks a lot!
Glad it was helpful!
Your videos are unbelievably simple to follow and intuitive. You know that someone is a master of their craft when they can explain it to someone else in a concise and easy-to-understand way. Well done, and thank you for all of your videos!
Wow, thank you!
I'm studying Data Science at MIT, you really can't imagine Josh how much StatQuest is helping me, and a couple more channels, before I start any topic I like to tackle it first or just take a general idea, and you can't imagine how much your videos helped! Short, concise, and to the point! I even started to like mice man for god sack! Thank you Josh 🙂
Thank you! I'm glad my videos are helpful.
Hello, can you share what other channels you watch? I'm a student too, and I wish to gain as much knowledge before looking for a job. Have a wonderful day!
@@jarosawszyc8287 No one is perfect in everything! Look up by the topic, not by the channel, you'll ALWAYS find someone who explains what you looking for in a very simple and very straight forward way.
@@atharvambokar573 Massachusetts Institute of Technology 🙂
@@atharvambokar573 I wanted to change my career to a Data Analyst, and got accepted for post graduate at MIT. I literally started from scratch.
Plus what is wrong in creditting people for their efforts.
Isn't it 'square root' of the number of variables at the end, instead of 'square'?
You are correct. I make a slight mistake.
Absolutely, thats what I was breaking my head to understand and thankfully got it clarified from this comment !!!
Yes You are absolutely Correct..
I was just about to write up this query. Absolutely correct
@@statquest I am pretty sure StatQuest will be canonized as a "Video Bible of Machine Learning" some time soon. Sooooo, any slight mistake will have a probability (or likelihood?) of huge changes in the course of humankind history. :))
This is absolutely awesome and so clearly explained. I wish all textbooks were like this. Keep it up !!
Indeed! Wikipedia is usually a way to get the information more simply than textbooks, but they bombard you with math formulae right from the beginning on these subjects! ^^
So this is so much more intuitive!
Words cannot express how much I love StatQuest!!!!!! Thanks Josh for the amazing videos! I'm definitely gonna recommend StatQuest to my classmates!
Thank you very much! :)
Your service to the data science community is very very much valuable...!!! Your videos on basics leaves a long lasting memory..!! Thank you..!!
Thank you!
"Sir, after your examination we found that you've got heart disease. BAAM"
"All 100 trees of the random forest predicted that you have a heart disease. HOORAY! "
It is actually "BAAM?" :D
@@phan9995Double baam!!
One of the best tutorial channels. Keep up the good work and helping us all who are in need.
Thank you so much!!! :)
Your videos deserve a million kudos. They should be a legit applied statistics class for biologists like myself. Please continue your amazing work! I invite everybody to support Josh with as little as $1 a month.
Thank you very much for supporting StatQuest! It means a lot to me.
sir, you are always so helpful, when I have problems, the first thing came to my mind is to search the relative topic in your videos, looking forward more videos from you, thank you very much!
You are the only one who clearly explains how the *size of bootstrap* can be the same as the *size of the original train data* . Thanks!
You're welcome!!! Hooray! I'm glad it was clearly explained. :)
the presentation is so simple and adorable, and what's more the topic is truly clearly explained! Thanks!! I love this animation and color choice
Thank you very much! :)
I was required to give a ppt on random forest in my class and after going with such a lucid video I am a felling great. Thanks STATSQUEST
Hooray! :)
dont forget the BAAMMM!!!
I love your content! It's super helpful to have all the information presented verbally and written at the same time. I usually use captions, but those often cover up the content which is annoying and the way you do it is so much better. Plus your intros make me giggle. Thank you so much!
Hooray! I'm glad you like my style! :)
i searched for so many tutorials but no one explained it like you. Thank you!!
Thanks! :)
Everytime he starts "Bo pi do pi do pi doo" i inadvertently start laughing
Heavily relying on these videos to understand the techniques relevant for my Master Thesis in experimental particle physics. Thank you so much, you are the best!
Thank you! I'm glad the videos are helpful. :)
I'm also using this for my Master Thesis :D
You're the best! Out of many tutorials on Internet...this best explains Random forest and its working! Thank you so much :)
Wow! Thank you Josh, only after watching your videos about decision tree and random forest, I really get the concepts of them!
Hooray!!! I glad you like the videos and that they were helpful! :)
Great tutorial, as always!
The "Oh no! Terminology/Jargon alert !" always gets me
:)
This is the best video I've ever seen to explain Random Forrest! Thanks so much for making this!!Please keep making videos! Love the humor also :)
You're welcome!!! I'm glad you like the video (and my silly jokes)! :)
These graphical examples are awesome. I love your videos!!
I must say,this is the best video! You made it so easy to understand. And the way you explain it is perfect!
Thank you!!! :)
Can I say BAAM in an interview after explaining what is Random Forests?
Yes but then interviewer will also say DOUBLE BAAM after kicking you out
@@imrankhanissm XD
@@imrankhanissm lmao
@@imrankhanissm 😂🤣😁
haha lol
This is one lovely presentation on Random Forests! Thanks a ton for making it easy for us to understand.
Thank you very much! :)
Thank you so much Statquest. in my "quest" to build random forest from scratch in C++, I found this resources soooo usefull. Including the video on the decision tree classifier
Glad I could help!
easily the best channel for data science/machine learning. respect
Thank you very much! :)
@@statquest agreed 100%
I really like the pace of these. It helps me get around the terms and have chance for my brain to keep up
Awesome! :)
These videos are so amazing, I absolutely love them! Thank you so much!
Hooray!!! I'm so happy to hear that you like the videos! :)
It just hit me that its called random forest because its a forest of random trees. This channel is pure gold for bioinformatics students
Thanks!
very good approach, genious simplicity and to the point, helps anyone learn(even me). I even stopped skipping the "music" to pay my respects to you Josh!
Thanks so much! I love that you put "music" in quotes. That cracked me up. As long as I get to have my fun, I'm happy to make these videos. ;)
I've seen multiple videos and articles explaining Random Forests and I must say this is the best so far. Awesome work!
Thank you very much!!! :)
Just discovered your channel. I really like your work. Love the funny moments you add in it. Thank you for clarifying those concepts!
Awesome, thank you!
You're so cool Josh, by only first listening to your song, I knew you'd nail this topic just as usual. You're much appreciated Sir Starmer
Thanks!
Man you're a legend. LOVE Your content especially the music.
Glad you enjoy it!
Great video. Very simple and easy to understand. Nice job!
Thank you! I'm glad to hear you like the video. :)
I found this on top of the list while searching and now I don't need any other video to understand it. Thanks
Glad it helped!
I'll keep saying stat-quest is the best stat tutorials channel ever
bam!
I'm in grad school and I'm supposed to "quote" litterature, yet I just watch StatQuest's videos because they are easy and fast.
BAM!
In grad school and am planning on applying the RF method to my air quality data. Your videos have been a life-saver!
Awesome and good luck! :)
@@statquest Hi, I am doing my PhD, I am inspired to do a project just because I learned so much from your video. I learn more from one single video than an entire year of my coursework.
OMG you saved my day ! I am currently preparing a phd exam, and I was lost about random forest, now I can easily breath thanks to your amazing video ! Thanks a lot
Good luck!
I love StatQuest. You really make statistics enjoyable. BAM!!!!!!!!
Thank you for your time and effort in making these videos. You have helped out a lot of students.
Please keep up the good work. Keep them coming.
Thank you very much! :)
@@statquest hello there.. could you please clarify the doubt that was asked in your pinned comment.. about choosing the same features again at different level in the decision tree..
This video is totally "Out-of-Boot" among all other Random Forest videos on TH-cam 😀
BAM! :)
You are a boon to humanity...Hope you make a lot of money:)
That would be awesome! Maybe one day that will happen. :)
Thank you, Josh, for making things much easier. This 10 minutes tutorial is almost tantamount to reading one whole book about random forest decision tree. I heard the piece at 9:39 as '.... square root of the number of variables...."
Thanks! Yes, I made a typo at the end of the video and it is mentioned in a pinned comment.
Great job StatQuest! I am in love with this channel.
Hooray!!! I’m so happy to hear that you like StatQuest!
my project is about using random forests on accelerometer data from goats, but Josh is the real goat
bam! That sounds like a pretty cool project!
Oh Really, This is somewhat what i am working on. I am studying different speedup processes and strategies and I am specifically working on stroke prediction..
nice. I am working on an activity tag to classify behavior automatically without observing the animal directly@@aristide_F
After learning so much from your videos and then listening to your beautiful sentimental songs makes me cry! ;)
Thank you very much!!!! :)
Thank you very much for this video! It was a great use of visuals to explain the progression of various aspects of this topic (e.g. eventually using the out-of-bag samples to calculate accuracy)!
Glad you enjoyed it!
This guy is a living legend, giving us quality content for free
Thank you!
In evaluating the random forest, you mention: use trees to classify samples that are not used to build the trees. As a result, the number of trees to test an out-of-bag sample is different from one sample to another, aren't? E.g. a sample can be tested by 4 trees (there are 4 trees that do not use the sample to build), while another sample can be tested by 10 trees.
9:36 Typically we start by using the square [root?]
Was wondering the same thing. If we have 5 features we can't bootstrap with 25. It makes more sense if we have 25 variables, to start with 5 for bootstrapping.
Thank you for another great video, i love how you are to the point and explain everything in such an easy way
Glad you like them!
This is the best video that very well explains random forests.Very Helpful!
Thank you! :)
3:37 looks like a warning on a cigarette pack
Double out-of-bam
I like it! :)
This makes it so easy. I understand it well enough to explain it to others. Thanks.
Thank you!
Your channel has saved my research career
Hooray! :)
Finally found an ML tutorial without accent.
He has an American accent
@@digit432 lol
There are 7 idiot heaters who down-voted this amazing video... They probably live in pain!!
:)
Thank you so much for a very easy-and-nice-to-walk-through video! I really enjoyed the explanations and well-prepared slides! Also you made it very nicely paced. Thank you :))
Thank you!
Respect ,The way you have present Random forest
Thank you! :)
This patient has heart disease. BAM!! Congrats
:)
I just love your videos. Im attending a bootcamp about Datasceince and all the concepts that the instructor is not explaining as i would understand them, i come to your channel to learn them better. Thanks man ❤❤
Glad to help!
Really good tutorial. It's always the best to explain using real examples. Good job!!!
This was such an awesome and a cool video !! Found this helpful :) Kudos for making this video, and hope you continue with your Quest !!
Thank you! Will do!
Oh good..! I understood random forest concept after watching this video. Thanks so much
Hooray!! I'm glad the video was helpful. :)
For everything, there's a statquest episode.
bam!
Thank you for sharing this easy way to understand RF!
Thanks! :)
Bam Bam Double Bam!!! Awesome Tutorial Videos and very helpful!!
I am saved. This is what my professor struggles to explain for more than an hour. It is so simple.
bam!
best explanation so far on TH-cam. thank you!!
Thank you! :)
I like this, quite clear explanation and hit the points we need to know. Thanks.
Thank you! :)
I enjoy the tutorial and your album a lot. Math + music, what a combination.
thanks for the content this helps me more than my couple hundred dollar textbooks.
Thank you very much!!!! :)
I love your opening song, it makes the content less stressful!
bam!
You are the cutest stat teacher i have came across in my life . Lots of LOVE and Respect.
Wow, thank you!
I can't wait to work through all your videos!
Hope you enjoy!
I can't be any happier lol! Thanks a ton for the vid, the explanation is awesome!
Hooray! :)
Hey Josh, your videos are very interesting and easy to understand. Keep up the good work, man. The gini impurity could be the 'Gain Ratio Impurity'. The Information Gain Ratio is an important parameter in decision tree learning, so seems plausible to me.
Your channel is like a course in ML and Stats but better than most !!!
Thank you! :)
probably these best video fora beginner on the internet. Thanks!
U have got great skills in explaining concepts. Thank you!
thank you very much for this well-crafted and excelllent video!! finally understand bagging
bam!
My second time commenting on your musics, they are also the best part of the video, watching a study video and a comedy at the same time, I'm so motivated everytime watching your videos very much cuz of the musics lol. You should consider a music PHD as well :)
BAM! :)
Excellent tutorial! His tutorials are underrated.
Thank you!! I'm glad you like this one! :)
Just this video was enough for me to subscribe, Thank you Josh!!!
Awesome! Thank you!
I love the way you explain. so clear and helpful ! Thanks
You're welcome!!! I'm glad to hear you like this video. :)
I trust the information you provide so much, to an extent that I press the like button even before I watch the video
Awesome!!!
Your songs are also amazing. I loved the last album you shared on Bandcamp. It motivates me to continue my passion for music as well as my studies in Computer Science.
Happy to hear that!
Oh my god you made it easy to understand. Please keep this process.
Thanks!
The best explanation for Random forests!
Thank you! :)
Excellent! You definitely have to continue making educational videos. Congratulations for your work.
the way statquest Channel makes learning Maching Learning easy is just B.A.M
Bam! :)
I must say I went through a few mathematical books in order to understand it, but this explanation is just BAMMMM!! :)
Hooray! :)