NOTE: You can support StatQuest by purchasing the Jupyter Notebook and Python code seen in this video here: statquest.gumroad.com/l/uroxo Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/
Dear Josh ... I have a request for new videos or livechats ... could you explain us these tests maybe ? ... Tukey, Bonferroni and Scheffé , it's hard for me to understand , you explain everything so well ... could be very helpful for a lot of people out there ... have a nice day , greetings from Europe
Please keep doing these long-form Python tutorials on the various ideas we've covered in earlier 'Quests. They're great for those of us working in Python, and they give me another way to support the channel. It has been a more-than-pleasant surprise that as I've grown from learning the basics of stats to machine learning and eventually deep learning, StatQuest has grown along with me into those very same fields. Thanks Josh.
You are amazing. Can't imagine how much work you put into those step-by-step tutorials. Just bought the Jupyter Notebook code and it's beyond worth it! Thank you :)
Hey Josh, I just purchased all of your 3 Jupyter Notebook! I transferred from Econ major to Data Science, it was a nightmare before I find your channel. Your channel shed the light upon my academic career! Look forward to more of the 'Python from Start to Finish' series, and I will definitely support it!
Outstanding! Complete with the thinking path - how to analyze variables in a logical way - and with common errors. Just purchased the Notebook. Thank you for all of your work on this channel.
I am purchasing the Jupiter notebook to contribute to your work! Thanks a lot for this video! You are awesome! Will be very very happy to have more ML tutorials and thank you Josh!
Wow. Finally I see a face for the name. Your previous videos have had immensely helpul. I assumed you are a very senior person. I am not measuring your age. I mean, your way of explaining seemed like a professor with half a century of experience. But in reality, you are quite young. Thank you for all your simple-yet-detailed videos. No words to quantify how much I appreciate them. 🙏
Josh, this video is epic and really helped me understand the actual process of tuning hyperparameters, something that had been a bit of a black box until I saw this video. Your channel is awesome too - great jingles as well :D
@@statquest one small request..can you provide some valuable information through a video like which model to chose for different datasets..how do we decide what model we should chose...thanks in advance
Hey Mate, amazing tutorial. Very complex problem explained in really simple and effective way. I am using XGBOOST for one of the classification model and after watching your video it made me realise I can further improve my model. So thank you again and keep making those videos. Kudos to you and long live data science 🙏🙏
Absolutely loved this video Josh. It breaks down everything into understandable chunks. Thank you and God bless. BAM! The only thing I missed (and its very minor) was taking in a new data row and making an actual prediction by using the model.
It was extremely helpful. Please continue making these videos. I suggest making a video to explain the clustering with unlabeled data, and predicting the future trend in time-series data.
This is a great piece of work, thanks for sharing it! Maybe the only additional piece I'd add which I've found useful on the documentation of XGBoost is that one can take advantage of parallel computing (more cores or using a graphic card your machine or you could have on the cloud) by simply passing the parameter (n_jobs = -1) while doing both, the RandomizedSearchCV stage and the setting the XGB regressor type (XGBRegressor for example).
You're a very kind human being Josh!! Thank you so much for making these videos. Your content is gold!!! I am new to data science and this is exactly what I needed!! :) Much love from India!
@@statquest Hey Josh! I am learning about Bayesian Optimizer and I don't seem to get it even after watching tons of tutorials, can you suggest where I should learn it from please? I couldn't find a video on your channel on this.
Thank you for this great tutorial Josh! Your videos have immensely helped in understanding some of the complex topics. One thing I noticed while watching this tutorial is the handling of categorical features. I think the explanation you gave for "Why not to use LabelEncoding?" is applicable for models like Linear Regression, SVM, NN but not for Trees because they only focus on the order of the feature values. For example, in a set of [1,2,3,4], threshold < 1.5 would be equivalent to threshold == 1. Please let me know if my thought process in wrong.
To be honest, I'm not really sure I understand your question or your example. If we have a categorical feature with 4 colors, red, blue, green, and black, and we give them numbers, like 1, 2, 3 and 4, then a threshold < 2.5, would not make much sense and, based on how trees are implemented, there would be no options for threshold == 2 or threshold == 3. So we wouldn't be able to separate colors very well.
@@nikhilshaganti5585 Sure, you can continue to separate things in later branches - but the greedy nature of the algorithm doesn't ensure that you'll get to those later branches. So you start by making a guess that it makes sense to group red and blue together.
First you’ve saved me this is super clear! I love all your videos so much 😊 I do have two questions… 1. How would you handle a classification problem with time series data? 2. Is there any other evaluation test you should or could do to evaluate the effectiveness of your model?
1. I've never used XGBoost with time series (or done much of any time series stuff before), so I can't answer this question. 2. There are lots of ways to evaluate a model. I only present a few, but there are many more, and they really depend on what you want your model to do. Just google it.
Wonderful video josh.....pleasee pleasee pleasee make more videos on start to finish on python for different models.....i havr actually submitted my assignments using your techniques and got better results than what i have learned in my class Waiting for more to come especially on python :)
Again another quality video, I was following along with your every word, which did bring up come questions: 1. When XGBoost deals with missing data, does it ever consider splitting the missing data in half? --Using your example, would it ever do 1 blue and 1 green? What would happen if XGBoost encountered a data set with alot of missing values? 2. When you ran your Cross-Validation, was there a reason you only used 3 values for each hyperparameter? Could you have done more if you wanted to? 3. When I ran my Cross-Validation, my scale_pos_weight didn't change even though I used the same parameters you did. What do you think the problem could be?
1. Not that I know of. If there was a lot of missing values, it would still proceed just as described. 2. I wanted the cross validation to run in a short period of time, so I picked 3 values for each hyperparameter. If I had more time on my hand, or a cluster of computers, I might have considered trying more. 3. I'm not sure.
@@statquest Gotcha! Thank you for actually answering my questions haha I really appreciate the help as someone getting into more Machine Learning. Not sure what your upcoming videos will be but I think some great videos would be: -How you did the Cross Validation for the hyper parameters in this video? ( I have watched your Cross-Validation video, but I want to learn more about actually doing it) [I also still can get my first run to output the same values you had lol] -XGBoost for Regression in Python -How to select the best algorithm based on the scenario for Regression or Classification Just some thoughts :)
Appreciate the Python related videos...helps to manoeuvre the code when I try to replicate the method later on...easy to follow the whole thing, also for beginners... 🙂
Thanks Josh for another GREAT video! Just some sharing and minor questions. 1. try pandas_profiling when doing EDA. I personally love it. :) 2. some features are highly correlated (eg: city name and zip code). Do we need to handle that before running XGB? 3. Why choose 10 for early_stopping_rounds 4. What’s the difference between - df.loc[df['Total_Charges']==' '] - df[df['Total_Charges']==' '] 5. What’s the difference between - y=df['Churn_Value'].copy - y=df['Churn_Value'] Many thanks in advance! H
1) Thanks for the tip on pandas_profiling. 2) No. 3) It's a commonly used number 4) I don't know. 5) I believe the former is copy by value and the latter is copy by reference.
Greatly appreciated this videoLike you said, telcos should gives more effort to tie the current customers. In real practice you want to know what the probability is that a current customer will no longer renew the subscription. You should then try to bind the customer with a high risk with incentives.
Yikes, if I ever understand something enough to explain it as succinctly as you do then I'd be very happy. I've been smashing through a lot of your videos the last few days after spending countless months on python, sklearn and all the usual plug and play solutions and it's not been until I've started watching these that I've started to feel things click into place
Amazing. Wonderful videos. I started only 3 months ago and with your videos I am very confident to do nalysis with Python. Manny thanks. Is it possible to create a video for Nomogram for competic#ing risks for Time-Event (survival analysis) based on CPH outputs?
41:21 In my version of xgboost were parameters early_stopping_rounds=10 and eval_metric='aucpr' moved to XGBClassifier, so if it is not working I suggest trying this. Although I really appreciate value of StatQuest videos, I have to admit I really hate all that singing and Bams. Makes me feel like attending school for slower children
@@statquest Sir your double and triple bam has really taught lots of things for me to be honest. Me as a student doing Post-Graduate in AI, lots of love form Nepal.
38:05 stratify=y is not for yes is for dependent variable y. if you have a different variable, you gotta pass your response variable's name to stratify
Expertly explained. I've just come off a bootcamp style analytics course and I'm currently reviewing and building upon everything I've learnt. I'm finding StatQuest such a valuable ressource...Although, I think that XGboost just broke my pc😅
This is gold, thank you! I am a rookie of this stuff, still I am unsure one-hot encoding is the best to do, especially to encode the city; being a category with high cardinality, all those variables for 1-hot encoding will require many splits (I guess). Perhaps using a different encoding, like mean encoding or frequency encoding, would be better, may allow to have a good fit with fewer splits.
Thank you for very nice videos. Very helpful indeed. I am working in R and I believe you dont have tutorials in R. I have one question to ask. I am using XGBoost for regression and instead of using one predicted value for each of the data point, I want to extract closest neighbors, hence I am trying to get all values from the terminal nodes. My question is. 1) If i have nround =100, should i get terminal nodes only from 100th tree because i believe that is the most accurate one. 2) Can Terminal node values using predict function(nodes = TRUE) be considered as final output value or should i use this formula previous probability + 0.3*T1 +0.3*T2+....+0.3*T100.
Thank you for the excellent content and walk through. While One Hot Encoding the categorical variables should not we use k-1 variables for k categories? For example, for the Payment_Method column if Mailed_Check=0, Electronic_Check=0, and Bank_Transfer=0 doesn't that imply Credit_Card=1 and make Credit_Card column redundant?
Hello Sir, Thanks a lot for the video with step by step very detailed explanation. Have a doubt, understand that from data science models we can do predictions, eg like in this video we tried to predict whether customer would stop taking company service or not, but does model answers (or provides info) about what were the main reasons due to which customers are deciding to stop taking service i.e which inputs datapoint/observations are the main drivers of predictions. Reason to ask this question is, in practical scenario when we do such customer churn predictions and present it to senior leaders/Management, the question from management will be, do we know what are the top 3 reasons why our customers would decide to stop taking our service ? Sorry for lengthy question, but got this doubt when I was thinking about how management would react when we present such predictions from model. Thanks in advance and for the awesome video once again.
hey first of all thank you so much for all your videos! I understood everything and i am hyped about trying out what i have learned, but i have one question. So after we built our model and we are happy with how it performs, how do we feed it with actual new unseen data? If i have for example just the data for one specific customer and i want to check if he/she leaves or stays, what would the code look like.
josh, i just absolutely love this video, after watching your videos about the theory part, then implement with real coding, is the best comb for learning. just have a few questions 1. when you use get_dummy, do we have to specify drop_first = True? normally, when should we drop first? 2. after converting using onehot, too many features were created and making X very sparse, is there a conventional way to deal with this problem? (say if instead of using xgb, using random forest, too many categorical converted features will dominate in feature selection for building the tree, how can we handle this problem?) 3. for labels, do we have to oversample the 1's since it's only 26%? i think this is a very interesting topic. as always, love your video and i want to show you my support and my appreciation to your amazing videos
just came up with a very silly question, i saw people are recommending using onehot to convert categorical variable, and not labelencoder (even doc says so), but tree based model like xgb and random forest, dont actually care about the ordinality, they care about gain and impurity reduction, so why we cannot use labelencoder?
1) In this case we don't need to drop the first - we're not worried about inverting a matrix or something like that to solve for parameters (i.e. like we would with linear regression). 2) If you use RandomForest in R, then you don't have to convert the categorical variables to one-hot, and you can avoid the sparse matrix problem. In general, the R implementation of RandomForests is infinitely better than the Python implementation. 3) It really depends.
Thank you so much for your hard work! I've learn so much watching your channel. Could you please explain why I shouldn't use one hot encoding while doing linear regression and what should I use instead?
Quick note: At 43:30 instead of using the plot_confusion_matrix() which is now depreciated, you need to use ConfusionMatrixDisplay.from_estimator(). This can be done as follows: Include: from sklearn.metrics import ConfusionMatrixDisplay at the start with the other imports. Then when printing the confusion matrix you need to use the line: ConfusionMatrixDisplay.from_estimator(clf_xgb, X_test, y_test, display_labels=["Did not leave", "Left"])
I am getting this error following what you say: XGBoostError: [17:07:52] c:\buildkite-agent\builds\buildkite-windows-cpu-autoscaling-group-i-08de971ced8a8cdc6-1\xgboost\xgboost-ci-windows\src\c_api\c_api_utils.h:167: Invalid missing value: null Do you know why it might be? Thanks.
I solved the above error by setting missing=0 on the above code as follows: clf_xgb = xgb.XGBClassifier(objective='binary:logistic', missing=0, seed=42, ## the next three arguments set up early stopping. eval_metric='aucpr', early_stopping_rounds=10)
We need more advanced statistics content :) how about stochastic processes e.g. Brownian motion. A lot of us would like more complicated topics to be explained by you :)
Hey! Thank you so much for all the work you have been doing! I have a question regarding the leaves values if we use xgboost in a regression problem. What do they mean then? Is it probability that the average of the observations that were segregated to that particular leaf has this and this probability of contributing to the loss function? Thank you so much for your help!
NOTE: You can support StatQuest by purchasing the Jupyter Notebook and Python code seen in this video here: statquest.gumroad.com/l/uroxo
Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/
Dear Josh ... I have a request for new videos or livechats ... could you explain us these tests maybe ? ... Tukey, Bonferroni and Scheffé , it's hard for me to understand , you explain everything so well ... could be very helpful for a lot of people out there ... have a nice day , greetings from Europe
Please keep doing these long-form Python tutorials on the various ideas we've covered in earlier 'Quests. They're great for those of us working in Python, and they give me another way to support the channel. It has been a more-than-pleasant surprise that as I've grown from learning the basics of stats to machine learning and eventually deep learning, StatQuest has grown along with me into those very same fields.
Thanks Josh.
That's the plan!
You are amazing. Can't imagine how much work you put into those step-by-step tutorials.
Just bought the Jupyter Notebook code and it's beyond worth it! Thank you :)
Thank you very much for your support! :)
Hey Josh, I just purchased all of your 3 Jupyter Notebook! I transferred from Econ major to Data Science, it was a nightmare before I find your channel. Your channel shed the light upon my academic career! Look forward to more of the 'Python from Start to Finish' series, and I will definitely support it!
Awesome! Thank you!
Your pronunciation is the most authentic and clearest that I have ever heard
Wow! Thank you!
Outstanding! Complete with the thinking path - how to analyze variables in a logical way - and with common errors. Just purchased the Notebook. Thank you for all of your work on this channel.
Triple bam!!! Thank you very much for supporting StatQuest! :)
I am purchasing the Jupiter notebook to contribute to your work! Thanks a lot for this video! You are awesome! Will be very very happy to have more ML tutorials and thank you Josh!
Thank you very much! :)
This is hands down the best Python tutorial on TH-cam.. not just for XGBoost, but overall Python logic and syntax. Nice work, subscribed!!
Wow! Thank you!
Wow. Finally I see a face for the name. Your previous videos have had immensely helpul. I assumed you are a very senior person. I am not measuring your age. I mean, your way of explaining seemed like a professor with half a century of experience. But in reality, you are quite young. Thank you for all your simple-yet-detailed videos. No words to quantify how much I appreciate them. 🙏
Wow, thanks!
Extremeley helpful - would love to see more from the "start to finish" series
I'm working on it.
What did we do to deserve a great guy like Josh ? Thank you Josh!
Thanks! :)
I love your teaching style. Extremely helpful for a beginner like me. Really helped me a lot in my exams. No words. You are the best!!!!
Thank you!
Josh, you’re well and truly phenomenal ! Love from Madras !
Chennai
BAM! Thank you very much!!!
Hi Rahul I taught atIIT-madras 19192-1993 lived on campus across from post office josh visited us there
Frank Starmer Hello Frank, wow! That’s great to know ! :) I’m sure you must have had a good time here. Cheers :)
@@starmerf wow the world is a small place ☺
Josh, this video is epic and really helped me understand the actual process of tuning hyperparameters, something that had been a bit of a black box until I saw this video. Your channel is awesome too - great jingles as well :D
Thank you!
"25:36" that's what i was waiting for from the beginning...Truly amazing.. You are providing precious information..CHEERS
Glad it was helpful!
@@statquest one small request..can you provide some valuable information through a video like which model to chose for different datasets..how do we decide what model we should chose...thanks in advance
@@RahulVarshney_ I'll keep that in mind. In the mean time, check out: scikit-learn.org/stable/tutorial/machine_learning_map/index.html
@@statquest that is amazing ...i will complete it today itself thanks again for your prompt reply
Can i get your email
This kind of content is SUPER HARD to produce. I really understand and appreciate your effort here. Thanks and congratulations.
Thank you very much!
Liked, favorited, recommended, shared, and sacrificed my first-born to this video.
TRIPLE BAM! :)
😂😂😂
I'm very grateful to have you as my teacher.
Thanks!
Thanks for the great tutorial! You covered a lot of details (mostly data cleaning) that are often overlooked or skipped as 'trivial' steps.
Thank you! Yes, "data cleaning" is 95% of the job.
I watched your all video for XGBoost. It helps me a lot. very appreciated!
Glad it helped!
Yes pls more videos with python❤thank u for the webinar
Thanks! :)
I was wondering how to find stuff regarding dealing with actual churn data and sampling issues. The tutorial addressed a lot of them. Thanks!
Thanks!
Can't thank you enough for the clearest and best explanation on TH-cam
Thank you!
Awesome video! The cleanest xgboost explanation a have ever seen.
Wow, thanks!
Hey Mate, amazing tutorial. Very complex problem explained in really simple and effective way. I am using XGBOOST for one of the classification model and after watching your video it made me realise I can further improve my model. So thank you again and keep making those videos. Kudos to you and long live data science 🙏🙏
Glad it helped!
I haven't watched it yet but I know this will be great!!!!!!!! Thank you Josh.
BAM! :)
Hi Josh, great job really helpful material as I'm discovering XGBoost just now.
Thank you and keep you great work!
Thank you very much! :)
Absolutely loved this video Josh. It breaks down everything into understandable chunks. Thank you and God bless. BAM! The only thing I missed (and its very minor) was taking in a new data row and making an actual prediction by using the model.
Thanks! For new data, you just call clf_xgb.predict() with the row of new data.
Thank you Josh. Needed this tutorial to better solve a ML Problem as part of my internship :)
Glad it helped!
Such an amazing job Josh.. Couldn't find any better explanation than this!
Mesmerizing!
Wow, thanks!
I really appreciate your content Josh. Thanks for your time
Thank you!
A true, real Master Class - You got my support!
Thank you! :)
Jesus, i just learned more over 10 minutes of this than i did throughtout an entire semester of a similar subject on CS. ++ tutorial
Thank you!
Great video! Love it!
request that you do a comparison of XGBoost, CatBoost, and LightGBM, and a quest on ensemble learning.
I'll keep those topics in mind.
It was extremely helpful. Please continue making these videos. I suggest making a video to explain the clustering with unlabeled data, and predicting the future trend in time-series data.
I'll keep that in mind. :)
This is a great piece of work, thanks for sharing it!
Maybe the only additional piece I'd add which I've found useful on the documentation of XGBoost is that one can take advantage of parallel computing (more cores or using a graphic card your machine or you could have on the cloud) by simply passing the parameter (n_jobs = -1) while doing both, the RandomizedSearchCV stage and the setting the XGB regressor type (XGBRegressor for example).
Great tip! BAM!
Thank you for your job, the explanation of the topic is very clear and transparent.
Thank you very much! :)
You're a very kind human being Josh!! Thank you so much for making these videos. Your content is gold!!! I am new to data science and this is exactly what I needed!! :)
Much love from India!
Glad you like my videos!! BAM! :)
@@statquest Hey Josh! I am learning about Bayesian Optimizer and I don't seem to get it even after watching tons of tutorials, can you suggest where I should learn it from please? I couldn't find a video on your channel on this.
@@muskanroxx22 Unfortunately I don't know of a good source for that.
BAMMMMM !!!
This is awesome 👍 Josh !! Thank you for your contribution, really helpful for new learners.😊😊😊
Glad you liked it!
@@salilgupta9427 Thanks!
It is really lovely to be able to put a face to the "Hooray!", "BAM !!!" and "Note:"s 😄❤
bam!
Thank you so much for the work that you used in step by step tutorial. it was amazing.
You're very welcome!
Amazingly organized and well explained!
Thank you!
This man deserves Nobel Prize for peace of mind ❤❤
bam! :)
Josh, you're the didactic in person form.
Thanks!
I appreciate that!
Thank you for this great tutorial Josh! Your videos have immensely helped in understanding some of the complex topics. One thing I noticed while watching this tutorial is the handling of categorical features. I think the explanation you gave for "Why not to use LabelEncoding?" is applicable for models like Linear Regression, SVM, NN but not for Trees because they only focus on the order of the feature values. For example, in a set of [1,2,3,4], threshold < 1.5 would be equivalent to threshold == 1. Please let me know if my thought process in wrong.
To be honest, I'm not really sure I understand your question or your example. If we have a categorical feature with 4 colors, red, blue, green, and black, and we give them numbers, like 1, 2, 3 and 4, then a threshold < 2.5, would not make much sense and, based on how trees are implemented, there would be no options for threshold == 2 or threshold == 3. So we wouldn't be able to separate colors very well.
right, in your example, the threshold of threshold
@@nikhilshaganti5585 Sure, you can continue to separate things in later branches - but the greedy nature of the algorithm doesn't ensure that you'll get to those later branches. So you start by making a guess that it makes sense to group red and blue together.
Hard work here, I'ts funny how the responsabile scientist and the funny guy coexist, very useful lesson, thanks!
Thanks! 😃!
First you’ve saved me this is super clear! I love all your videos so much 😊
I do have two questions…
1. How would you handle a classification problem with time series data?
2. Is there any other evaluation test you should or could do to evaluate the effectiveness of your model?
1. I've never used XGBoost with time series (or done much of any time series stuff before), so I can't answer this question.
2. There are lots of ways to evaluate a model. I only present a few, but there are many more, and they really depend on what you want your model to do. Just google it.
Good to also see you sing rather than just hear :).. i had to comment this even before starting the training
😊 thanks
many thanks to your great and so understandable video. It literaly helps me a lot in Python and XGBoost package
Glad it helped!
Wonderful video josh.....pleasee pleasee pleasee make more videos on start to finish on python for different models.....i havr actually submitted my assignments using your techniques and got better results than what i have learned in my class
Waiting for more to come especially on python :)
Thanks! There should be more python coming out soon.
Great video! You did a wonderful job of explaining the process. Thanks!
Thanks!
I'm so glad you are a bad-ass stats guru and a teacher waaaaaaaaay before a singer and a guitarist ...Thank you! ;)
joshuastarmer.bandcamp.com/
StatQuest with Josh Starmer ...not bad. A poor man’s Jack Johnson 🤔
Just pulling your leg. Thanks for all the content on stats
hurray, i picture you totally different!
Thanks a lot for all the videos!
Glad you like them!
could not help u with money right now , but i watched all the adds in video , hope that helps u financially . love u videos . keep up!!
I appreciate that
Lovely and priceless video Josh...BAM BAM BAM as usual !! :) God bless. .
Thank you very much! :)
Again another quality video, I was following along with your every word, which did bring up come questions:
1. When XGBoost deals with missing data, does it ever consider splitting the missing data in half?
--Using your example, would it ever do 1 blue and 1 green? What would happen if XGBoost encountered a data set with alot of missing values?
2. When you ran your Cross-Validation, was there a reason you only used 3 values for each hyperparameter? Could you have done more if you wanted to?
3. When I ran my Cross-Validation, my scale_pos_weight didn't change even though I used the same parameters you did. What do you think the problem could be?
1. Not that I know of. If there was a lot of missing values, it would still proceed just as described.
2. I wanted the cross validation to run in a short period of time, so I picked 3 values for each hyperparameter. If I had more time on my hand, or a cluster of computers, I might have considered trying more.
3. I'm not sure.
@@statquest Gotcha! Thank you for actually answering my questions haha I really appreciate the help as someone getting into more Machine Learning. Not sure what your upcoming videos will be but I think some great videos would be:
-How you did the Cross Validation for the hyper parameters in this video? ( I have watched your Cross-Validation video, but I want to learn more about actually doing it) [I also still can get my first run to output the same values you had lol]
-XGBoost for Regression in Python
-How to select the best algorithm based on the scenario for Regression or Classification
Just some thoughts :)
@@andrewnguyen5881 I'll keep those topics in mind.
Love u Josh.... you are a TRIPLE BAM!!! Greetings from Bogotá, Colombia.
Muchas gracias!!! :)
Appreciate the Python related videos...helps to manoeuvre the code when I try to replicate the method later on...easy to follow the whole thing, also for beginners... 🙂
Thanks! There will be a lot more python stuff soon.
Thanks Josh for another GREAT video! Just some sharing and minor questions.
1. try pandas_profiling when doing EDA. I personally love it. :)
2. some features are highly correlated (eg: city name and zip code). Do we need to handle that before running XGB?
3. Why choose 10 for early_stopping_rounds
4. What’s the difference between
- df.loc[df['Total_Charges']==' ']
- df[df['Total_Charges']==' ']
5. What’s the difference between
- y=df['Churn_Value'].copy
- y=df['Churn_Value']
Many thanks in advance!
H
1) Thanks for the tip on pandas_profiling.
2) No.
3) It's a commonly used number
4) I don't know.
5) I believe the former is copy by value and the latter is copy by reference.
That's the Best video I've ever seen. Period.
TRIPLE BAM! :)
Wow, thanks!
just here to say thank you! will come back in a month when I have time to watch it. :)
BAM! :)
This guy is amazing.
DOUBLE BAM 💥 💥
Thank you! :)
you are simply an amazing human being, also the notebooks are great! :D
Thanks!
Greatly appreciated this videoLike you said, telcos should gives more effort to tie the current customers. In real practice you want to know what the probability is that a current customer will no longer renew the subscription. You should then try to bind the customer with a high risk with incentives.
True!
Great explanation and walk-through, big thanks!
Glad you enjoyed it!
Very enjoyable webinar Josh. Thanks for posting. I'm not 100% sure how to interpret the leaves. The largest leaf value is 0.188 where Dependents_No
The leaves are how much to increase or decrease the log(odds) for one category. For more details, see: th-cam.com/video/8b1JEDvenQU/w-d-xo.html
Yikes, if I ever understand something enough to explain it as succinctly as you do then I'd be very happy. I've been smashing through a lot of your videos the last few days after spending countless months on python, sklearn and all the usual plug and play solutions and it's not been until I've started watching these that I've started to feel things click into place
Awesome! I'm glad my videos are helpful! :)
Love these Python tutorials after watching theory videos:D
Glad you like them!
it would be a great idea to have an entire series of "start to finish"
Thanks!
Man, this video is awesome! Congratulations!
Thank you! :)
I love the channel! Eu aprendo + aqui do que a Graduação! You great josh!
Muito obrigado! :)
Amazing. Wonderful videos. I started only 3 months ago and with your videos I am very confident to do nalysis with Python. Manny thanks.
Is it possible to create a video for Nomogram for competic#ing risks for Time-Event (survival analysis) based on CPH outputs?
I'll keep that topic in mind.
Freaking amazing! You explain everything so well. Thank you!
Thank you!
all week searched for this thank u very much
Enjoy!
awesome!! I hope you can do a video about XGBoost with regression too.
I'll keep that in mind.
Thank you Josh!!
As a suggestion, you could do a StatQuest explaining the measures in market basket analysis?
I'll keep that in mind.
41:21 In my version of xgboost were parameters early_stopping_rounds=10 and eval_metric='aucpr' moved to XGBClassifier, so if it is not working I suggest trying this.
Although I really appreciate value of StatQuest videos, I have to admit I really hate all that singing and Bams. Makes me feel like attending school for slower children
Sorry to hear that.
Hello josh, you are doing amazing work keep doing
Thanks!
Amazing! Thanks so much for the detailed video
Thanks!
I am not a math geek. I am here strictly for the intro song!
same here
DOUBLE BAM! :)
@@statquest Sir your double and triple bam has really taught lots of things for me to be honest. Me as a student doing Post-Graduate in AI,
lots of love form Nepal.
Excellent Video @StatQuest ! Can we please have more Start to Finish python videos? Like Lightgbm maybe?
I'll keep that in mind! :)
38:05 stratify=y is not for yes is for dependent variable y. if you have a different variable, you gotta pass your response variable's name to stratify
Oops! thanks for catching that.
Expertly explained. I've just come off a bootcamp style analytics course and I'm currently reviewing and building upon everything I've learnt. I'm finding StatQuest such a valuable ressource...Although, I think that XGboost just broke my pc😅
bam! :)
Great video! Very informative and clearly explained! Could you please also present BART?
I'll keep that in mind.
Thank you so much Josh Starmer! BAM!
bam!
This is gold, thank you! I am a rookie of this stuff, still I am unsure one-hot encoding is the best to do, especially to encode the city; being a category with high cardinality, all those variables for 1-hot encoding will require many splits (I guess). Perhaps using a different encoding, like mean encoding or frequency encoding, would be better, may allow to have a good fit with fewer splits.
Maybe. Try it out and let me know if you get something that works better.
Thank you for very nice videos. Very helpful indeed. I am working in R and I believe you dont have tutorials in R. I have one question to ask. I am using XGBoost for regression and instead of using one predicted value for each of the data point, I want to extract closest neighbors, hence I am trying to get all values from the terminal nodes. My question is. 1) If i have nround =100, should i get terminal nodes only from 100th tree because i believe that is the most accurate one. 2) Can Terminal node values using predict function(nodes = TRUE) be considered as final output value or should i use this formula previous probability + 0.3*T1 +0.3*T2+....+0.3*T100.
amazing tutorial Josh! Shared with my friends =D
Could you do one of these about pygam? It would be amazing :)
I'll keep that in mind.
Thank you for the excellent content and walk through. While One Hot Encoding the categorical variables should not we use k-1 variables for k categories? For example, for the Payment_Method column if Mailed_Check=0, Electronic_Check=0, and Bank_Transfer=0 doesn't that imply Credit_Card=1 and make Credit_Card column redundant?
In this setting it doesn't matter - unlike linear models, xgboost doesn't have to worry about inverting a matrix to solve for parameters.
Hello Sir, Thanks a lot for the video with step by step very detailed explanation. Have a doubt, understand that from data science models we can do predictions, eg like in this video we tried to predict whether customer would stop taking company service or not, but does model answers (or provides info) about what were the main reasons due to which customers are deciding to stop taking service i.e which inputs datapoint/observations are the main drivers of predictions. Reason to ask this question is, in practical scenario when we do such customer churn predictions and present it to senior leaders/Management, the question from management will be, do we know what are the top 3 reasons why our customers would decide to stop taking our service ? Sorry for lengthy question, but got this doubt when I was thinking about how management would react when we present such predictions from model. Thanks in advance and for the awesome video once again.
You can always look at the features that are relatively high in the trees to find out what are the most important.
hey first of all thank you so much for all your videos! I understood everything and i am hyped about trying out what i have learned, but i have one question. So after we built our model and we are happy with how it performs, how do we feed it with actual new unseen data? If i have for example just the data for one specific customer and i want to check if he/she leaves or stays, what would the code look like.
Just use "clf_xgb.predict(YOUR DATA)", where "YOUR DATA" is formatted in the same order as the data used for training.
@@statquest oh god of course thank you! I had a blackout for a moment haha
Grate video just one comment for the eval_metric = 'aucpr' it is the area under the percussion recall curve and not the FP TP curve
Noted
josh, i just absolutely love this video, after watching your videos about the theory part, then implement with real coding, is the best comb for learning. just have a few questions
1. when you use get_dummy, do we have to specify drop_first = True? normally, when should we drop first?
2. after converting using onehot, too many features were created and making X very sparse, is there a conventional way to deal with this problem? (say if instead of using xgb, using random forest, too many categorical converted features will dominate in feature selection for building the tree, how can we handle this problem?)
3. for labels, do we have to oversample the 1's since it's only 26%? i think this is a very interesting topic.
as always, love your video and i want to show you my support and my appreciation to your amazing videos
just came up with a very silly question, i saw people are recommending using onehot to convert categorical variable, and not labelencoder (even doc says so), but tree based model like xgb and random forest, dont actually care about the ordinality, they care about gain and impurity reduction, so why we cannot use labelencoder?
1) In this case we don't need to drop the first - we're not worried about inverting a matrix or something like that to solve for parameters (i.e. like we would with linear regression).
2) If you use RandomForest in R, then you don't have to convert the categorical variables to one-hot, and you can avoid the sparse matrix problem. In general, the R implementation of RandomForests is infinitely better than the Python implementation.
3) It really depends.
To be honest, I'm not sure.
@@statquest thank you for your clarification josh, i will definitely dig deeper into how to solve the sparsity problem once utilize onehot
@@statquest this is a quite interesting problem, lol, never mind, let me find out what others do on kaggle
Thank you very much for nice video! Very helpful for me.
Glad it was helpful!
wow you are great instructor, very well explained! Are you also familiar with big data such as apache spark?
Not yet.
Thank you so much for your hard work! I've learn so much watching your channel. Could you please explain why I shouldn't use one hot encoding while doing linear regression and what should I use instead?
I explain how to encode things for linear regression in this video: th-cam.com/video/CqLGvwi-5Pc/w-d-xo.html
Quick note: At 43:30 instead of using the plot_confusion_matrix() which is now depreciated, you need to use ConfusionMatrixDisplay.from_estimator(). This can be done as follows:
Include: from sklearn.metrics import ConfusionMatrixDisplay at the start with the other imports. Then when printing the confusion matrix you need to use the line: ConfusionMatrixDisplay.from_estimator(clf_xgb, X_test, y_test, display_labels=["Did not leave", "Left"])
Thank you very much! I really appreciate it.
@@statquest no problem!
I am getting this error following what you say:
XGBoostError: [17:07:52] c:\buildkite-agent\builds\buildkite-windows-cpu-autoscaling-group-i-08de971ced8a8cdc6-1\xgboost\xgboost-ci-windows\src\c_api\c_api_utils.h:167: Invalid missing value: null
Do you know why it might be? Thanks.
I solved the above error by setting missing=0 on the above code as follows:
clf_xgb = xgb.XGBClassifier(objective='binary:logistic',
missing=0,
seed=42,
## the next three arguments set up early stopping.
eval_metric='aucpr',
early_stopping_rounds=10)
@@emilioluissaenzguillen5719 thanks it works
We need more advanced statistics content :) how about stochastic processes e.g. Brownian motion. A lot of us would like more complicated topics to be explained by you :)
I'll keep those topics in mind.
Hey! Thank you so much for all the work you have been doing! I have a question regarding the leaves values if we use xgboost in a regression problem. What do they mean then? Is it probability that the average of the observations that were segregated to that particular leaf has this and this probability of contributing to the loss function? Thank you so much for your help!
see: th-cam.com/video/OtD8wVaFm6E/w-d-xo.html
If "Stay Cool" had a face, it'd be you :)
Bam!