If you're not already familiar with ColumnTransformer, go back and watch tip #1 here: th-cam.com/video/NGq8wnH5VSo/w-d-xo.html Let me know if you have any questions, and thanks as always for watching! 🙌
@data school , Ok, but what if I want to select 3 columns. 2 applying a transformation e.g. Embarked and Sex but Fare keeping at it is and excluding Age from my transformation?!
@@JoaoVitorBRgomes You only pass the columns you want to use to the fit_transform method. So, you define your X as Embarked, Sex, and Fare. Then, you create a ColumnTransformer with a single tuple specifying the transformation for Embarked and Sex, and you set remainder='passthrough' so that Fare is passed through unmodified. Hope that helps!
@@dataschool will you be adding more to this playlist, im planning on starting ML now and im really looking forward to your lessons . Big Fan ❤️ keep posting ⚡️
I'm not currently planning to add to that playlist, though I have many more hours of content in my ML courses: www.dataschool.io/ml-courses/ Hope that helps!
You can also pass a callable (e.g. a self defined helper function that excpects the column series as input and returns a TRUE/FALSE) to the columns parameter. If the callable returns false the column will not be processed, if the callable returns true the column will be processed. Example: pasteboard.co/JC5zccR.png This way allows a very granual column selection and distinction e.g. seperate high cardinality columns in a different pipeline.
Hi, kevin i have a question about the Column Transformer, how do you manage to avoid redundancy? for example when you perform one hot enconding on sex column, normally we kept only one column instead of 2, how do you manage that? thanks and great job
You can drop the first column using the drop='first' parameter of OneHotEncoder, but in general, I recommend avoiding that. Here's why: 1. Multicollinearity is rarely an issue with scikit-learn models 2. drop='first' is incompatible with handle_unknown='ignore' 3. May be problematic if you standardize all features or use a regularized model
Is it possible to inverse transform the ct object to see the (orginal to encoded) transition. In the docs of SCIKIT learn i could find for the single data type transition (onehot,label) but not for the heterogenous column transformer. can you help please....
Unfortunately, inverse_transform is not currently available for ColumnTransformer. You can see the open issue here: github.com/scikit-learn/scikit-learn/issues/11463
@@dataschool I will be looking forward to the video😇 Actually I am stuck in the time series project, I have the last 8 years of stock data and have to predict for the next 4 years
![[LIVE] : ONE ลุมพินี 89 | คู่เอก "ยอดไอคิว vs คิริลล์"](http://i.ytimg.com/vi/Z3vYkHJ_5Ig/mqdefault.jpg)
If you're not already familiar with ColumnTransformer, go back and watch tip #1 here: th-cam.com/video/NGq8wnH5VSo/w-d-xo.html
Let me know if you have any questions, and thanks as always for watching! 🙌
Yes that is helpful ❤️
@data school , Ok, but what if I want to select 3 columns. 2 applying a transformation e.g. Embarked and Sex but Fare keeping at it is and excluding Age from my transformation?!
what I did was to use passthrough with a slice of x_train, but isnt there a way built int sklearn?
@@JoaoVitorBRgomes You only pass the columns you want to use to the fit_transform method. So, you define your X as Embarked, Sex, and Fare. Then, you create a ColumnTransformer with a single tuple specifying the transformation for Embarked and Sex, and you set remainder='passthrough' so that Fare is passed through unmodified. Hope that helps!
you are a real life saver.
Wow, thank you!
Thank you so much, Kevin. You are making everything easy.
Happy to hear that!
Absolutely Beautiful! I was working on the same problem.
Thank you so much for your kind words! 🙏
Finally you made explanation video on scikit learn thank you so much 👍
You're very welcome! I also have more scikit-learn videos here: th-cam.com/play/PL5-da3qGB5ICeMbQuqbbCOQWcS6OYBr5A.html
@@dataschool will you be adding more to this playlist, im planning on starting ML now and im really looking forward to your lessons . Big Fan ❤️ keep posting ⚡️
I'm not currently planning to add to that playlist, though I have many more hours of content in my ML courses: www.dataschool.io/ml-courses/ Hope that helps!
Thank you. Is quite handy and useful
Glad it was helpful!
slice() worked with me , thanks
Great to hear!
You can also pass a callable (e.g. a self defined helper function that excpects the column series as input and returns a TRUE/FALSE) to the columns parameter. If the callable returns false the column will not be processed, if the callable returns true the column will be processed. Example: pasteboard.co/JC5zccR.png
This way allows a very granual column selection and distinction e.g. seperate high cardinality columns in a different pipeline.
Wow, thank you so much for sharing, I look forward to checking it out!
Hi, kevin i have a question about the Column Transformer, how do you manage to avoid redundancy?
for example when you perform one hot enconding on sex column, normally we kept only one column instead of 2,
how do you manage that? thanks and great job
ohe = OneHotEncoder(drop = 'First') will drop the first column of each encoded feature.
You can drop the first column using the drop='first' parameter of OneHotEncoder, but in general, I recommend avoiding that. Here's why:
1. Multicollinearity is rarely an issue with scikit-learn models
2. drop='first' is incompatible with handle_unknown='ignore'
3. May be problematic if you standardize all features or use a regularized model
thank you u explain sklearn how to work with model and every thing fine the next step (production )
You're welcome! One of my courses might be useful to you: www.dataschool.io/ml-courses/
Is it possible to inverse transform the ct object to see the (orginal to encoded) transition. In the docs of SCIKIT learn i could find for the single data type transition (onehot,label) but not for the heterogenous column transformer. can you help please....
Unfortunately, inverse_transform is not currently available for ColumnTransformer. You can see the open issue here: github.com/scikit-learn/scikit-learn/issues/11463
@@dataschool Thanks for your feed back and update
Sir, pls make video on Time series (Full topic cover in single tutorial)
Thanks for your suggestion!
@@dataschool I will be looking forward to the video😇
Actually I am stuck in the time series project, I have the last 8 years of stock data and have to predict for the next 4 years
Problem :
> install.packages("dplyr")
Installing package into ‘C:/Users/Nobody/Documents/R/win-library/4.0’
(as ‘lib’ is unspecified)
trying URL 'cran.rstudio.com/bin/windows/contrib/4.0/dplyr_1.0.2.zip'
Content type 'application/zip' length 1299904 bytes (1.2 MB)
downloaded 1.2 MB
Error in install.packages : cannot open file 'C:/Users/Nobody/Documents/R/win-library/4.0/file2f04521f5516/dplyr/help/figures/logo.png': Permission denied
I won't be able to help you troubleshoot, I'm sorry!