Thanks krish for this video .. I have a doubt, at last part of the video .. while converting from categorical feature to numerical feature 2001 pincode represented at one instance as 1 and at other instance it is represented as 0 .. on what basis we represented like this ?
what happen if we have all independent variable as categorical i.e. movies data set country origin,movie_type,director now i want to predict the imdb missing data how can i handle those categorical variable
I want to create box plot for categorical variable (like subscribed: yes/no) Firstly I wrote the code: train= pd.get_dummies(train['subscribed'], drop_first=1) And then for creating box plot: train['subscribed'].plot.box() But this will show error as- keyword: 'subscribed' Please let me know my mistake.
One Question Sir. I was working on a classification dataset. My out put variable is also categorical in nature . I applied OHE and later when i saw the heatmap it made no sense because the columns were bit blank. Correct me where i am wrong here
I have a doubt... When dealing with categorical values having many classes, you took all 2001's and find out the probability where O/P is 1. Suppose, that is coming 0.6(as in video). Now you are replacing all 2001's with 0.6, no matter O/P is 0 or 1... WHY? Should we not replace 2001's by 0.6 only if O/P is 1, else replace it with 0.4? Thanks for the video btw!
Hi Krish, could you please guide me how can I handle text column for a regression problem. It's not about encoding categorical features. But what I am looking for is---extracting some meaningful information from the existing column containing text data using string manipulation method from regex...Please recommend me an effective way of doing this.
Kris, but what if we have a regression problem then we would not have output as 0/1, then how do we encode the categorical features like pincode, do we use frequency/count encoding in there??
i think, we can use feature_selection like SelectKBest to get top k(any N no. upto total columns) which means these new features have strong relationship with target
@@aakashsinghrawat3313 yes, your approach is good. I also find one more approach. Here we will replace categorical values with their number of count. Eg) We have 29 states. and suppose we have 10K records and Delhi has come 500 times, Karnataka come 900 times. So, i will replace delhi by 500 and Karntaka by 900.
Hi Krish.. I wrote exactly the same codes simultaneously to practice, but my score came out to be -5.667 (I got a negative value) whereas you got 0.9304. I am not able to understand why am I not getting the same value. Please explain.
This is the simplest way of encoding the categorical features. Thanks man!!
Thanks krish for this video ..
I have a doubt, at last part of the video .. while converting from categorical feature to numerical feature 2001 pincode represented at one instance as 1 and at other instance it is represented as 0 .. on what basis we represented like this ?
Thank you so much, sir! You are the best teacher
Hi Krish,
can you show how to convert categorical variable to numeric variable through coding ?
thank you so much, this is actually clearer than the stupid class I enrolled earlier
Exactly what I was looking for! Thank you
you explain like pro bro.....
Thank you so much... It was so easy...
Thanks Krish
Well explained🎉🎉
what happen if we have all independent variable as categorical i.e. movies data set country origin,movie_type,director now i want to predict the imdb missing data how can i handle those categorical variable
I want to create box plot for categorical variable (like subscribed: yes/no)
Firstly I wrote the code: train= pd.get_dummies(train['subscribed'], drop_first=1)
And then for creating box plot: train['subscribed'].plot.box()
But this will show error as- keyword: 'subscribed'
Please let me know my mistake.
One Question Sir. I was working on a classification dataset. My out put variable is also categorical in nature . I applied OHE and later when i saw the heatmap it made no sense because the columns were bit blank. Correct me where i am wrong here
I have a doubt...
When dealing with categorical values having many classes, you took all 2001's and find out the probability where O/P is 1.
Suppose, that is coming 0.6(as in video). Now you are replacing all 2001's with 0.6, no matter O/P is 0 or 1... WHY?
Should we not replace 2001's by 0.6 only if O/P is 1, else replace it with 0.4?
Thanks for the video btw!
The O/p column comes from where can you explain me that?
Same doubt here
@Premjith Augustine what if the output is not a classification variable. Target variable can be Continuous like Price, Fees,Profit
that 0.6 is mean for 0 as well as to 1
Sir, please upload a video on how to perform mean encoding !!
Hi Krish, could you please guide me how can I handle text column for a regression problem. It's not about encoding categorical features. But what I am looking for is---extracting some meaningful information from the existing column containing text data using string manipulation method from regex...Please recommend me an effective way of doing this.
At 17:08 u made it clear for 2001 as 1 as output will be 2/3=0.6 what about 2001 as 0 as output?
actually the idea is getting mean of 2001, suppose 2001 has value 1 and 0 obvious mean will be 0.5 for both
how can we save the count of a particular category obtained to be used later in any calculation
Thanks for the video Krish.
When I ran the "df" command after concating, why all the values of Florida & New York comes as "NaN" ?
Thank you Krish.
please make video on visuvalistion using matplotlib
Kris, but what if we have a regression problem then we would not have output as 0/1, then how do we encode the categorical features like pincode, do we use frequency/count encoding in there??
if we have more than 5 categorical-feautures column, what to do for that? for example -- country, age group like this?
How to apply onehot encoding if we have categorical data in Y (dependent column).
Thank you so much for the video
what to do if there is mixed data, continuous and categorical?
Hi Krish,
suppose in the case if we have 8 categorical names at that time it will generate 8 new columns?
yes
getting key error for the column which I used for categorical data, please help
If we have large number of categorical variables say 21, then if we use get dummies we'll have large number of columns so how to deal such a case?
are u able to find, how to cater your problem ie 21 categorical variables?
i think, we can use feature_selection like SelectKBest to get top k(any N no. upto total columns) which means these new features have strong relationship with target
@@aakashsinghrawat3313 yes, your approach is good. I also find one more approach. Here we will replace categorical values with their number of count. Eg) We have 29 states. and suppose we have 10K records and Delhi has come 500 times, Karnataka come 900 times. So, i will replace delhi by 500 and Karntaka by 900.
@@gurjotsingh752 isn't it inefficient to labelling 500,900 to categorical feature? This method might be good to ordinal features.
Hi Krish.. I wrote exactly the same codes simultaneously to practice, but my score came out to be -5.667 (I got a negative value) whereas you got 0.9304. I am not able to understand why am I not getting the same value. Please explain.
Just set the seed once
Thanks bro
can someone please give me link of solved example using target encoding, mean encoding like above
What is the last encoding type called?
mean encoding