Most underrated TH-cam channel ...even though he teaches much better and easy to understand than so many TH-cam channels which confuse you with lot of stuff.
@@deepeshmhatre4291 dude comback after a year and reply. i was talking about the number of views he was getting and yeah a lot can change in 4months fool.
Great work. I started studying statistics with your video, and i find them amazing. I just wanted to ask you to check if the value of Q3 on your example is wrong. According to my calculations Q3 should be 6.15 and not 6.27. I calculated this using excel formula percentile.inc(array;0,75). I understand this is not so important, cause the value in your videos is the understanding of how to use these methods to do a better data analysis. Thank you.
I have followed your python and machine learning playlists and learnt many new things. They are quite amazing and helped me to solve many machine learning problems. Sir would you please suggest a playlist to learn and code in JAVA at Intermediate level as I have to attend campus placement within 3 months.
Thanks for another amazing video.Could you also please make a video explaining different scenarios on how to decide when to use which Outlier technique?
such a great explanation, you are teaching really well. I have watched your entire ML videos. Thanks for providing us such a good explained ML videos I just wanna request you to please add one more Algorithm to your ML video series of XGboost , how to install it and all the necessary stuffs thank you so much
I came here after seeing friends spending 12-13lakh rupees on "university" courses which don't realize value and later repent. Thank you for this content brother. May you achieve many more milestones in life !
Thank you so much Sir for this amazing techniques. Though one request, could you please make some tutorials on how to select best features from the dataset i.e. FEATURE SELECTION
Sir...this playlist is really awesome.....I just want to know...this is all about Feature Engineering or you will upload videos soon...Thank You.....We all feel Glad If you were reply us.
your teaching skill is excellent ! alot of subtle point has been address directly to the viewer in a neat and professional manner. What is the average time you spend on making the script and video ?
Thanks Dear, be real, produce genuine content and you will be successful. Average time for making the script and video depends on the content. Let’s say for making video on project might take weeks while video on tips takes some days only.
You have shown us an example of only 1 column 'height'. Thanks for the concept. What if we have a larger dataset of multiple columns for example 15 columns out of which there are 12 numerical columns. Do we need to plot Box Plots for each column to check the outliers ? Or going by IQR method, do we need to write the above IQR codes for each and every variable ? Is there any shorter method ? Please advise.
to detect the outliers we can simply use boxplot or histogram to give us a clear representation about the data. I Usually use IQR to detect the specific index of the outliers to be then treated.
@@codebasics I am doing well. I have to watch all your recent videos on deep learning I saw in notification they all are awesome. I will watch like and comment.
Hello sir thanks for sharing your knowledge sir i am worked on outliers by using z score after filter data if I checked outliers by using box plot it shows outlier please tell me sir. Whether I need continue with next process or I need remove outlier one more time sir .
Hi, thank you for this great video. I tried to code it exactly but the display of the dataset is different, separating the columns with a semicolon. Any idea why? Name;Height 0 Mohan;1.2 1 Maria;2.3 2 Sahib;4.9 3 Tao;5.1 4 Virat;5.2 5 Khusbu;5.2 6 Dmitry;5.5 7 Selena;5.5 8 John;5.6 9 Imran;5.6 10 Jose;5.8 11 Deepika;5.9 12 Joseph;6 13 Binod;6.1 14 Gulshan;6.2 15 Johnson;6.5 16 Donald;7.1 17 Aamir;14.5 18 Ken;23.2 19 Liu;40.2
Hi Sir, outliers should only be removed from x variables or y variable as well? My understanding is that we should do it for only for x variables. Appreciate your advise sir
@@codebasics Sure sir, what I meant was that outlier removal exercise should be performed only x (independent) variables, not on y (dependent) variable. Yes, when we remove x, entire row will be deleted including y, but outlier removal is focused only on x variables. Hope this statement is correct.
No always necessary.. it depends. but majority of the time yes we end up *treating* them which means not necessarily remove it but change the value to say median.
@@codebasics *Threshold* can be set with the specified number of Standard Deviation. *Default value is 3* . Outliers increase the standard deviation, so it may fail to detect outliers. more extreme the outlier, more SD is affected. So, 3 is the default.
This thing of how to calculate the percentile is messing me up........by using the formula percentile=100*(i-0.5)/100....we get different values of percentile.... please clear me this basics
I separated deep learning videos into a separate playlist because it is not appropriate to have deep learning videos in a playlist "machine learning for beginners" To find other videos in youtube search "codebasics deep learning"
sir I have looked at all 3 tutorials of your removing outlier, Now i have data set I need to remove outliers but I have no idea, which method i need to use,what are the thing i need to consider to find bet outlier removal method for my data set?
Sir I have question, if i have 20+ columns in dataset then should select each column and find out lier and delete ? If so what is the column3,4 has the outlier but rest all columns values are perfect. Then do we delete the outlier rows data in whole dataset?
You don't need to necessarily delete them but rather *treat* them. Which means if outlier seems to be occuring due to a data collection error may be one approach is to replace it with either a median or mean value or any other value that is appropriate based on a situation
From the given dataset I am lower_bound as a negative value is it okay because I read a blog in which it is mentioned that an IQR cannot be negative. Any help would be really appreciaed
I would suggest doing a degree in data analysis if your country has such colleges providing specialization in data analysis. Otherwise go for either computer science or a major in mathematics, statistics etc. Once you study for few years you will have good grasp on many different concepts and then you can start applying for data analyst job. As such you can learn everything on your own too but that would require lot of self discipline, hard work. But yes you can do it in your own for sure without going to a college
Check out our premium machine learning course with 2 Industry projects: codebasics.io/courses/machine-learning-for-data-science-beginners-to-advanced
i am interested for data science course
Came back to this video after a long time. Just noticed that the video description itself explains everything straight to the point Sir. Great job
The way of explaining in simply way is very good. Thanks a lot .
Most underrated TH-cam channel ...even though he teaches much better and easy to understand than so many TH-cam channels which confuse you with lot of stuff.
underrated ? you gotta be a fool then, he has more than 280K subs.
@@deepeshmhatre4291 dude comback after a year and reply. i was talking about the number of views he was getting and yeah a lot can change in 4months fool.
Interested VS committed.... awesome.... one have to be committed..... it's very clear now...thanks for the resources bro...
All the best :)
Thanks man very useful you are the best . I've watched your entire machine learning tutorial . I learned many things from this.
Thank you! I needed to address outliers in a data set that I was performing an ANOVA test on, and this helped a lot.
Amazing Video Series. You are the best teacher on internet. Huge Respect from Pakistan.
All I can think about is aamir skipping the stairs and entering through the windows xD
btw. great tutorial!!
Kudos to sharing your knowledge in such a simplified manner. Exercises at the end gives a huge confidence. Thanks !! :)
🙂👍
You always teaches complex topics with so much ease, great lec...sir please add more tutorial in list
eres un excelente profesor, entiendo la explicacion en Ingles perfectamente, desde peru , gracias por compartir tus conocimientos!!
Very very good simple and beautiful explanation
Sir you are awesome 👍
Sir please upload video on reinforcement learning.Your teaching is great.May God bless you and your family.
Thank you so much for this very helpful video. You managed to explain the needed concept straight to the point. Awesome!
Just a clear cut explanation
Great work. I started studying statistics with your video, and i find them amazing. I just wanted to ask you to check if the value of Q3 on your example is wrong. According to my calculations Q3 should be 6.15 and not 6.27. I calculated this using excel formula percentile.inc(array;0,75). I understand this is not so important, cause the value in your videos is the understanding of how to use these methods to do a better data analysis. Thank you.
Happy teacher's day Sir 🎉
Just Awesome. Really loved the teaching style. Waiting for more tutorials..
I am glad it was helpful
I have followed your python and machine learning playlists and learnt many new things. They are quite amazing and helped me to solve many machine learning problems.
Sir would you please suggest a playlist to learn and code in JAVA at Intermediate level as I have to attend campus placement within 3 months.
such an amazing series..Thank you very Much Sir..I am Learning A lot daily from You Sir..
Glad to know kirandeep
thanks for all your efforts..may God bless you Sir.
Such an amazing video series, Thank you very much sir, keep continue.
Thanks a lot. Awesome tutorials. Looking forward to more amazing contents in Feature engineering series.
Thank you, such a simple explanation about the IQR and Python Code too!
Thanks for this very useful video!
Thank you sir, for explaining the concept so lucidly.
Happy to help
Thanks for another amazing video.Could you also please make a video explaining different scenarios on how to decide when to use which Outlier technique?
Thanks sir.i used to mug up those code but now i can code myself
I am pretty much thankful for this video sir, I improved my understanding!
such a great explanation, you are teaching really well.
I have watched your entire ML videos.
Thanks for providing us such a good explained ML videos
I just wanna request you to please add one more Algorithm to your ML video series of XGboost , how to install it and all the necessary stuffs
thank you so much
Sure. Point noted
Sir please continue this series
Thanks for sharing this content sir
sir, how to remove outliers from multiple columns? do we have to always do it 1 by 1, or is there any trick?
I came here after seeing friends spending 12-13lakh rupees on "university" courses which don't realize value and later repent. Thank you for this content brother. May you achieve many more milestones in life !
Clear explanation..love it
Thank you so much Sir for this amazing techniques. Though one request, could you please make some tutorials on how to select best features from the dataset i.e. FEATURE SELECTION
Excellent sir
very simple video sir...tks. a lot..
Sir...this playlist is really awesome.....I just want to know...this is all about Feature Engineering or you will upload videos soon...Thank You.....We all feel Glad If you were reply us.
Mohit the playlist is not complete. I will upload more videos in future
@@codebasics thnq so much sir
your teaching skill is excellent ! alot of subtle point has been address directly to the viewer in a neat and professional manner.
What is the average time you spend on making the script and video ?
Thanks Dear, be real, produce genuine content and you will be successful. Average time for making the script and video depends on the content. Let’s say for making video on project might take weeks while video on tips takes some days only.
Thank you for these amazing videos.
You have shown us an example of only 1 column 'height'. Thanks for the concept. What if we have a larger dataset of multiple columns for example 15 columns out of which there are 12 numerical columns. Do we need to plot Box Plots for each column to check the outliers ? Or going by IQR method, do we need to write the above IQR codes for each and every variable ? Is there any shorter method ? Please advise.
to detect the outliers we can simply use boxplot or histogram to give us a clear representation about the data. I Usually use IQR to detect the specific index of the outliers to be then treated.
waiting for next episode 🤞
I think the 1.2 and 2.3 height ranges are possible because we don't have age data. the lower limit is not clear .. plz explain ...
Good video!!
Great work Dhaval. My name is there in the excel :)
Yes 😊👍 hope you are doing well aamir.
@@codebasics I am doing well. I have to watch all your recent videos on deep learning I saw in notification they all are awesome. I will watch like and comment.
Hello sir thanks for sharing your knowledge sir i am worked on outliers by using z score after filter data if I checked outliers by using box plot it shows outlier please tell me sir. Whether I need continue with next process or I need remove outlier one more time sir .
Hi, thank you for this great video. I tried to code it exactly but the display of the dataset is different, separating the columns with a semicolon. Any idea why?
Name;Height
0 Mohan;1.2
1 Maria;2.3
2 Sahib;4.9
3 Tao;5.1
4 Virat;5.2
5 Khusbu;5.2
6 Dmitry;5.5
7 Selena;5.5
8 John;5.6
9 Imran;5.6
10 Jose;5.8
11 Deepika;5.9
12 Joseph;6
13 Binod;6.1
14 Gulshan;6.2
15 Johnson;6.5
16 Donald;7.1
17 Aamir;14.5
18 Ken;23.2
19 Liu;40.2
Hi Sir, outliers should only be removed from x variables or y variable as well? My understanding is that we should do it for only for x variables. Appreciate your advise sir
When you remove outlier you are removing entire row which includes x and y both
@@codebasics Sure sir, what I meant was that outlier removal exercise should be performed only x (independent) variables, not on y (dependent) variable. Yes, when we remove x, entire row will be deleted including y, but outlier removal is focused only on x variables. Hope this statement is correct.
What is the dataset has multiple feature columns instead of just one as you have shown here? how would the code change? Please respond.
Great sir
Sir is this technique applicable to any dataset?
How did you find that "1.5" for upper and lower limit?
Thank you so much
I have a doubt here 3:10, to find lower/upper limit, 1.5 can be used for all type of data or only to few data types in particular
ALL
thanks for sharing
what if one of our observations is exactly equal to the upper or lower limit? Should we use = instead of < and > ?
brilliant
Don't want to remove outliers, want to boundary them as per the IQR values, what to do sir, stuck in this
Make a video series on seaborn please
Added that in my Todo list
Must we remove "outlier" from a data set? Thank you Teacher!
No always necessary.. it depends. but majority of the time yes we end up *treating* them which means not necessarily remove it but change the value to say median.
now i'm able to identify outliers easily by watching this video. *Why we are multiplying IQR with 1.5 why not 2.5?*
That's just the statistical rule of thumb. It is like asking why do we use 3 STD deviation or more for removing outliers..😊👍
@@codebasics *Threshold* can be set with the specified number of Standard Deviation. *Default value is 3* . Outliers increase the standard deviation, so it may fail to detect outliers. more extreme the outlier, more SD is affected. So, 3 is the default.
Sir I want to be a data scientist...should I learn ml 1st and then start data science?..plz do reply...
ML is a way to do data science. They are not separate
Why do we have to multiply IQR by 1.5 only? (while calculating lower and upper limit)
Sir plz suggest me best youtube channel for learning tableau
Thank you!
Sir Please make video on real-world data EDA
This thing of how to calculate the percentile is messing me up........by using the formula percentile=100*(i-0.5)/100....we get different values of percentile.... please clear me this basics
There were many other videos in this playlist, where are they now?
I separated deep learning videos into a separate playlist because it is not appropriate to have deep learning videos in a playlist "machine learning for beginners" To find other videos in youtube search "codebasics deep learning"
Thank you
what if data has no outlier. In that case we will loose tiny data? how to know if not outlier removal is needed in big dataset?
sir from where you take 1.5 plz tell nah
Is there any difference between quantile & percentile?
What if the dataset has many attributes like height here? shall we check individually for all?
perfect... perfect... thank you..
I have a question hopefully a quick reply I do understand the concept but I need to see how to remove outliers for multiple columns
same, did u identify?
Hi Sir while doing assignment I am getting error saying can only compare identically labeled series objects. Please explain why I am getting this.
Is it complete tutorial for ml????
sir can you tell me if i have more columns how i use iqr
amazing, thank you!
#If there is more than one, variable is code?;
Q1=df.height,weight,age.qaurtile(0.25)
Q3=df.gieght,weight,age.quartile (0.75)
sir I have looked at all 3 tutorials of your removing outlier, Now i have data set I need to remove outliers but I have no idea, which method i need to use,what are the thing i need to consider to find bet outlier removal method for my data set?
@@neerajv4799 thank you 🙏
@@neerajv4799 yes
Sir I have question, if i have 20+ columns in dataset then should select each column and find out lier and delete ? If so what is the column3,4 has the outlier but rest all columns values are perfect. Then do we delete the outlier rows data in whole dataset?
You don't need to necessarily delete them but rather *treat* them. Which means if outlier seems to be occuring due to a data collection error may be one approach is to replace it with either a median or mean value or any other value that is appropriate based on a situation
how to perform this when the dataset has around 100 independent variables?
same, did u identify?
love u sirrrrrrr
Hi Can you please give me any reference book for this videos
From the given dataset I am lower_bound as a negative value is it okay because I read a blog in which it is mentioned that an IQR cannot be negative. Any help would be really appreciaed
hey,yes i got the same,negative value ,pls can any one reply with answer
Take a bow🙏
Glad it was helpful!
Sir can l become data analyst after 12
I would suggest doing a degree in data analysis if your country has such colleges providing specialization in data analysis. Otherwise go for either computer science or a major in mathematics, statistics etc. Once you study for few years you will have good grasp on many different concepts and then you can start applying for data analyst job. As such you can learn everything on your own too but that would require lot of self discipline, hard work. But yes you can do it in your own for sure without going to a college
@@codebasics
Sir if I do direct data analyst course after 12th ,can I get success on this path and thanks for replying me Sir.
Please do not remove outlier, you should either keep it or preprocess the values never remove
What is data frame here?
Plz watch my pandas tutorials playlist. First one, you will get an idea