K- means becomes handy when dataset is large whereas Hierarchial clustering inefficient in this regard because it becomes computationally expensive . Time complexity is very high in case of hierarchial clustering as each data point has been considered as one single cluster to start off and then based on the euclidian distance with other data points the membership of the big cluster has been obtained. Whereas in K means , based on the K, the membership of the cluster is obtained following the centroid movement . Time complexity_k-means : order of n o(n) , hierarchial : o(n2).
A friend of mine suggested your channel and now I can see why. You are doing an amazing job! Please never stop. This is a community service kind of thing. I am trying to make a career shift and now I know where I am lagging behind. Thank you so much!
Hi, K Means - Here, no of centroids are formed and number of optimal centroids is decided by WCSS(Within Cluster Sum of Squares), could be achieved by Elbow method. Hierarchical Clustering - The goal is also same here, to find best features clustered together , technique being is DENDOGRAM. It is memory unit that stores that vectors which starts being Clustered together. If we have large dataset, K-Means would be easy to be used since Optimal number of clusters could be shown by method itself whereas in Hierarchical clustering we are supposed to manually find optimal clusters through dendogram.
K-means is when we have unlabeled data i.e, data without defined categories or groups. And centroids of k clusters can be used to label new data Hierarchical clustering : we use when we have a predetermined ordering from top to bottom it deals with small datasets where as k-means deals with large datasets
1.When there are huge number of points then Hierarchical clustering is avoided because of extremely large time complexity. 2. In the case of non globular shapes Min hierarchical clustering performs better than Knn clustering, as knn tends to cluster them in a more globular fashion. 3.typically when the clsusters are of different sizes or different densities Knn doesn't perform well. 4.each hierarchical clustering has its own adv and dis adv..Min hierarchical is sensitive to outliers.Max hierarchical though it is robust to noise, it tends to break down large clusters
*Firstly both are Unsupervised Learning Technic where the Data set has only Features and No Target Values in it. *In K-Mean Clustering first we will select the K value by using elbow method then we will initialize the Centroid randomly based on the K-value has taken points near to Centroid we will group and find the mean value and we keep changing the Centroid . and we will use Euclidean method to find the distance between the points. *In Hierarchical clustering we will use same Euclidean method to find the distance between the points then we will group the points, then to find the number of clusters we will use hack then we will check through how many points it has passed if the Hack has passed to 2 lines then we take 2 clusters or if 3 lines means we will take 3 clusters. * The major Difference is Hierarchical Clustering can't Handle Big Data Sets but K-Mean Cluttering Can Handle it.
We go for heircarchial clustering when less number of observation are there because in this we use dendogram,whera kmeans clustering (non-heircarchial ) we go when we have large number of observation.
The key to a successful Data Science solution is to treat every problem as an optimization problem. Which means, start with a basic model - try and improve it with hyperparameter tuning. Move to better models.
When we have prior knowledge of the number of clusters we can form with our dataset we can use k-means clustering. It is most probable with low dimensional datasets. When we have no prior information about the number of clusters we will go with Hierarchical clustering. In Hierarchical clustering, the final number of clusters is decided by the data scientist.
This is really a nice explanation about experience , but few things need to be added -Emphasis is more on algorithms and hyperparameter tuning, but what about data cleansing/Feature Engineering ..which consumes most of time in creating ML/DS project ..... Giving weightage for an interview candidate only on ML knowledge / hyperparameter tuning is bit a biased approach ... probably getting to know the problem solving capability of a candidate right from understanding business problem, sourcing data, data cleansing, model building,metric selection and interpretation capability would the key Points to consider in a potential candidate - Hyperparameter tuning has given more weightage ... But is there any set rule to use the combination of hyperparameter tuning..?? It's completely depends upon the data and it's patterns , need to try multiple approach , ML knowledge will help in identifying right hyperparamers ..so again emphasise put more on ML but not on problem solving skills - Also, while Interviewing a candidate , it's necessary to ask and understanding what value creation has provided to business either in terms of cost savings/reduced resolution time , ROI given the business problem -Model interpretability capability of a candidate should also be considered , which Will be easy to understand how good the candidate is good at solving/ communicating complex problems In short, Datascience / ML is not just a traditional IT job ... One has to be creative enough to solve business problem with the domain knowledge creative thinking and ofcourse coding , all together which suits the experience bracket Datascience/ML should be seen as automating the process of decision making with intelligence coming from data
firstly K Means and hierarchical cluster are unsupervised machine learning. in K means we are using for large data set most importantly by using "ELBOW CURVE" we are taking the no of clusters. but in k means we have 2 disadvantages 1. finding the no of clusters 2.finding the centroid. but in hierarchical cluster we easily finding the no of clusters by using "DENDROGRAM" ,in hierarchical cluster also some drawbacks are their the Dendrogram will no work for the large data set, most of the time we are using k means algorithm.
K-means clustering is basically used to group similar things like in image segmentation, grouping similar pixels. Grouping of similar customers so as to give any kind of deals or offers. Coming to Hierarchial clustering, it is used to when we need not to specify the number of clusters and can go deeper from one cluster to other get better result very easily.
I have 4 years and 5 months of experience working as a windows application developer, I want to transit to Data Science, So I have started learning , but after watching this Video I am having dicey feeling :(
Hi Krish, without going much into technical details of it, whenever we have the domain expertise which falls in sync with assumptions of k-means assumptions (assumptions like we should have k groups in the data , variance of all features are nearly equal) then it is advisable to go for k means. If we do not have much domain help and no reference to validate these assumptions we should go for hierarchical clustering. Also, on the processing side, more data we have , it will be difficult for hierarchical algorithms to process however we can process it in k means relatively smoothly.
are bhai bhai bhai... I am having an interview tomorrow, and this video is just so helpful. BTW I am feeling confident and scared after watching this. Thank You Krish bhai for the video.
i think we should concentrate on deep learning algorithms especially CNN rather than machine learning models,because they have very limited use they dont have any use in day to day problems ,according to me its only deep learning can have solutions to the problems !!!!
krishna i am planning to enter into data science side.If i finish all your videos from starting to till now. Is this knowledge is enough to secure a job. where i have 1.5 yr of experience in IT sector(Testing).
Hi sir, I'm Saurav Sir I have a question, I am a simple graduate person I did my Graduation in Arts (B.A) and I have learned a Data Science course from UDEMY online learning platform, so sir, can I get a job in Data Science, This is the most important question for me. Please guide me, I need Your help.
Sir I'm in Class 11 Commerce I love Statistics and Computer Science So what contents I should start to learn from now a little baby steps towards this career Plz specify!
Hi Krishna I follow your videos a lot, I am interested in learning data science, just one question kindly answer. I am basically a CSE professor working in a college, how can I transition from teaching to data scientist position and wheather my teaching experience counts, path to transit. Expecting your reply
You are basically transitioning from teaching to IT domain. So in It industry, Sorry, but your experience won't count. I'm saying this on the basis of what the HR will consider.
Hi Mahesh, I have worked as an Assistant Professor in Computer Science Department. Teaching experience doesn't count in IT industry. However if you have implemented some research based projects in data science and you have hands on the skills such as Machine Learning in python and your interview goes well example you know when to use which algorithm , you are good in stats etc there are good chances of you getting selected. Although I was into academics now I am a data scientist for a US based firm.
Hello Krish, You are doing a great service to the Aspiring Community. 9+yrs is more towards Principal Arch or Tech Evangelist. If one comes with Managerial experience then Arch and Technical stuff will be different ball game. Now that some of these want to get into ML world. would stiff say they should learn all about Architecture, Deployment, different media like Mobile devices ..... ? can you share your view please.
I'm a fresher and going to learn data science at pune institute so i have one confustion that...after completion of this course when i'm going for my placements through institute placemnets or generally in any company myself is apptitude test is taken by company or direct insterview is taken on the basis of data science knowledge for fresher?
K- means becomes handy when dataset is large whereas Hierarchial clustering inefficient in this regard because it becomes computationally expensive . Time complexity is very high in case of hierarchial clustering as each data point has been considered as one single cluster to start off and then based on the euclidian distance with other data points the membership of the big cluster has been obtained. Whereas in K means , based on the K, the membership of the cluster is obtained following the centroid movement . Time complexity_k-means : order of n o(n) , hierarchial : o(n2).
Yes correct answer 👏
A friend of mine suggested your channel and now I can see why. You are doing an amazing job! Please never stop. This is a community service kind of thing. I am trying to make a career shift and now I know where I am lagging behind. Thank you so much!
Hi,
K Means - Here, no of centroids are formed and number of optimal centroids is decided by WCSS(Within Cluster Sum of Squares), could be achieved by Elbow method.
Hierarchical Clustering - The goal is also same here, to find best features clustered together , technique being is DENDOGRAM. It is memory unit that stores that vectors which starts being Clustered together.
If we have large dataset, K-Means would be easy to be used since Optimal number of clusters could be shown by method itself whereas in Hierarchical clustering we are supposed to manually find optimal clusters through dendogram.
K-means is when we have unlabeled data i.e, data without defined categories or groups. And centroids of k clusters can be used to label new data
Hierarchical clustering : we use when we have a predetermined ordering from top to bottom it deals with small datasets where as k-means deals with large datasets
1.When there are huge number of points then Hierarchical clustering is avoided because of extremely large time complexity.
2. In the case of non globular shapes Min hierarchical clustering performs better than Knn clustering, as knn tends to cluster them in a more globular fashion.
3.typically when the clsusters are of different sizes or different densities Knn doesn't perform well.
4.each hierarchical clustering has its own adv and dis adv..Min hierarchical is sensitive to outliers.Max hierarchical though it is robust to noise, it tends to break down large clusters
krish sir make a series for just tricky interview question for 0-5 year people .thank
*Firstly both are Unsupervised Learning Technic where the Data set has only Features and No Target Values in it.
*In K-Mean Clustering first we will select the K value by using elbow method then we will initialize the Centroid randomly based on the K-value has taken points near to Centroid we will group and find the mean value and we keep changing the Centroid . and we will use Euclidean method to find the distance between the points.
*In Hierarchical clustering we will use same Euclidean method to find the distance between the points then we will group the points, then to find the number of clusters we will use hack then we will check through how many points it has passed if the Hack has passed to 2 lines then we take 2 clusters or if 3 lines means we will take 3 clusters.
* The major Difference is Hierarchical Clustering can't Handle Big Data Sets but K-Mean Cluttering Can Handle it.
We go for heircarchial clustering when less number of observation are there because in this we use dendogram,whera kmeans clustering (non-heircarchial ) we go when we have large number of observation.
The key to a successful Data Science solution is to treat every problem as an optimization problem. Which means, start with a basic model - try and improve it with hyperparameter tuning. Move to better models.
That's the key to success in data science field..!
When we have prior knowledge of the number of clusters we can form with our dataset we can use k-means clustering. It is most probable with low dimensional datasets. When we have no prior information about the number of clusters we will go with Hierarchical clustering. In Hierarchical clustering, the final number of clusters is decided by the data scientist.
This is really a nice explanation about experience , but few things need to be added
-Emphasis is more on algorithms and hyperparameter tuning, but what about data cleansing/Feature Engineering ..which consumes most of time in creating ML/DS project ..... Giving weightage for an interview candidate only on ML knowledge / hyperparameter tuning is bit a biased approach ... probably getting to know the problem solving capability of a candidate right from understanding business problem, sourcing data, data cleansing, model building,metric selection and interpretation capability would the key Points to consider in a potential candidate
- Hyperparameter tuning has given more weightage ... But is there any set rule to use the combination of hyperparameter tuning..?? It's completely depends upon the data and it's patterns , need to try multiple approach , ML knowledge will help in identifying right hyperparamers ..so again emphasise put more on ML but not on problem solving skills
- Also, while Interviewing a candidate , it's necessary to ask and understanding what value creation has provided to business either in terms of cost savings/reduced resolution time , ROI given the business problem
-Model interpretability capability of a candidate should also be considered , which Will be easy to understand how good the candidate is good at solving/ communicating complex problems
In short, Datascience / ML is not just a traditional IT job ... One has to be creative enough to solve business problem with the domain knowledge creative thinking and ofcourse coding , all together which suits the experience bracket
Datascience/ML should be seen as automating the process of decision making with intelligence coming from data
firstly K Means and hierarchical cluster are unsupervised machine learning. in K means we are using for large data set most importantly by using "ELBOW CURVE" we are taking the no of clusters. but in k means we have 2 disadvantages 1. finding the no of clusters 2.finding the centroid. but in hierarchical cluster we easily finding the no of clusters by using "DENDROGRAM" ,in hierarchical cluster also some drawbacks are their the Dendrogram will no work for the large data set, most of the time we are using k means algorithm.
In India people want to hire a data scientist who is more of a software engineer than a ML expert when in fact it should be opposite.
K-means clustering is basically used to group similar things like in image segmentation, grouping similar pixels. Grouping of similar customers so as to give any kind of deals or offers.
Coming to Hierarchial clustering, it is used to when we need not to specify the number of clusters and can go deeper from one cluster to other get better result very easily.
Making the complex things easier!!
Great video.Please include projects in your upcoming course that can be used for people transitioning to DS/ML role.
Your enthusiasm just wooow 😊
pls, make more tutorial on deployment of machine learning model
I will do silhouette analysis on both k-Means and hierarchical clustering whichever will give higher accuracy will go with that.
Namaskaar Sir, Good Talk, ...SANJEEV KUMAR
I have 4 years and 5 months of experience working as a windows application developer, I want to transit to Data Science, So I have started learning , but after watching this Video I am having dicey feeling :(
Sir without deep learning and machine learning can I enter into data analytics field?
you are genius @krish
Hi Krish, without going much into technical details of it, whenever we have the domain expertise which falls in sync with assumptions of k-means assumptions (assumptions like we should have k groups in the data , variance of all features are nearly equal) then it is advisable to go for k means. If we do not have much domain help and no reference to validate these assumptions we should go for hierarchical clustering. Also, on the processing side, more data we have , it will be difficult for hierarchical algorithms to process however we can process it in k means relatively smoothly.
When will you post the deploy models? Thanks!
are bhai bhai bhai...
I am having an interview tomorrow, and this video is just so helpful.
BTW I am feeling confident and scared after watching this.
Thank You Krish bhai for the video.
which company you have interview in?
You are great man, thank you for this share
Nice Video Krish.I learn lots of good things from your video.Please make a video on ML model deployment by using AWS and Azure.
hello krish
can you share your number of experince in data science
Sir Not Yet started Data Scientists Course How I Know the answer.Sir I Have 12+ years of gap.
i think we should concentrate on deep learning algorithms especially CNN rather than machine learning models,because they have very limited use they dont have any use in day to day problems ,according to me its only deep learning can have solutions to the problems !!!!
Waiting for next python video
Hi Sir! Can you suggest me some good universities in USA for MS in Data Science!!
Very nice Sir...your videos are very inspiring..
Valuable information thanks a lot
Thanks a lot Sir.
Hi krish, can you please tell me about jigsaw academy data science bootcamp? Shall I join?
sir tell about online course,date,fee,sylabus
Is there any WhatsApp group or website or something similar where we can ask doubt regarding Machine Learning? Please tell me
There are many telegram channels . join them
@@rahulrd2713 can you give me few telegram group links
Is Just ML Knowledge is sufficient for freshers... Without knowledge of computer vision and deep learning?
krishna i am planning to enter into data science side.If i finish all your videos from starting to till now. Is this knowledge is enough to secure a job. where i have 1.5 yr of experience in IT sector(Testing).
Naik when are you starting the classes?? I would like to enroll
what is data analyst work salery scope
Hi sir, I'm Saurav
Sir I have a question, I am a simple graduate person I did my Graduation in Arts (B.A) and I have learned a Data Science course from UDEMY online learning platform, so sir, can I get a job in Data Science, This is the most important question for me. Please guide me, I need Your help.
Is there a platform to study and understand different use cases regarding data science and ML algorithms? From learning and interview point of view.
what does a data science consultant do? and what kind of questions could i expect in the internship interview of tha same?
In India there are no jobs in data science field for Freshers.
Please ask next question series
your videos are good !!
Sir I'm in Class 11 Commerce I love Statistics and Computer Science So what contents I should start to learn from now a little baby steps towards this career Plz specify!
learn python,along with numpy, pandas, seaborn and matplotlib library codings with python...
Hi Krishna I follow your videos a lot, I am interested in learning data science, just one question kindly answer.
I am basically a CSE professor working in a college, how can I transition from teaching to data scientist position and wheather my teaching experience counts, path to transit. Expecting your reply
You are basically transitioning from teaching to IT domain.
So in It industry, Sorry, but your experience won't count.
I'm saying this on the basis of what the HR will consider.
Hi Mahesh, I have worked as an Assistant Professor in Computer Science Department. Teaching experience doesn't count in IT industry. However if you have implemented some research based projects in data science and you have hands on the skills such as Machine Learning in python and your interview goes well example you know when to use which algorithm , you are good in stats etc there are good chances of you getting selected.
Although I was into academics now I am a data scientist for a US based firm.
What is the difference between standardization and normalization when to use it
When applying ML algo. that exploit gaussian distribution data, we can use standardisation, otherwise normalisation.
Hello Krish,
You are doing a great service to the Aspiring Community.
9+yrs is more towards Principal Arch or Tech Evangelist. If one comes with Managerial experience then Arch and Technical stuff will be different ball game. Now that some of these want to get into ML world. would stiff say they should learn all about Architecture, Deployment, different media like Mobile devices ..... ? can you share your view please.
I will do silhouette analysis on both k-Means and hierarchical clustering whichever will give higher accuracy will go with that.
Which institute is best for data science in Bangalore with placements
Freshers how to prepare interview data science
I'm a fresher and going to learn data science at pune institute so i have one confustion that...after completion of this course when i'm going for my placements through institute placemnets or generally in any company myself is apptitude test is taken by company or direct insterview is taken on the basis of data science knowledge for fresher?
For freshers, aptitude and dsa is must
Iam fresher can I get into data science
How many round
Raise Your Hand If you came here after taking live stream class of Data Science Module :) (y)
Which is your favourite Machine Learning Algorithm? Comment below
Random Forest
I just started so is linear regression
Logistics regression
cnn ,multiivariate
Random Forest, artificial NEURAL NETWORKS