Silhouette Score for clustering Explained | Silhouette (clustering)- Validating Clustering Models

Silhouette Score for clustering Explained | Silhouette (clustering)- Validating Clustering Models
#SilhouetteScore #UnfoldDataScience
Hello ,
My name is Aman and I am a Data Scientist.
About this video:
In this video, I speak about Silhouette Score and explain step by step how Silhouette Score works for cluster validation. I explain how to validate clusters and how to measure goodness of clusters. I explain the mathematical formula of Silhouette Score and intuition behind it. Below points are discussed in this video:
1. Silhouette Score for clustering
2. Validation on K-means clusters
3. Cluster validation techniques
4. How to measure goodness of clusters
5. Unsupervised machine learning accuracy
About Unfold Data science: This channel is to help people understand basics of data science through simple examples in easy way. Anybody without having prior knowledge of computer programming or statistics or machine learning and artificial intelligence can get an understanding of data science at high level through this channel. The videos uploaded will not be very technical in nature and hence it can be easily grasped by viewers from different background as well.
If you need Data Science training from scratch . Please fill this form (Please Note: Training is chargeable)
docs.google.co...
Book recommendation for Data Science:
Category 1 - Must Read For Every Data Scientist:
The Elements of Statistical Learning by Trevor Hastie - amzn.to/37wMo9H
Python Data Science Handbook - amzn.to/31UCScm
Business Statistics By Ken Black - amzn.to/2LObAA5
Hands-On Machine Learning with Scikit Learn, Keras, and TensorFlow by Aurelien Geron - amzn.to/3gV8sO9
Ctaegory 2 - Overall Data Science:
The Art of Data Science By Roger D. Peng - amzn.to/2KD75aD
Predictive Analytics By By Eric Siegel - amzn.to/3nsQftV
Data Science for Business By Foster Provost - amzn.to/3ajN8QZ
Category 3 - Statistics and Mathematics:
Naked Statistics By Charles Wheelan - amzn.to/3gXLdmp
Practical Statistics for Data Scientist By Peter Bruce - amzn.to/37wL9Y5
Category 4 - Machine Learning:
Introduction to machine learning by Andreas C Muller - amzn.to/3oZ3X7T
The Hundred Page Machine Learning Book by Andriy Burkov - amzn.to/3pdqCxJ
Category 5 - Programming:
The Pragmatic Programmer by David Thomas - amzn.to/2WqWXVj
Clean Code by Robert C. Martin - amzn.to/3oYOdlt
My Studio Setup:
My Camera : amzn.to/3mwXI9I
My Mic : amzn.to/34phfD0
My Tripod : amzn.to/3r4HeJA
My Ring Light : amzn.to/3gZz00F
Join Facebook group :
www.facebook.c...
Follow on medium : / amanrai77
Follow on quora: www.quora.com/...
Follow on twitter : @unfoldds
Get connected on LinkedIn : / aman-kumar-b4881440
Follow on Instagram : unfolddatascience
Watch Introduction to Data Science full playlist here : • Data Science In 15 Min...
Watch python for data science playlist here:
• Python Basics For Data...
Watch statistics and mathematics playlist here :
• Measures of Central Te...
Watch End to End Implementation of a simple machine learning model in Python here:
• How Does Machine Learn...
Learn Ensemble Model, Bagging and Boosting here:
• Introduction to Ensemb...
Build Career in Data Science Playlist:
• Channel updates - Unfo...
Artificial Neural Network and Deep Learning Playlist:
• Intuition behind neura...
Natural langugae Processing playlist:
• Natural Language Proce...
Understanding and building recommendation system:
• Recommendation System ...
Access all my codes here:
drive.google.c...
Have a different question for me? Ask me here : docs.google.co...
My Music: www.bensound.c...

[@shaneglean217]

The explanation is so concise and simple, even in spite of the fact that english isn't your 1st language. Amazing work my dude.

[@ankitsapkota3713]

Never expected the conceptual delivery of this level without random steps out of the air in this accent.

[@isaacyekoniya8067]

This is simple and well explained, thank you Aman!

[@UnfoldDataScience]

Most welcome!

[@RoopaNandu-k8b]

Aman, your explanation is really good. Very precise and in detail

[@awd_junkie]

Clear and right to the point! Thank you for your work! Much appreciated!

[@UnfoldDataScience]

Thanks for watching Chris. Stay safe. Tc

[@JuliaCh47]

Thank you so much, best explanation I've seen on the Internet!

[@UnfoldDataScience]

Thanks a lot for watching.

[@aaryalad7775]

I'm glad i found a teacher like you

[@Pannafreestyle]

Sir you are the best teacher for DS ! Keep up the great work

[@Anon11229]

Too good...Way better than online courses that take thousands and thousands of money
keep up the hard work

[@UnfoldDataScience]

Thanks Zaid.

[@KHakimin]

This is gold! thanks for putting it very simple to understand

[@UnfoldDataScience]

Glad it was helpful Khairul.

[@ravisavaliya4333]

at first it was very difficult for me to understand this, but you explained this in a very easy way....THANKYOU AMAN SIR FOR YOUR EASY AND INTUITIVE EXPLANATION!!!

[@UnfoldDataScience]

It's my pleasure

[@kushik.naveen]

This is so useful for understanding & revising. Amazing work! Thankyou!

[@nandankakadiya1494]

Amazing explaination thank you sir

[@UnfoldDataScience]

Welcome Nandan.

[@lhadz7290]

Thank you so much for Sir, I am so glad I came across your video. I look forward to learn more from you. Thank you again.

[@UnfoldDataScience]

So nice of you

[@YouTribalChief]

What a gem of an explanation you give bhai! Truly you make things so simple and understandable!
You deserve more!

[@UnfoldDataScience]

Thanks a lot. Pls share with friends as well. Have a nice weekend ahead.

[@izb1275]

Great explanation !

[@blackyogurt]

Great Explanation . thanks

[@victorcardenas9621]

Simple but powerful, thanks and greetings from Colombia!

[@sanjithaamarathunga9949]

Thank you! Simple and well explained!

[@UnfoldDataScience]

Glad it was helpful!

[@user-hs6gz4yn1u]

Ur Explanation is simple and understandable,TQ sir

[@RezoanurRahman]

You explained it so well even a 5th grader would get it. Its so easily explained!.

[@bangarrajumuppidu8354]

excellent

[@UnfoldDataScience]

Thanks a lot.

[@shaheerrehman3151]

Thank you. Much love to you from Pakistan

[@UnfoldDataScience]

Thanks Shaheer, pls share with friends if you like the content.

[@kamal_douma]

very very good explnation

[@liqs_xd]

Great video, thanks for your time!

[@UnfoldDataScience]

My pleasure!

[@user-dd3mf2sp7s]

Great explanation, Thank you!

[@UnfoldDataScience]

Welcome

[@anshuraghav5620]

Great video

[@vallimuthaiyah5098]

Thank you sir for very clear explanation with example.. 👍

[@UnfoldDataScience]

Welcome Valli :)

[@MyerNore]

Clearer than my professor, for sure

[@shaikhuzma786]

Tqsm sir so well explained ❤

[@nandinik7816]

Always to the point...and you dont waste any time actually...

[@UnfoldDataScience]

Thanks again.

[@nicolaevasiliu9430]

Amazing explanations! THANK YOU !!!
You are an excellent teacher !!!

[@stevemungai3542]

Crystal clear

[@omniscienceisdead8837]

you are the best bro!

[@mdyounusahamed6668]

very concise and simple explanation.

[@sourabhpramanik3957]

Its really a simple and good explanation for anybody to understand...thanks for making this simple explanation video.

[@UnfoldDataScience]

Welcome Sourabh

[@Felipe90820]

Amazing explanation! Congrats and thank you!

[@samiayasmin5782]

Thank you so much. Simple and clean explanation.

[@anisadillap.7131]

oke i understand, thankyou sir ...

[@UnfoldDataScience]

Welcome

[@Krishna-pm8ty]

Excellent Amen. Perfect explanation👏👏👏

[@parvsharma8767]

Thanks bro for explaining so perfectly

[@UnfoldDataScience]

Welcome Parv.

[@PranavVashisth]

verry well explained !!!!

[@UnfoldDataScience]

Thanks Pranav.

[@MohdRashid-hi7cp]

Very nice explanation

[@UnfoldDataScience]

Thanks Rashid.pls share with friends also

[@paulholzapfel1856]

Perfect explanation. Thank you Aman

[@UnfoldDataScience]

Welcome Paul. Your feedback is precious to me.

[@puodzius]

Very good explanation, thanks

[@UnfoldDataScience]

Welcome.

[@beprepare8128]

Very well explained sir

[@UnfoldDataScience]

Thanks and welcome

[@sajjaddehghani3280]

Thanks for the very simple and easy to understand explanation

[@UnfoldDataScience]

Welcome

[@keinestudiere-chillzone633]

Straight to the point. Thank you

[@UnfoldDataScience]

Thanks a lot for watching

[@uchennanwosu5327]

Excellent illustration and explanation.

[@chrisx742]

Thank you.

[@sandipansarkar9211]

finished watching

[@theone-jz2rf]

keep up the good work!!!

[@UnfoldDataScience]

Thank you.

[@dbapple]

thank you, straight to the point!

[@the.quantum.paradox]

Very simple and easy to understand!

[@Suno-99]

beautiful explanation, keep it up

[@sangeethaagoudar2175]

Thank You so much for this video sir

[@dosendaring]

This is very good explanation, thank you Sir....

[@UnfoldDataScience]

Welcome Yudy.

[@SACHINKUMAR-px8kq]

Thanks you so much sir

[@UnfoldDataScience]

Welcome Sachin.

[@mmmommm237]

Great, ty!

[@venkataprashanth9831]

great and simple explanation, a big clap to ur efforts

[@UnfoldDataScience]

Thanks Venkat.

[@veyselaytekin8734]

thank you

[@dr.zliiiiii]

nice video

[@diyap7343]

This is amazing!

[@UnfoldDataScience]

Thanks Diya.

[@legolife3012]

Thanks a lot for this simple explenation

[@UnfoldDataScience]

You are welcome

[@almonddonut1818]

Thank you!!

[@UnfoldDataScience]

Welcome.

[@muhammedthayyib9202]

Clearly explained!

[@prashanths4455]

super bro

[@UnfoldDataScience]

Thanks Prashanth

[@optimalsolutionprogramming1299]

Thank you for your help. Really great explanation!

[@UnfoldDataScience]

Welcome

[@akashpb4044]

Awesome ...plain and simple 👍🏼

[@UnfoldDataScience]

Glad to hear that Akash 🙂, please share with others as well who could be benefited from such content.

[@kavyasharma4738]

how can i thank you brother . you explained this topic so easily.

[@user-ur2en1zq4f]

thanks sir

[@UnfoldDataScience]

Most welcome

[@SurajReddy-hx2dx]

concept is well explained . But what do we do after this step. You have only calculated the silhouette score for one point. Do we need to calculate for all the points? What is the range of values present for silhouette score? When is it good or bad?

[@mohe4ever514]

Very well explained!

[@UnfoldDataScience]

Glad it was helpful!

[@gnaneshgn8341]

Nice video Sir thank you. please make an video of EDA sir ..!

[@UnfoldDataScience]

Thanks Ganesh.

[@bipulnath4602]

Amazing explaination..plz make a video on Dunn index.

[@UnfoldDataScience]

Thanks Bipul.

[@christopherlegarda5164]

This guy is god!

[@vishwass5103]

Hi Aman thanks for this explanation. Please explain about assumptions of k means clustering

[@UnfoldDataScience]

Assumption at high level is, you data has clusters and centriods.

[@salilshukla08]

thanks

[@UnfoldDataScience]

Welcome.

[@ranajaydas8906]

Sir please make a video on PCA . It would be very helpful

[@UnfoldDataScience]

Sure Ranajay.

[@subhasisdutta3711]

you have said "Min distance of A1,A2,A3 is the value of b ". Here you are talking about mean value of three or the which one has the minimum value among three , we'll chose it. Please clear it once, if possible

[@UnfoldDataScience]

HI Subhasis, A1, A2 etc are "mean" Distance from various other clusters ok. Now let's take att these A1 TO AN values and take the minimum from this set. This mimum is b.

[@subhasisdutta3711]

@@UnfoldDataScience thanq for the clarification. Have a blessed life ahead

[@neajinnovations8587]

I got all the concept clear, but please when is it appropriate to use silhouette distance? it when you have mixed data? or mix data is best for elbow method?

[@iftikhar58]

clear

[@UnfoldDataScience]

Thank you

[@jeanhwang18]

Thank you so much for your explaination. I have a small question: after I got point i ' s silhouette coefficient, I calculate other two points in cluster A, then I take the average of the three silhouette coefficients, let's say it's NO.1. What we can learn from No.1? If No.1 is close to +1, it's good, it means data points in cluster A are most similar to each other, right?

[@mahipatil369]

hello sir.. great video i understand this topic very well,, thank you so much...
just want to ask did you make video of implementation of silhoutte score in python?? if yes plz provide link..

[@UnfoldDataScience]

Thanks a lot. Its very simple to do in python - see this link - stackoverflow.com/questions/59919627/how-to-calculate-the-silhouette-score-for-each-cluster-separately-in-python

[@Live_life_with_no_regrets]

is there any way to run it in SPSS? Is it correct if I use the average of the Euclidean distance of the cluster?

[@sampathvinaykumarreddymajj790]

Topic is very well explained sir ..
But should we learn DB Index, Dunn Index, Jaccard score ... or only this Silhouette score is enough

[@UnfoldDataScience]

Thanks Sampath.

[@megalaramu]

Hi Aman, I do have a general question in clustering -I have heard of people do clustering in the dataset initially and then train separate model for each cluster. my question is lets take an example of telecom churn - there will be many services like internet, Phone, Dish etc. is there anyway we can cluster like internet users in one cluster, phone service in one cluster like that.

[@parikshitgurjar5545]

hello sir, Plese make a video on the explanation and implementation of ppf,pdf,cdf in python with simple explanation and understanding, I am very confused in this.

[@UnfoldDataScience]

pdf cdf video is available Parikshit. Please search in my channel.

[@parikshitgurjar5545]

@@UnfoldDataScience Thankyou sir❤️

[@himanshumangoli6708]

As we have find cohesion and separation for only one point in one cluster A, so we have to find Silhouette score for all points for all points in same cluster so just to ensure that points are correctly clustered ??
Please explain??

[@UnfoldDataScience]

Yes , one point is just for example.

[@pruthvi491]

How to get silhouette score for spectral clustering

[@UnfoldDataScience]

I Will check.

[@handsomemehdi3445]

Thank u for simpler explanation! A question: In our case, it seems that for both clusters, b should be the same unless we have more than two clusters, (or since those clusters are mutually nearest) doesn't it?

[@lkjhlkjh1954]

I think that’s why we do not calculate silhouettes score for less than 2 cluster

[@TheGentlemanGR]

So which score is better? -0.2 or 0.21 ?

[@arisurya9867]

Nice video sir. But i have problem when calculate cohesion in cluster just has one data point. How i should set value of cohesion? 0 or 1?

[@UnfoldDataScience]

Thats not a good clustering process, clusters need to be reformed.

[@arisurya9867]

Thank you sir 🙏

[@gretchenbekker2371]

How do I get the amount of data points within the clusters ?

[@UnfoldDataScience]

Levels can be found out after clustering then we can count using pandas.

[@abeerhamid]

How can i read the graph of silhouette score? Most important thing you didn't tell

[@UnfoldDataScience]

Positive Scores (close to 1): Samples are well-clustered, and there is good separation between clusters.
Scores around 0: Samples are on or near the boundary between clusters, indicating potential overlap.
Negative Scores: Samples are likely in the wrong cluster.

[@nerdymath6]

is 0.4 is good silhouette score?

[@UnfoldDataScience]

Dont think so

[@dariomelconian9502]

Is 0.39 a bad score?

[@UnfoldDataScience]

can be better

[@sameeransari9129]

apki english samajhne me time lg jata hai.. ki aap kya bolna chahte ho

[@sameeransari9129]

Sir aap hindi me samjhaya karo toh zyada aacha samjha payoge...

[@sameeransari9129]

apki english me confuse ho jate hai log ki aap kya bolna chahte ho...isko negitive comment ki tarah mt lena...aap samjhate aacha ho pr agar hindi me samjhaoge toh hm logo ko clear samaj ayega