StatQuest: t-SNE, Clearly Explained
ฝัง
- เผยแพร่เมื่อ 16 มิ.ย. 2024
- t-SNE is a popular method for making an easy to read graph from a complex dataset, but not many people know how it works. Here's the inside scoop.
Here’s how to create a t-SNE graph in R (this is copied from the help file for Rtsne)…
library("Rtsne")
iris_unique <- unique(iris) # Remove duplicates
iris_matrix <- as.matrix(iris_unique[,1:4])
set.seed(42) # Set a seed if you want reproducible results
tsne_out <- Rtsne(iris_matrix) # Run TSNE
Show the objects in the 2D tsne representation
plot(tsne_out$Y,col=iris_unique$Species)
This StatQuest is based on the original t-SNE manuscript, and it's not super hard to read (especially if you understand the general idea of how it works): lvdmaaten.github.io/publicati...
For a complete index of all the StatQuest videos, check out:
statquest.org/video-index/
If you'd like to support StatQuest, please consider...
Buying The StatQuest Illustrated Guide to Machine Learning!!!
PDF - statquest.gumroad.com/l/wvtmc
Paperback - www.amazon.com/dp/B09ZCKR4H6
Kindle eBook - www.amazon.com/dp/B09ZG79HXC
Patreon: / statquest
...or...
TH-cam Membership: / @statquest
...a cool StatQuest t-shirt or sweatshirt:
shop.spreadshirt.com/statques...
...buying one or two of my songs (or go large and get a whole album!)
joshuastarmer.bandcamp.com/
...or just donating to StatQuest!
www.paypal.me/statquest
Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
/ joshuastarmer
0:00 Awesome song and introduction
1:19 Overview of what t-SNE does
2:24 Overview of how t-SNE works
4:12 Step 1: Determine high-dimensional similarities
9:26 Step 2: Determine low-dimensional similarities
10:33 Step 3: Move points in low-d
11:05 Why the t-distribution is used instead of the normal distribution
Corrections:
6:17 I should have said that the blue points have twice the density of the purple points.
7:08 There should be a 0.05 in the denominator, not a 0.5.
#statquest #tsne
Corrections:
6:17 I should have said that the blue points have twice the density of the purple points.
7:08 There should be a 0.05 in the denominator, not a 0.5.
Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/
Thanks very much for the informative lecture and it is really helpful. UMAP is more and more popular now, could you explain it and compare with tSNE as well? Thanks in advance.
@@linweitao6470 I should have a UMAP StatQuest ready in a few weeks. I'm working on it right now.
@@statquest Thanks again!
@@statquest UMAP is great, I dont know if it is more popular. There are more stringent reductions out there like ICA. I wonder the thoughts of Josh about it?
@@CompBioQuest I guess it largely depends on the field. Right now, genetics and molecular biology are going bonkers over UMAP. However, ICA is very interesting. Thanks to your question, I found this article which is fascinating: gael-varoquaux.info/science/ica_vs_pca.html
I am always blown away by how you make statistics & machine learning algorithms so simple to understand and how you graciously share your knowldege. Keep up the great work man, you are awesome!
Thank you very much! :)
Whenever I find statistics technique I have never seen in scientific article, I always visit your channel. Thanks a lot!!
Happy to help! :)
Fantastic video. I really appreciate all the slides that you made to get the animation effect. It really helped. Possibly the best explanation of t-SNE around. Keep up the good work.
Josh is so far my favorite TH-camr that is able to explain complex stats concepts so smoothly.
Thank you so much! :)
As entertaining as watching a Walt t-SNE movie!
You made me laugh out loud! BAM! :)
Best stat-word-play of the year! 😂
"This is Josh Starmer, and you're watching Tisney Channel!"
Triple BAM! :)
I regret I can't put 1000 likes! I read about 20 articles about t-SNE, they are similar to one another, almost identical - and they don't get me closer to the point. But your video - I watched it 4 times (because the topic is hard, at least for me) with making some and drawing - but finally I understand how it works, up to the point that I can explain it to someone else. So many thanks to you!
HOORAY!!! TRIPLE BAM! I'm glad the video was helpful. BAM! :)
I never leave comments, but I really feel the need to thank you for being able to explain this in such a simple way
Thank you! :)
Josh.. Your explanation is always "simple and easy to understand" even for layman.You are simply "The life Saviour" !!!
Thank you so much :)
Hooray! I'm glad my video was helpful. :)
I never knew machine learning could be as simple as... BAM
Thats like the most important lesson.
Double bam 💥
Just a random comment so that someone can say triple bam
Triple bam 💥
hurayyyy we have made it to the END !!!
I really can't appreciate you enough for your videos.
Books and blogs only make sense after I watch your videos!
Thank you very much! :)
Excellently explained! I really like your simple, clear, concise explanation - those 3 factors make a world of difference. And, great animations.
Awesome, thank you!
It's impressive how you managed to explain the essential concepts of this chain of algorithms in such a clear way! I'm sharing this video with my beginner fellows, who normally flee as soon as I say words like nearest-neighbor or stochastic.
Thank you very much!
Thank you very much! :)
🤣🤣🤣🤣it's that terrifying?!? Barbara Oakley in her book, "a mind for numbers" called them zombies🤣🤣🤣
Awesome explanation, thank you so much! I read a few papers/books multiple times and barely have a clue, but with your vid I understand the concept just by watching it once!
Great explanations! Can you please do a video explaining UMAP and potentially how it compares to t-SNE? Thanks!
+1
+1
+1
+1
+1
Very clearly explained!
Loved the way you explained such a complicated concept so intuitively.
Thank you.
Glad it was helpful!
You make a complex idea becomes so simple and understanding ! Great video. Thanks a lot
Thanks for such a clear explanation. You know, your channel already in the top list for me and very soon I'll watch all your videos..
Great as always. I've heard of t-SNE before, but this was my first real introduction to it. Definitely want to go look at some more resources now.
Josh, i literally love your videos, they are really helping me get through my ADV CS degree. I am going to buy one of your shirts, and wear it on campus as a thank you!
That would be awesome!!! Thank you very much! :)
Thanks a lot. I really struggled to understand the concept first time I came across it in a book. Your video helped a lot. Great job!
I was so confusing about t-SNE until I watched this. It's clear and very easy to understand! Thank you! Like your BAM. :D
BAM! :)
This is the best video for t-SNE that I have ever seen. Thanks a lot, man
I just love the way you start all your videos! Stat-Questtttttt :)
BAM! :)
this is such an awesome explanation of tsne that i dont need to watch any other video or read any other website/book. I dont think there can be a better explanation. Superlike.
Very nice way of teaching ! ML concepts CLEARLY EXPLAINED and BAM adds lot of curiosity in the videos :) Thanks for your videos. And not to forget your songs are really nice :)
Thank you!
Very well explained ! Your video was recommended to us by our professors at ETH-Zürich.:)
It's rare to come across such a brilliant explanation.
Thank you! :)
I'm writing this comment while having watched only half way into this video, which is pretty unusual for me!
It is so clearly explained! I once glanced at the t-SNE paper and didn't understand it. If this is what it does then this is how things like this should be explained!
Really, we need people explaining science like this! It's possible to read scientific papers, but what they fail to do is properly communicate the core idea to the reader so that the reader quickly grasps the big picture and the intent of the mathematical details without getting lost in the details.
Frequently, even a missing definition can make reading papers much harder for non experts.
I'm glad you liked this video so much! :)
The Best tutorial and explanation for TSNE so far! It's of great help! Thanks a lot!
Thanks! :)
Super Mega BAM !! So great at what you do as always ... Tons of love sent your way ! Keep up the amazing work :D
Thanks so much!!
The only educational channel which brings a smile to my face.
bam!
Thanks a lot!! These videos are much more clear than any article!
A video explaining UMAP (related to t-SNE) would be awesome !
I'm working on UMAP. For now, however, know that it is almost 100% the same as t-SNE. The differences are very subtle.
Love the vid. I was wondering how tsne works and you broke it down great and the explanation for the t distribution was short and to the point.
Thank you! :)
Why I couldn't stop bamming the like button??!! You're the best Josh!!
Thanks!
You are incredible, Josh Starmer!! I loved this
Thank you! :)
Incredibly helpful and well presented. Thank you.
Well done! I would love to see videos on handling data outliers for regressions. Thanks!
Hello Josh, thank you for coming with such incredible videos. Data scientist’s life becomes easy.😬
Thank you! :)
StatQuest with Josh Starmer Hi a request to do a tutorial of UMAP.
excellent explanation , this intuition helps to follow maths behind t-SNE
Came here for understanding the t-SNE plots used in single cell transcriptomics - which I finally did, thanks! Overall, you helped me out already plenty of times!
To display cells in during cell fate transition/acquisition e.g. different time points during neurodevelopment, often pseudo-temporal ordering is used.
Since scRNA seq is becoming more and more popular, this might be a good next topic
Same here, and I did not expect to understand so fast and clearly!
VERY CLEAR EXPLANATIONS :) THANK YOU FOR ALL YOUR VIDEOS
you are the hero, keep explaining complex thing into simple. thankss
Thank you! :)
Dude this is super clear. Love the content! BAM
Thank you very much! :)
I am a student in Japan.
I'm not good at English, but it was very easy to understand and I learned a lot:)
Awesome! :)
Hi Josh, I can't thank you enough for how much I have benefitted from your videos even though I do data science as part of my day job. Thank you so much for sharing your knowledge!
One request for a video: could you do a video of when to use which methods / models in a typical data science problem? Much appreciated.
That's a great idea.
Thank you very much Josh . You made it easier to understand.
Hooray! I'm glad the video was helpful! :)
Hey, love your videos!
Just a typo but it should be 0.05 on the values to the right at 07:19. Confused me for a second so might clear things up for others.
Nice explanation of t-SNE for beginners.
Subscribed because that intro gave me life!
Ha!!! Thanks! :)
your explanation is very very good! thanks!!!
i am a huge fan of this channel! greetings from brazil ^^
Muito obrigado! :)
Brilliant explanation, this has been bugging me all day, thank you!!
Glad it helped!
Hi Josh, quality content! This channel continuously helps me to understand the idea behind so that the dry textbook explanations actually make sense. I still have a question. When you calculate the unscaled similarity score, how do you exactly determine the width of your guassian? I get it in the example that we already know the cluster. If I only want to visualize the data without having pre-defined clusters, what happens then?
I talk more about the details of t-SNE and how it works in my videos on UMAP: th-cam.com/video/eN0wFzBA4Sc/w-d-xo.html and th-cam.com/video/jth4kEvJ3P8/w-d-xo.html
Excellent video! Perhaps you could add another video where you go through the actual algorithm and how the moves is actually computed.
yes!!! pleasee!!
Just hear about t-SNE and I did not quite understand how it works so I crossed my fingers hoping that josh did a video of this and of course he did!! haha
I have my popcorn ready to enjoy this video :)
Worth it!
BAM! :)
Amazing explanation! Thank you!
Thank you so much! Right now everyone in our department (Systems Genetics at NYU Langone) is using UMAP. There aren't many great videos about it - it would be awesome if you could help us understand what all the hype is about!
UMAP is on the to-do list. I hope to get to it in the spring.
Kudos, I understood so effortlessly....tripple BAM!!!
Thanks! :)
thank you so much for this nice explanation. will help me a lot in my exams
Glad to hear that!
Amazing work! perfectly explained!!!
Thanks a lot!
Great video - thank you! One small insertion that I think would improve it: at ~2:07, right after showing what projecting on to the X or Y axis would look like, show one more example of projecting onto an arbitrary line to try to retain as much variance as possible (basically PCA). I think this could be done in 15-20 seconds, and would be helpful in comparing t-SNE to one of its most popular alternatives, which is helpful in deciding *when* to use an algorithm - one of the hardest things for beginners like myself.
Thanks for the tip!
OH God, this is a great explanation, as Radel mention below, it would be nice to have an extended video of the algorithm as the one from PCA!!
Thank you! Yes, one day I'll break the actual equations down and do "step-by-step" explanation of t-SNE.
Looking forward to this.
Very clear explanations, thanks a lot!
Great videos! Great channel! Big thumbs UP!
Big thanks!
Fantastic explanation and comments. Thanks so much!
Thank you!! I'm glad you like the video. :)
Your speak like Kevin from The Office. Great explanation, thanks a lot:)
Great Video. Thanks StatQuest! One question, in your computation of the similarity scores, you have taken a normal distribution etc. But would be correct if the computation is done through Euclidean distance or some other distance formulae?
The distance between any two vectors can be calculated by taking the length/magnitude/euclidian norm of the difference of two vectors. This is the same as the distance formula in R^2. For example, here are 2 points (1, 3) and (5,2). The difference in the x axis is -4 and in the y axis 1. The distance is the square root of the sum of squares; sqrt((-4)^2+1^2)=sqrt(17). This can be generalized to R^n by defining two R^n vectors a and b. distance = ||a - b|| = ||b - a|| = sqrt( (a_1 - b_1)^2 + … + (a_n - b_n)^2 )
Note that when these distances are plotted on a normal distribution. There won’t be a value which would be lower than the mean since the distance is the absolute value of the relative distance between two points. This results in only points to the right of the mean and having points in different clusters not close to each other falling close to each other on the number line since their absolute distances are the same. A signed distance can be computed by doing a - b or b - a without squaring the differences nor taking its square root.
The minor difference of getting a negative or positive signed “distance” doesn’t change how the distance will be used apart of t-SNE. This is because the points being plotted on a number line is ultimately a means to getting the likelihood of the data point when a fixed point is at the center. The normal distribution is symmetrical to the left and right of this mean, so the y-axis or likelihood value will be the same regardless of whether a point is -1 or 1 units away from the mean.
Awesome! You've done a lot of videos on dimensionality reduction, and I'm wondering if there is any criterion for which dimensionalty reduction algorithm to use?
My initial guess would be performance differences between each of them on huge amount of data. tSNE is not super efficient. Takes a while to run. my theory would be PCA is faster.
Super clear. Is the small move carresponding to some learning rate multiplying the gradient of some distance between the expected distance matrix and the one we have?
I believe so.
A great one! Easy to follow easy to understand ❤️ i love the BAM ahahahahah
Thank you!!!! :)
hey Josh! great video as always. Is it necessary to normalize or scale the data before applying this algorithm?
I'm not sure. In theory, no, but in practice, PCA is usually used as a first pass to remove noise, and PCA requires things to be on the same scale.
Great explanation! Congratulations!
Hi John! Your videos are amazing! Could you please explain SPADE and SOM algorithms? Thanks!
I never thought I'd not understand a statquest video! :(
Bummer. What time point was confusing?
"Bam, I made that terminology up" :D :D , great vid, keep up the good work.
Thanks! 😁
Wish I could *Triple Bam* like this video! Such a simple explanation. Thanks a lot Josh :-)
Glad you liked it!
Thanks a lot for your videos! The explanation is simple and clear. I was wondering whether you could do a similar video for Topological Data Analysis? Thank you :)
Thank you Joshua for this amazing video! I just want to make sure I understand, so basically t-SNE determines the clusters in the high dimensional data then takes it "randomly" to the lower dimensional data and it tries to follow the patterns made in the high dimensional data. The only thing I was a bit confused at the end is what does the algorithm do at each step that it moves the point closer to the other clusters? (where you show the two matrix) The matrix to the left was all mixed up and the one to the right was organized due to the scaling scores we previously did. So how does the matrix to the left (lower dimension(1-D)) learns from the matrix to the right (higher dimension (2-D))? I hope you understood me but anyways Joshua best video on t-SNE in youtube. Thanks for sharing!
Thanks Joshua! Now it makes perfect sense to me thanks for giving the explanation. I really find this algorithm useful for finding clusters and I just find fascinating how it determines groups of clusters. Again, thanks for your quick response to my question!
Thanks for the videos Joshua. Could also explain - Manifold learning and Markov models in your videos? Thanks
Great video! This video helped me a lot
Awesome explanation. Thank you, sir. Quick question - if two clusters are placed close to each other does that mean they are similar than a cluster place far away from it?
That's the idea.
Hi Josh! Thank you for this video! It makes so much more sense now. But I'm still confused why the normal distribution was used in the 2D graph, but a t-distribution for the line? How do you know when to use which?
That's just the way it is and it seems to work.
One word reaction after watching this video --> AWESOME!!
Thank you so much 😀!
Thank you Josh . I love the way you present concepts with simple examples.
Could you please explain how you decided the red dot directions to the left, where as the orange on right side @5:30 ?
It doesn't matter what side of the curve the points are on, since the distance from the y-axis values on the curve will be the same (normal curves are symmetrical). However, in order for the points to be easily seen, I spread them out on different sides rather than piling them all up on top of each other.
@@statquest Thank you again
Hey, love your videos! We are actually using it to help explain key concepts in our application-focused courses. I'd love to see UMAP (similar to t-SNE), which is a bit more scalable.
Thank you so much! It's on the to-do list. :)
@@statquest Awesome! I'm using your content in my courses - Students love it. PCA, K-Means, & t-SNE. Will be using your ML videos as well. Your explanations are the best!
Hi @StatQuest with Josh Starmer, great video!
The thing I am missing is what is happening in each of this steps to move each point. What are the 'attract' and 'repel' real values and how they are use to make the Similarity Matrices closer each of these steps?
This is a good question. The actual math is a little too messy to put in this comment, however, the idea is that the matrices are made similar using Gradient Descent, and that's where the attractions and repulsions come in. Here's a quote from the original paper (the link to the paper comes after the quote):
Physically, the gradient [ minimized by gradient descent ] may be interpreted as the resultant force created by a set of springs between the [low dimensional point A] yi and all other [low dimensional points] yj. All springs exert a force along the direction (yi −y j). The spring between yi and y j repels or attracts the map points depending on whether the distance between the two in the map is too small or too large to represent the similarities between the two high-dimensional datapoints. The force exerted by the spring between yi and y j
is proportional to its length, and also proportional to its stiffness, which is the mismatch (p j|i −qj|i + pi| j −qi| j) between the pairwise similarities of the data points and the map points.
Here's the link to the paper: www.jmlr.org/papers/volume9/vandermaaten08a/vandermaaten08a.pdf
Thanks really great videos understood concepts so well
Glad it was helpful!
John Starmer,
This video impressed a lot.
I would like to know how to ensure whether all similar points are grouped perfectly or any spill out by mistake.
Kindly suggest
You ask a good question and I'm not sure I know the answer. I do know that t-SNE will often run for as many iterations as you let it run, meaning that it does not converge on an optimal clustering. So, in that case, you could say that the clustering is never perfect.
Excellent Explaination. Tripple BAM !!!
Thank you!
Thank you. I am not sure if you remember me from the PCA video. I have a job now. My job do not have high salary, but I could now support you by donating and thank you now. 😊
WOW! Thank you so much. And congratulations on getting a job!!! HOORAY!!! TRIPLE BAM! :)
@@statquest Keep doing great work sir! Also, it would be great if you could make a video about the comparation between clustering methods. 😁
@@tuongminhquoc Thanks and I'll keep that in mind!
This is a great explanation thank you!
Glad you enjoyed it!
Thanks for the video - awesome work. I don't get where you get the width of the normal distribution from? How do you know that it is 1 for the tight cluster and 2 for the wide cluster?
You can estimate that from the data.
Thank you very much for these (and all the other) videos, they are very helpful to understand the basics. Regarding t-SNE, the only think I do not fully understand is the use of a t-distribution to model the scores of the points in the low-dimensional space. Why "without it the clusters would all clump up in the middle and be harder to see"? Is it due to the fact that the bell of the normal distribution is higher in the middle?
It has to do with the t-distribution having fatter tails - more wiggle room for variation.
Thanks, love the channel. Question: since the t-SNE involves randomly plotting data on a number line, will identical plots be created if the function is repeated? i.e. could you get different looking t-SNE plots on repeated analyses?
Unless you set the seed for the random number generator, you will get a different graph every single time.
This explanation almost makes tSME sound like a clustering technique not a reduction technique..... That said, this was by far the best explanation I've heard to date.
That's a good observation. In many ways t-SNE is a hybrid method that reduces dimensions by clustering.
@@statquest Now if you can explain how to interpret a tSME plot. This would help immensely as it's virtually impossible to determine the correct perplexity number without understanding how to interpret the plot. This seems like one of those "blackbox" methods which we just trust. Keep up the great work!
Great, never heard of t-sne. Is it only for visualisation purposes? In PCA points are clustered based on the correlations. Do you have also an analogy with the loadings of the features like in PCA?
It's only for visualization. It does not have the equivalent of loading scores in PCA.
Made it look real simple.. thanks!
Hooray! :)
Hi, your videos are really nice. If possible can you upload videos with some math formulas related to this topic and maybe others as well and some sample problems and solutions to them, that would really help us understand even better. Thank you!
No problem. I have plans to make a video about the math behind t-SNE on my To-Do list.