The Central Limit Theorem, Clearly Explained!!!
ฝัง
- เผยแพร่เมื่อ 22 พ.ค. 2024
- The Central Limit Theorem is a big deal, but it's easy to understand. Here I show you what it is, then I describe why this is useful and fundamental to Statistics!
This StatQuest follows up on the one that describes the normal distribution...
• The Normal Distributio...
...and the StatQuest on Sampling from a Distribution:
• Sampling from a Distri...
For a complete index of all the StatQuest videos, check out:
statquest.org/video-index/
If you'd like to support StatQuest, please consider...
Buying The StatQuest Illustrated Guide to Machine Learning!!!
PDF - statquest.gumroad.com/l/wvtmc
Paperback - www.amazon.com/dp/B09ZCKR4H6
Kindle eBook - www.amazon.com/dp/B09ZG79HXC
Patreon: / statquest
...or...
TH-cam Membership: / @statquest
...a cool StatQuest t-shirt or sweatshirt:
shop.spreadshirt.com/statques...
...buying one or two of my songs (or go large and get a whole album!)
joshuastarmer.bandcamp.com/
...or just donating to StatQuest!
www.paypal.me/statquest
Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
/ joshuastarmer
#statquest #CLT #statistics
NOTE: Unfortunately I was a little sloppy with my terminology and that the word "samples" can mean different things, so let me try to rephrase it. If we collect 20 measurements and calculate the mean, and then do that a bunch of times (collect 20 measurements and calculate a mean), a histogram of those means will be a normal distribution. This suggests that an individual mean, calculated from 20 measurements, is, in and of itself, normally distributed. For example, if we had a uniform distribution and we collected 20 values from it and calculated the mean, then that mean would be normally distributed. We know this because if we repeated the process (collected another 20 values, calculated the mean, and then did that a bunch of times) the histogram of all the means we calculated would be a normal distribution.
ALSO: If you want to play with the central limit theorem, and see it in action, check out this page: cltapp.fly.dev/
Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/
I wonder since there is a rule of thumb for the sample size at each draw(at least 30), is there any rule of thumb for the number of times you have to repeat the process to get a normal distribution?
@@andreaxue376 Are you asking how many collections of 30 samples we would need in order to get a histogram of the means to look like a normal distribution? I don't know. I guess the answer is somewhat subjective. However, you could make an objective criteria, like how many collections of 30 samples would you need until a K-S test gives a p-value > 0.05. (A K-S test compares distributions). Hmm... An interesting question.
BAM! Thanks again! "Even if I'm not normal, the average is normal" is indeed the best way for me to remember the Central Limit Theorem :D
@@aditya4974 Awesome! :)
I got same doubt when i see the video because im from latam and we make a diference between samples and random measurements.
If you watch many StatQuest videos, the distribution of BAMs will be approximately normal 😂😂😂😂
BAM! :)
@@statquest you are a great man!!!
*a little correction the sample means of BAMS will be normally distributed
Do I have to watch at least 30?
@@simongross3122 Only in the wild!
This channel is a treasure.
Thank you! :)
that was indeed very clearly explained hah you've won yourself another subscriber!
Mr Starmer, I am a professional scientist with many years experience in the academic and commercial worlds and I must say that your videos are truly excellent. They really convey the central ideas so well and run that tightrope between too much detail and not enough perfectly. Keep up the excellent work !
Wow, thanks!!
@@statquest your explanations with slides are truly awesome! 👍👍👍
I am a 4th Year UG at IIT Kharagpur and you will be pleased to know that almost everybody on campus loves your lectures on Probability, Statistics and Machine Learning and consider it to be the best resource for cracking company interviews. Absolutely brilliant content!
Wow!!! That is great! Thank you very much. Maybe one day soon I can visit. :)
@@statquest IIT would be very happy to host you, do visit :)
@@amitavaroy5723 Yup
@@statquest Same at IIT BHU, you are pretty popular among engineering students! Everyone just refers you for anyone starting ML
@@burstingsanta2710 That's so cool. Thank you!
I just realized that the entire CLT was encapsulated in the 8s lyrics - "Even if you're not normal, the average is normal!" Hats off to you, man... I never imagined an ukulele being used to teach stats!!
bam!
Damn this dude is stellar at making statistics engaging!!
Thanks! :)
Triple BAM!
The fact that you are still replying to every new comment on a half-decade old video is amazing and commendable! Thanks for this, helping with my stats course for Uni :)
bam! :)
Sir, Your way of explaining is beyond Normal in brilliance. Could I request you to please make such enlightening videos on Linear Algebra and other Mathematical concepts in order to interpret the math behind the machine learning algorithms. The academic and text book notation as well as explanations gives me nightmares!
Thank you!!! One day I'll do it. In the mean time, check out 3Blue1Brown - he's got a series on Linear Algebra. It's good. When I make my own series I'm going to focus more on how the math is applied in practice (to statistics and machine learning), but his videos will give you a great start.
@@statquest looking forward to your explanations of lin algebra and yes 3Blue1Brown is great and I would love to see how you explain the application in ML
@@elsavelaz I heard the book "Hacking the matrix" does a great job of explaining Linear algebra with a view towards CS/ML ... maybe it would help
Hi, I just wanted to thank you for the videos,
I am doing a degree in statistics at the moment, my general method for learning is to work through what the professor give me (which I find very confusing), then come to your videos to get an easy to understand explanation.
You are really helping me out with my degree and I want to say thanks!!!
did u get ur degree yet
hi, I'm also studying undergraduate statistics. may I connect with you?
Just came across your channel. You explain every concepts with so much simplicity. The examples are spot on and helps to relate the concept with the problem at hand. Great work StatQuest!
Thank you very much! :)
Great video. I do want to point out that the Central Limit Theorem is why statisticians celebrate the Normal Distribution at all, because let's be honest, the normal density function is supremely ugly to look at and near impossible to fuss with.
The CLT is one of those "too good to be true" laws of the universe, and it is actually more miraculous than this video presents itself.
The most generalized form claims that the sum (not just the mean, which is just the sum divided by a constant) of any random variables will be roughly normally distributed. These random variables don't even need to come from the same distribution. You can sample from a uniform, a beta, a lognormal, an inverse gaussian, and the sum of those 4 values will be normally distributed. (fine print, the variances and means need to be in comparable range otherwise one sample will dominate).
It's also the reason why waiting time starts to become normally distributed, because it is the sum of exponential (which is a gamma distribution, which converges to normal very fast).
It is also the reason why most variables in life are normally distributed, because you can usually break them down into sums of smaller categories of unknown distributions.
I got your idea. I am thinking about the convolution of LTI system which is kind of sums, those sums would be a normal distribution as well, no matter what distrbuted input is. thank for the comment.
My math lecturer told me exactly that, she was amazing. She told me that the significance of Normal distribution was related to CLT, in that plotting sample size (30, 30 +)of any distribution function yielded to our beloved bell curve.
waiting time of? Any waiting time? E.g. waiting for a medical treatment
TRIPLE BAM!
GOD BLESS YOU, HONESTLY I WAS LOST. TILL I FOUND THESE VIDEOS. ITS REALLY VALUABLE TO ME. THANK YOU
I thought I was hopeless with statistics and I was sure I wouldnt pass my college stat exam, but you make it very simple, and you even make me laugh will the songs in the beginning. I cannot thank you enough. I hope god blesses you. Thanks dude.
Hooray!!! I'm so glad my videos are helpful! :)
Cauchy has some practical implications, like decay of radio active material in nuclear fall out, or chemical decomposition of material, where process tends to slow down at the end.
Josh--you are an inspiring teacher. Tidbit about distributions that don't follow the CLT. I believe the condition for the CLT to hold is that at least the first and second moments of the distribution are finite. There are many phenomena in nature that are, more or less, modeled by power law distributions (Pareto, Zipf, etc.) or ones with power law tails (Levy). Any distribution with a tail that decays slower than x^(-3) (i.e. x^-a where a
Awesome! Thanks for filling in all the details! :)
The Cauchy has a strong physical and mathematical background. E.g. the conf interval for the mean of a normal distribution with unknown sigma has a Cauchy distrbution if we have one sample. Also dividing normal samples gives a Cauchy. And firing in a uniform random angle, the projection to a line would be a Cauchy distribution. That can explain why archers sometimes make really bad shots.
Recently been working on modelling the effects of shocks in production in large firms in an economy to the shocks in the production of whole economy. The proposition is that the share in value added by the firms to the total GDP of the economy is log-normally distributed with a power law tail (Pareto). Hence we couldn't apply CLT as previous studies had done so.
There are plenty of things which can be modeled as a Pareto distribution. That's why the 80/20 principle (also called Pareto principle) is so famous, which gives a Pareto distribution with a=1.16. Also, if a distribution gets close to a Pareto, it still converges to normal, but can take an unreasonable amount of time. Taleb writes about it beautifully in his book Statistical Consequences of Fat Tails under the name of sub-asymptotic analysis.
Wish had discovered you sooner
Thank you! I love the way you explain the statistics. Much easier to understand with examples. I really hope I can find these videos earlier. Thank you for all the help.
I'm so happy to hear that you like my videos! :)
Its incredibly clear explanation. I just got lucky to find your channel while I was starting to find statistic boring...Thank you so much for your sense of humor and your great ability to explain something in a very simple way, i know it takes a lot of experience and knowledge.
Thank you! :)
The video and source is extremely helpful in understanding concepts. The visual examples are great and the humor helps demystify difficult topics. Thanks Josh!! I wouldn't be able to make it through my classes without it!
Glad it was helpful!
Thanks, thanks and lots of thanks... I love your way of explanation BAM!!!. Can you please make videos on the following topics-
1. Bayes for ML, I mean how Bayes helps us to find the best parameter of a model and probability of a prediction.
2. MCMC sampling methods.
YOOOOO, YOU ARE MY EXAM SAVIOUR!!!! PLEASE KEEP THIS CHANNEL UP AND GOING.
The way you say 'clearly explained' really reflects. Keep up the good work please!!!!!
Right now I’m studying to take the first actuarial exam in probability, and I just discovered your channel. You just earned a new subscriber!
Thanks and good luck!
I have not found a single video that explains this better than you do. Great work + 1 sub
Thank you so much! BAM! :)
Thanks a lot! I've tried the examples you gave with python. I sampled from uniform and exponential distributions, computed means and draw histograms and bam! This actually feels like magic. I'm looking forward to understand the theorem more. I read the wikipedia page and it actually seems like there are lot to learn!
You're off to a great start!
The first line of this video explained everything.
Hands down the best channel on YT to learn statistics. Thanks for sharing your knowledge.
Wow, thanks!
Thank you very much to provide us the more understandable way of teaching. It is just simple and pure.
Thanks! :)
This an amazing lesson Josh. Every student in statistics could benefit from this video alone.
Thank you!
Your "Triple Bam!" encouraged me more to review Stat subject for my FE exam, thank you wizard! :D
Good luck! :)
"Even if you're not normal, don't worry the average is normal". That's so deep.
bam! :)
Wow. I can adopt some of your teaching techniques for future classes I may have. You're very good
Thank you! :)
Why can't all teachers be like you?
Thanks for the amazing content!
Thanks! :)
Because teaching talent is not uniformly distributed =]
@@johnmolokach_staff-southga3529 TRIPLE BAM!!!
Enjoyed your video very much. I have been teaching statistics and programming statistical on and off for 50 years and this is one of the best explanations I have seen. I particularly appreciate your pointing out that a sample size of 30 is not a magic number. I wish you added that consistency of the data affects the needed sample size for generalization, but it's probably in another lecture. It's good to see you are reaching so many students. Keep up the good work.
Thank you very much! :)
When I first learned about CLT, this was exactly the intuition that I got. It is one of the most intuitive concept. Yet, at graduate level stat course (from Econ department), they introduce all sorts of asymptotic approach to it and I got lost.
My question to you is, What can be a potential "practical" gain in going into the mathematics of asymptotic behavior of CLT?
The best and clearest explanation of the central limit theorem I have ever seen & heard.
Hooray!
reviewing stats for my ml course, found these videos super useful, thanks!
Awesome! Good luck with your course. :)
When i see your videos two words coming in my mind : "Bam", "Hooray" 😂
Hooray!!!! :)
@@statquest bummer
WOW your videos are the bests in statistic!!! Thank you!!!
I have seen many animated ways to describe mathematical/probabilistic concepts. But your one is short and simple that can stay in mind.
Thank you very much! :)
How can you have dislikes on your videos? I think it is also because of CLT.
BAM!!!!
I became a great fan because of the way you teach the concept. I will never forget the CLT in my life. BAM !!!
BAM! :)
My new favourite pastime is listening to Sal Khan say "Sampling distribution of the sample means" over and over.
Ps. learning maths from Khan Academy, followed by watching these videos, is a really effective way of learning statistics.
Cool! :)
I am doing the same 🥰.
i just graduated from pharmacy and started a job that requires knowledge about statistics and your channel helps a lot! thank you!
BAM! :)
THANK YOU SO MUCH! I have been looking for some videos for a while to finally understand statistics and I would never believe that learning this subject in English (and not in my mother tongue) will help me!
Glad it helped!
You are awesome Josh. I already knew the concept but felt just now ;)
Thanks!
This channel is an absolute gem 💎
Thanks!
It's such a simple and obvious concept but it didn't click in my head until you showed it. Thanks!
Bam! :)
spend 10 mins on your videos and cleared my 10 years doubt, paypal donate just sent, thank you so much, will watch all of your videos
Awesome, thank you!
Regards from Brazil, one of my favorites channels! Really didatic
Muito obrigado!
The central limit theorem does not apply to Pareto distributions since the mean and variance are infinite! Bammm!
😂😂
I've met folks hoping that we could understand this concept only looking at formulas. I wish your video existed earlier, thank you, never too late to understand!
Thanks!
heck yeah man! thanks for explaining concepts so simply, these are super helpful in my stats study :)
Happy to help!
i luv your classes
thank you from brazil!!!
Thanks! :)
"Saturday" a vivacious tune Josh keep up the music
Thank you very much! :)
Dude! Your videos are a joy to watch! Thanks for this gift to the world!
Wow, thank you!
Sir, your way of explaining the different concepts about statistics is really beautiful. It helps me a lot to clear my queries. So, Sir I just want to request u to make a stat quest video on factorial design...
Have you seen the linear models StatQuests? Factorial design is a type of linear model. If you have time, watch those - they'll get you 80% of the way there - there are few extra details (like how to check for interactions and what not) that I don't cover - but the main ideas are all there. Here are the links:
Linear Regression: th-cam.com/video/nk2CQITm_eo/w-d-xo.html
Multiple Regression: th-cam.com/video/zITIFTsivN8/w-d-xo.html
t-tests and ANOVA: th-cam.com/video/NF5_btOaCig/w-d-xo.html
Design Matrices: th-cam.com/video/2UYx-qjJGSs/w-d-xo.html
That last video (which builds on all the previous ones, is the most important thing. If you understand design matrices, you're just a step away from factorial design.
@@statquest ok sir.
I get so enthusuatic when he goes "BAM" 🤣🤣🤣
Hooray! :) BAM!!!!
These videos make my day. I'm a Quant Tutor and it really comes in Handy!
Awesome! :)
Thank you! You are a wonderful teacher! The theory has been explained so clearly. It is easy to understand.
Glad it was helpful!
Thank you for this video! The Central Limit Theorem was making my head spin but your video made it finally click! You have gained a subscriber :)
Hooray! Thank you.
Your 7 min TH-cam video was more useful
and clearly explained than my 2 hour lecture. Thank you!
Wow! :)
Thanks again Josh. Today my prof taught CLT in the class and as usual am here to understand what his words actually mean !! :)
Hooray! I'm glad the video helps! :)
The guy made the concept easy peasy lemon squeezy!!😎
Absolutely loved the way the things were elabrated.😍
Thanks!
Thank you Josh, so awesome explanation for very important theorem !!!!!!!!!!!
"Even if you are not normal averagre is normal" CLT
:)
Sir, my question is that, why there doesn't exist the mean of Cauchy distribution even if it is continuous.
I think the simplest explanation is that the tails for the Cauchy distribution are too "fat". If you compare a normal distribution to a Cauchy distribution, the tails in the normal distribution get smaller much faster than the tails in the Cauchy distribution. For the normal distribution, when we collect a large number of measurements, most of them will be from the middle (near the mean) and only a few will come from the tails. This allows the estimated average to converge on the center of the distribution as the sample size is increased. In contrast, a large sample from a Cauchy distribution will have a lot of measurements from the tails, making the average value unstable - it could be a value near the middle, but it could also be a value near the edge. Increasing the sample size simply increases the chance you'll get more measurements from the edges that prevent the average from converging on the center of distribution. Does that make sense? If you want to see the math, there are plenty of webpages that will walk you through it.
@@statquest very very thanks sir for this
I like how you explain things in a funny and simple way. Thank you so much!
Thank you!
Thanks Starmer im benefited from you. Salute you.
You have worked in biostatistics for twenty years!Awosome!
Thanks! :)
the BAM!!! gets me every time.
:)
Very helpful and clear! Because you asked for the generalized Pareto distribution breaks down the CLT: a combination of n extractions from a GPD with equal or different parameters is itself GP-distributed
Nice!
great way of teaching. Keep it up. The world needs it. Thanks
Thank you!
Quadruple Bam !! The distribution of 'the number of times you say "Bam" in your videos', in not Normal!
That's awesome! You made me laugh out loud. :)
Quintuple BAM!! The distribution of the mean of 'the number of times you say "Bam" in your videos', IS Normal!
@@JuanuHaedo I love it! This thread of comments is probably my all time favorite. :)
Bam! apply central limit theorem to make it normal
Sorry 2 Qs
1. Just to be 100% clear - When you say at 1:30 "20 random samples" you mean a random sample of 20?
2. The labels on Y axis are throwing me off. For example, on the uniform distribution how can all values have a probability of 1.0? My first thought was "1 means 100% probability of that value occurring" But they can't all have a 100% probability of occurring. I'm starting to suspect that 1 is referring to relative probability (even though that's not something I 'm super familiar with).
These are good questions!1) I mean that we collected 20 data points. Unfortunately, as you observed, "sample" is a somewhat vague term. I'll try to be more careful in the future.
2) Probability isn't the y-axis value for a specific position along the x-axis (that's actually called "likelihood" - see my video Probability vs Likelihood for more details: th-cam.com/video/pYxNSUDSFH4/w-d-xo.html ). Probability is the area under the line (or curve or whatever the shape you continuous distribution has) between two points on the x-axis. So, to calculate the probability of observing something between 0 and 0.5, you integrate the function between 0 and 0.5 to solve for the area under the line. In this case, with the uniform distribution, the line is set to y=1. The integral of this line between 0 and 0.5 = 0.5. So the probability of observing something between 0 and 0.5 is 0.5. The probability of observing something between 0 and 1 is the integral of the line (y=1) from 0 to 1. This integral = 1. NOTE: With the uniform distribution, the area under the line is always a rectangle, so you can, more easily, solve for the probability by just multiplying the width of the rectangle by the height of the rectangle. Does this make sense?
@@statquest Thank you that is helpful. I think I "knew" that at one point about area under the curve but forgot somewhere along the way. I'm also going to watch your other video on Probability vs Likelihood
I think the mistake you made is very common - and with the uniform distribution, it's super common. So no shame there. If you have time, you should also check out one of my videos on Maximum Likelihood - it will help you understand why people would even care about calculating likelihoods. th-cam.com/video/XepXtl9YKwc/w-d-xo.html
Hey Josh, these are pretty interesting videos. Could you do a video on the Poisson , Geometric and Weibul Distribution and details on its practical implications. More or less on , why do certain 'natural' events follow them. Like Airport arrival time for Poisson
Those are all good suggestions and I'll keep them in mind.
You've made me visualize statistics. When I now look at a model output at work or in a presentation, I can relate that to mice height, mice weight, gene expression and actually explain it, suggest another method and why it might provide better results. Although I'll have a graduate degree in the data science soon, it's the day I finish working through your videos I will confidently say that I am a data scientist. Thank you for teaching me to love statistics!
BAM! :)
next video: quadruple bam!!!!
Dang!!! :)
"After we collect 10 samples.." should be "10 times of 20 (or n) samples..."
Am I correct?
I'm a little loose with my use of the word "sample", and for that I apologize. Sometimes I use "sample" to refer to an individual, but technically a sample is a collection of individuals that represent a population. Google "Random Sample" for more details.
Beautifully explained !
Thank you very much sir, i recently started my data analysis journey. Your videos were lot helpful
Glad I can help! :)
While I appreciate parts of this video for being clear and easy to understand, it is very wrong in terms of the fine print. Although the *population mean* of a Cauchy distribution is undefined, you can ALWAYS calculate a sample mean. The CLT does rely on having a finite *population mean*, but that's not the important part of the fine print anyways! The part about the sample size is far more important. There are many distributions in real life (such as income for certain groups) which may require far more than 30 samples for the CLT to provide an accurate approximation.
And for any distributions which have not finite expected value (population mean), you can calculate the finite sample mean, and you MAY NOT realize that you estimate infinity with your sample mean calculations. Anyway, one of CLT (yes, there are many!) is for the standardized random variables, i.e., subtract the sample mean and divide this by the (corrected) sample standard deviation. The approximate distribution will be the standard normal one, if the expected value and the variance of the original distribution exist. And the histogram is wrong for equidistant based columns!
WHAT THE HELL!! I AM IMPRESSED! Well done mate, thank you very much.... In the beginning I was like, what the hek is this song?? and at the end I was like BAM! now I get it... I will probably take this for the rest of my life.
bam!
I finally understand this after so many years! Thanks and Double BAM!
Happy to help!
Thanks for videos. Very helpful. Please make video on convergence of random variables.
I'll put that on the to-do list. The more people that ask for it, the more I'll push it to the top of the list.
You are creating absolutely amazing content here! Keep moving on, please. I have a question about what you said. If we can use the central limit theorem for statistical tests, why do we have to choose non-parametric tests when data is not normally distributed. After all, we should be able to use the central limit theorem to get a normal distribution and do the parametric test.
What you say is correct. We can almost always use parametric tests that assume normality when comparing means. However, if we have a strong suspicion that the data is far from normal and the sample size is very small (like n=3), then we might have to resort to a non-parametric test.
@@statquest thanks for your reply
Good job! Please make a video about Law of Large Numbers.
It has really helped my understanding of the theory! Thank you.
Glad it helped!
Love your videos, keep up the good work!
Thank you!
just love your video! thank you. this is so helpful.
Thank you very much! :)
Thank you so much for your great explanation!
You're very welcome!
The BAM! earned my subscription. This is really entertaining.
Hooray!!! :)
great explanation! having fun listening to it as well. thank you!
Glad you enjoyed it!
Thank you very much to make statistics interesting and appealing!!!
Thanks!
This lecture is beautiful. Thank you sir.
Thank you! :)
I literally laughed so hard at the “Who cares?”
I wasn’t expecting to laugh while trying to understand statistics. You’re good..!!👍🏻
bam! :)
Love your videos and the way you say BAM!!
Thank you! :)
Thank you..u made me ..finally understand these complex terms so easily...BAM...😊
BAM! :)
Thanks! Thanks! Thanks! (1000x)
I'm loving all your videos!
Thank you very much! :)
You make statistics fun to learn Josh.🤩
Thank you! :)
Hi Josh, the only distribution I can think of that breaks the CLT is the Pareto distribution. It is used frequently in actuarial science and economics.
Cool!