Support StatQuest by buying my books The StatQuest Illustrated Guide to Machine Learning, The StatQuest Illustrated Guide to Neural Networks and AI, or a Study Guide or Merch!!! statquest.org/statquest-store/
Guys, I can't believe you are doing all this. I am trying to break into the field of data science, and your videos are really great because you are doing it in such an entertaining way. Big thank you!
Months ago, I found an idea to know how machine learning may make varied prediction given different sample orders in both train and test set under influence of bootstrapping for my thesis. But, knowing that I had spent too much time thinking on how to clearly communicate the CI results to supervisor, promptly I jumped to watch this video and this is exactly what I have been searching for. You have my deep gratitude.!
I'm currently writing my bachelor thesis and this video helped me a lot, thank you! What I like most is, that it's not too long and on point. Moreover, I'm not a native English speaker, but the video was very clear and easily understandable.
You know, I am a Data scientist and work in the banking sphere for 3 years. I noticed your videos in my TH-cam recomendation section and was like: "easily explained? Ye ye haha just another video for those who hope to easily learn ML and statistics, well let me watch it during my breakfast". And I was shocked. I realized that the use of Python made me completely blind about some connections between measurements. Sometimes I run tests without any true understanding. For example, those last 10 seconds about "when we should run t-test" were completely new for me! And that can be told about a lot of your videos. There is always a tiny detail that makes me say "oh, wow, that was something I've never noticed". You should definetely run a course on Coursera...
I usually don't comment on YT videos, but I'm eternally grateful that you are posting such incredibly useful videos. Thank you very much!!! God Bless!!!!
Thanks for such a clear explanation of bootstrapping and confidence intervals. The two concepts do go together so that understanding bootstrapping makes confidence intervals and their interpretation easy to understand.
Josh, you are genius. Finally i got the idea how t-test are made and confidence intervals and p-value relate to each other! And why one can simply check if "0" statistics belongs to conf int!
I consider myself one of the most stupidest people on earth in learning stats. and yet here I understood the CI concept very well. a big fat thank you to you 😊.
Your example also shows how backwards confidence intervals and p-values are. You already assume a mean of ~26. But you end up calculating a p-value to make a probabilistic statement about the mean being lower than ~21 ... given samples from a distribution with a fixed mean of ~26.
02:12 I can not understand why keep on taking samples from original samples and calculated the mean, Shouldn't it to take samples from the original data, Not the Sample Data? Any one can reply me please
In real world you almost never know the exact population mean. You can sample the original data say 100 times more than Josh, but I can always oppose - why not 1000 times more - that would be more precise. And so on. So there is no solution until you sample the whole population - but this is impossible. That why the goal is - not getting the most precise assessment of population mean (it's impossible) but getting to know the boundaries where it lies with some % of confidence. And this is _possible_.
First, if p-value < 0.025, then it is also < 0.05. Now, the reason I used 0.05 is because that is the most commonly used cut off for significance. In other words, we know that the p-value will be at least as small as it needs to be in order to obtain statistical significance.
Hi Josh, thank you for the nice video! One quick question, I learnt the interpretation of 95% confidence interval is 95% of confidence intervals will contain the true mean (i.e. if we have n=100 random samples of size 5, there are 95 confidence intervals will contain the true mean). It seems different from your explanation here?
It's the same. However, we are arriving at the confidence interval differently and we need to make sure we don't confuse a bootstrapped mean for a population mean. The interval that contains 95% of the bootstrapped means is a 95% CI, and thus, if we repeated the process a bunch of times, 95% of the intervals calculated that way will contain the population mean.
Sure. The point, however, is that 0.05 is the usual threshold for making a decision about the hypothesis. So as long as we are < 0.05, we will reject the hypothesis.
Hi Josh! Great videos (I'm currently on a StatQuest marathon and it has been incredibly helpful!) I have a question though. Could you explain the bit about p-value being less than 0.05 in case of the weights of female and male mice? Instinctively I understand that there's a statistical difference between the true means of these two but I'm struggling to relate it to the idea of p-value. Thank you!
Same thing occurred to me!! If in a 95% confidence interval, the remaining 5% do not cover means right? If so, then how come its p-value is significantly different?
By definition p-value denotes a probability of (something other which is equally rare + something rarer than null hypothesis) happening. Here the null hypothesis is that the means for both male and female mice are from same population. But we already know that 95% confidence intervals of both male and female means don't coincide. So there is only one possibility left that less than 5% cases will have the possibility of their means coinciding. Which is why p-value is
In the example where you want to get the p-value for true mean less than 20, and the result is less than 0.05. Does that mean it's very unlikely that the true mean is less than 20? Thanks!
I'm wondering if there's a correlation between this method and central limit theorem? Because if my understanding is correct, we can also construct a confidence interval using the latter.
The central limit theorem makes it possible to create confidence intervals for the estimate of the mean, but only the mean. In contrast, bootstrapping allows us to create confidence intervals for any statistic we want.
Total types of samples that we can take : 23C12 = 1352078, out of this we are asked to find sample mean of around 10,000 samples. Now, we can define confidence interval as : 95% confidence interval is just an interval that covers 95% of the above calculated sample means.
Hello Josh, @ 3:22 is it correct to say that 95% of all confidence intervals will contain the population mean? I am having a hard time understanding if this interpretation is the same as yours. Also, I am a bit confused about the bootstrapping. How do we construct the interval? How do we adjust the interval for different levels of alpha? Thanks again bro!
At 3:22 I say that when using bootstrapping a 95% CI is an interval that covers 95% of the (bootstrapped) means. Now, if we made a lot of 95% CIs using this method, then 95% of them would contain the population mean. For more details on bootstrapping, see: th-cam.com/video/Xz0x-8-cgaQ/w-d-xo.html
At 3:22 -- should the confidence interval bar has some constraints? (eg, should be the shortest bar to cover 95% means, otherwise I can shift and prolong it to cover 95% )
@@statquest ah I see, here we are talking about CI itself, not the CI we using in N-distribution calculated by "std". (even in distributions CI can shift and change) Thank you!
@@stevequan7306 That's exactly right. There are different ways to calculate the confidence interval, and you end up with different intervals, but as long as you are covering 95% of the means (or at least doing that in theory), then you have a 95% CI.
@@statquest Hi Josh, Hi Steve, I had the same question as Steve. But I don't understand the answer/explanation. Why can CI be any range? Shouldn't it be the shortest range that cover 95% of the mean values?
Why does the 95% CI select some means/values and not others? Does it need to be in the center? If so, how? I would suppose that if you force the mean of the interval to be the mean of all means, it would give you a CI similar to the ones you showed in the video.
Thank you so much Josh, I just watched your videos of standard error and confidence interval. Could you please verify if I understood it correctly? 95% confidence interval = mean of means ± 2 Standard Error
It depends on how you calculate it. If you are using bootstrapping, then your method is correct. If you are using a formula to approximate bootstrapping (so you are not using bootstrapping), then you have to appeal to the t-distribution (instead of the standard error). This is because for small sample sizes, the t-distribution is a little wider than a normal distribution, and that compensates for the fact that a small sample size means we have very limited knowledge of what is going on.
You are simply Awesome !! I have a doubt here though, Let's say the point estimate is the sample mean. We can repeatedly keep taking the sample means and then plot all these sample means in a histogram and we would observe a normal distribution called the sampling distribution of the sample means. The mean of this distribution would be a better estimate of the population mean and its standard deviation, called standard error would be the population standard deviation/sqrt (number of points in a sample). Won't the confidence interval(say 95%) range be (sampling distribution mean - 2 SE,sampling distribution mean + 2 SE) instead of (point estimate - 2 SE,point estimate + 2 SE)? Why would we use the sample mean(point estimate) in calculating the confidence interval range? What if that particular sample mean was like an outlier in the sampling distribution of the mean? In that case, doing +/- 2*SE wouldn't be a good judge to measure population mean right?
The technical definition of a 95% confidence interval is that if we repeat the process a lot of times, calculating 95% CIs each time, 95% of the CIs we calculate will cover the true (population) mean. So, sure, sometimes we get outliers, and our CI is bad, but that is expected about 5% of the time we calculate a 95% CI.
@@statquest Thanks for responding, Josh ! Does this mean that while calculating the 95% CI, we are assuming that our point estimate (sample mean) is always 1.96 SD away from the population mean(mean of the sampling distribution) ?
@@statquest Thanks for responding again :). I'm having a lil bit of tough time connecting all the dots, sorry for the long questions ! If we are given the pop SD and use z stats to calculate 95% CI for mu, we say z=xbar-mu/(sigma/sqrt(n)) where z=1.96, xbar is the sample mean or point estimate and sample SD can also be computed. Based on the definition of z score, does this not mean that xbar is 1.96sd away from mu ? In any case, What is the intuition behind using this formula. Thanks in advance !
A 95% confidence interval is just an interval that covers 95% of the means. This has cleared confidence intervals for me, without talking about the standard error and all other statistical jargon that's so confusing. Quick question: What is the difference between confidence intervals and confidence levels? Thank you !
Keep in mind that the 95% confidence interval is just the interval that covers 95% of the means created with bootstrapping. And I believe that confidence levels just refers to the "95%" or "99%" - 95 and 99 are the levels that can then be converted into intervals.
But I don't understand, maybe I'm wrong. Perhaps you are giving a Bayesian approach? The frequentist approach would tell us that it is the frequency with which 95% confidence intervals calculated from many studies, would contain the true effect. The true value would be in the 95% confidence intervals. Thank you in advance
This is the bootstrap approach to the frequentist CI. Since we are using bootstrapping, the interval covers 95% of the bootstrapped means and is equivalent to an interval that, if we repeated the experiment a lot of times and calculated the CI the same way, 95% of those CIs would cover the true mean.
I'm not sure I understand your question. Are you asking about the distribution of the samples? (like, are you asking about the whether or not the data come from a normal distribution?)
Josh, although you're crystal clear, i still don't get the following point: according to my interval i have a range of weights that can be considered an estimate of the true mean. But, now I get this 20 weight, and I know that it is out of my interval, so it's very unlikely that it represents a significative diference (it happened by chance). So what I do next? Discard this sample, and run another? What needs to occur so I say that yes, this value of 20 really show something that I need to pay attention? And, you're saving my as...s with all these simple explained knowledge. I cannnot thank you enough. greetings from brazil ;)
You might need to learn about hypothesis testing to understand the value of the confidence interval. Here's the link: th-cam.com/video/0oc49DyA3hU/w-d-xo.html
When you use bootstrapping, the 95% CI is any line that covers 95% of the bootstrapped means. So, take your pick! However, usually people center it over the estimated mean.
Does the number of random selections to calculate the bootstrap mean, from the sample need to equal the sample size as it does in your example? i.e. could you have chosen 8 random samples from selection and calculated the mean and bootstrapped mean and repeated this 10000 times?
Hey Josh, so if I build a confidence interval to the mean in a 95% confidence level, 95% of the bootstrapped means will be in the confidence interval, and there's a 95% chance the mean will be in that confidence interval?
Any chance you could do a video on frequentist confidence intervals, based on the central limit theorem? Also, with the bootstrap method, is the interpretation that you're 95% confidence that the population mean is contained in the interval still valid? Thank you.
Confidence intervals always have the same interpretation. If we repeated the procedure to calculate the CI a bunch of times, 95% of them would overlap the population mean.
Thank you so much for the explanation! i have one question tho - could there be more that a single 95 interval for the example above and does it matter? how do you construct it? thanks!
Hi Josh, a little confused about the p-value here as, if less than 0.05 is considered as less likely to reoccur then why are we considering variables with less than 0.05 as highly significant variables in the regression models?
A p-value < 0.05, means, in general terms, that the result is probably not due to random chance. Thus, when we do linear regression, a small p-value tells us the the relationship between the independent and dependent variable is probably not due to random chance.
Thanks for the video! Are confidence intervals always defined for distributions of the sample means (i.e. means obtained by boodstrapping)? Or can you also calculate them for one single experiment? Or the means of multiple experiments without bootstrapping?
Because of the central limit theorem, all means are normally distributed, so there is a closed form equation for all confidence intervals based on that and you don't need bootstrapping. In other words, you can calculate the CI with a single set of measurements. However, I believe the concept is easier to understand with bootstrapping.
One problem to discuss: In this video, you said the probability that the "true" mean is in this area has to be < 0.05. It was my definition of confidence interval before, but after some theoretical statistics courses, my definition has been refereshed. My statistics teacher told me that in this sentence, since neither the "true" mean nor that area is a random variable, this is in fact not a random event and give it a probability is logically wrong. However, the theoretically correct definition (i.e., after hundreds of repetition, 95% of 95% CIs will include the population mean) is usually not informative by itself for researchers, so the definition in the video got to be more popular in the end. I'm not sure whether you took all of it into consideration and then made that video, otherwise it may be better to review it if you remake this video in the future! Please don't treat it as a negative criticism, just because you have released so many helpful videos, I think it would be meaningful if we can help to make them better!
At that time point, we're discussing calculating a p-value with the null hypothesis being H0 < 20. By looking at the confidence interval, we know that the p-value (which is a probability) will be < 0.5 and thus reject the null hypothesis.
PI estimates range of RVs or some statistics of RVs. Its limits are not random (are not drawn from samples). For example for standard norm distribution ~99% PI for RV = mu+-3*sigma. And we must _know_ mu and sigma. On the opposite CI estimates range of not random but exact value - the population parameter (mu for example) though we don't know it's exact value. Its limits are drawn every time from different samples so they are random.
@@statquest thanks for the reply, figured this out myself right after watched another video from you after 5 mins yesterday. 😃. You helped me to review my lifetime learned statistic courses in 50 mins and cleared many cloudy concepts in the past. Love you!
Hi Josh. Sorry to bother you. I have a simple question. I always get confused by this. May be a dumb question. But want to get it clarified Its about how probability is realised. So, sometimes people say, for example in this case , it is said that 95% of the means lie in a certain confidence interval. Here its like speaking in terms of proportions . I understand this . Ok now lets take another statement. Lets take for reference in this video at 3.47, it states that " We know that anything outside of it occurs less than 5% of the time. So here , its speaking in terms of probability or percentage of chance. So, here is my big confusion.. So , if 5% of the total points (here means) lie outside the confidence interval, how or why it can/is said that " at 3.47, it states that " We know that anything outside of it occurs less than 5% of the time" Could you please explain me . Thanks in advance
I'm not really sure how to rephrase it. In this case, proportions and probabilities are equal. If 95% of the means end up in a region, then there is 95% chance that if we randomly selected one of the means, it would be within that region.
Thank you, Josh. Great video. However, I don't know how to calculate the confidence interval. Is it calculated through 2 times the standard deviation of the mean of the sample means?
There are lots of formulas for calculating confidence intervals. Conceptually, the easiest one to remember is bootstrapping, however there are lots of other formulas you can use. For details, see: www.statisticshowto.com/probability-and-statistics/confidence-interval/
i'm a little confused, is it true that if you use bootstrapping of a sample that will only tell you with what confidence you can state the mean of that particular sample? wouldnt you need to know the population standard deviation to get the confidence intervals for a sample from the population?
i think i've worked it out, i wasnt adjusting the margin of error in accordance with the sample size ie changing the t/z value, that is when you sample is really small the confidence interval becomes massive to account for that. Either way i would still like to hear your answer if you have time, thanks.
Dir Sir: I really enjoy watching your videos. Thank you very much! I still have one question regards to 'confidence interval'. Let's say that based on the sample of 12 female mice, we have got the 95% confidence interval is: 200grams +/- 30 grams. One interpretations is: any simple random sample of 12 female mice, its mean weight will be in the range of 170g to 230g, with 95% confidence. I guess if the sample size is 20 (female mice), I cannot say the sample mean will be in this range (170-->230g), with 95% confidence. Correct? (Because the sample size is not 12). Another question: I also intend to say, any random female mouse, its weight will be in this range(170-->230g), with 95% confidence. I cannot say that, correct? Thank you!
Unfortunately, the language surrounding confidence intervals is very tricky. A 95% Confidence Interval should be interpreted like this: "If I re-do the exact same experiment a lot of times and use the exact same method to calculate the 95% confidence interval, 95% of the intervals will cover the population mean". If you're not familiar with the concept of a "population mean", check out my StatQuest on Population Parameters: th-cam.com/video/vikkiwjQqfU/w-d-xo.html
@@statquest Thank you very much for your reply! It is very helpful. I think if the sample size is large enough(12 is not large enough), the sample mean(x-bar) could be treated as population mean(miu). If based on the sample statistics, the 95% confidence interval is 170g-->230g, indeed I can get the density curve and make this statement: the weight of a random female mouse will fall in this range, with 95% confidence. Is this correct?
@@xsli2876 Unfortunately, that statement is not correct. Again, the Confidence Interval only tells us that if we repeated the exact same experiment and used the same method to calculate the confidence intervals, 95% of the intervals would cover the population mean. In other words, the confidence interval tells us about the "mean" and not individual measurements. If you want to make a statement about an individual measurement, like "there is a 95% chance that the weight of a random female mouse will fall within this range", then you need look at the distribution (probably normal distribution), and find the range that 95% of the values fall in. That is the range you are interested in. In math terms, the 95% CI is usually the mean +/- 2 times the standard error. In contrast, for individual measurements, a region where 95% of the measurements fall is the mean +/- 2 times the standard deviation. If you want to learn more about the standard error vs the standard deviation, check out: th-cam.com/video/A82brFpdr9g/w-d-xo.html
@@statquest Thank you so much. I got it. For individual measurements, a region where 95% of the measurements fall is the population mean(miu) +/- 2 times the population standard deviation(sigma). In real life, we don't know miu or sigma. However, if we have one large sample, we may use the sample mean(x-bar) as the approximate population mean, the sample standard deviation(s) as the population standard deviation.
@@xsli2876 Yes that is correct. If the sample size is not very large, you can use a t-distribution (which has fatter tails than the normal distribution) to compensate for the uncertainty in your measurements of the mean and standard deviation. However, as your sample size increases, the t-distribution converges to the normal distribution and then then you can just use the sample mean and sample standard deviation.
Thank you for your wonderful video, here I have a question. When 95% CI do not overlap, we could say there is a significant difference between the two sample sets. I want to ask is we can conclude if the significant difference when the SD of two sets do not overlap, and how about SEM? Hope for your reply. :)
95% confidence intervals reflect the SEM, rather than the standard deviation of the raw data. For more details, see: th-cam.com/video/A82brFpdr9g/w-d-xo.html
Support StatQuest by buying my books The StatQuest Illustrated Guide to Machine Learning, The StatQuest Illustrated Guide to Neural Networks and AI, or a Study Guide or Merch!!! statquest.org/statquest-store/
Guys, I can't believe you are doing all this. I am trying to break into the field of data science, and your videos are really great because you are doing it in such an entertaining way. Big thank you!
Thanks and good luck with Data Science! :)
how is it going man
@@Dupamine I am still just in the beginning, but I have just started in my first analyst job
@@joarvat hey I'm 20 rn and learning all bout statistics for data science. Is it worth it?
@@applepeel1662 I have got my first analyst job, and then I decided to go through the Data Science track with dataquest.io It's a great program.
I am medical student with an Bachelors in Science and this is possibly the only Stats tutorial, I have EVER been able to understand!!! Thank you
bam! :)
Josh, when I finish the StatQuest Statistics Fundamentals playlist, will you send me a BAM certificate? I want to be BAM certified.
BAM!!! One day I will make certificates. :)
@@statquest can I make one for you 👻
Oh! He will get your BAM certified
Months ago, I found an idea to know how machine learning may make varied prediction given different sample orders in both train and test set under influence of bootstrapping for my thesis. But, knowing that I had spent too much time thinking on how to clearly communicate the CI results to supervisor, promptly I jumped to watch this video and this is exactly what I have been searching for. You have my deep gratitude.!
bam!
I'm currently writing my bachelor thesis and this video helped me a lot, thank you! What I like most is, that it's not too long and on point. Moreover, I'm not a native English speaker, but the video was very clear and easily understandable.
Thank you! :)
Why can't others explain things as clearly and easily as you do? Simply amazing
Thank you!
You know, I am a Data scientist and work in the banking sphere for 3 years. I noticed your videos in my TH-cam recomendation section and was like: "easily explained? Ye ye haha just another video for those who hope to easily learn ML and statistics, well let me watch it during my breakfast". And I was shocked. I realized that the use of Python made me completely blind about some connections between measurements. Sometimes I run tests without any true understanding. For example, those last 10 seconds about "when we should run t-test" were completely new for me! And that can be told about a lot of your videos. There is always a tiny detail that makes me say "oh, wow, that was something I've never noticed".
You should definetely run a course on Coursera...
Wow!!! Thank you very much!!! :)
I learned stat for 6 years, and this is the best tutorial about CI. Thank you very much.
Hooray! :)
You are a true life saver for person like myself who needs such knowledge but never had a chance to get educated in school…thank you.
I'm glad I can help!
I usually don't comment on YT videos, but I'm eternally grateful that you are posting such incredibly useful videos. Thank you very much!!! God Bless!!!!
Wow, thank you!
Respect and gratitude to you!! Your videos are in my interview prep playlist! Thanks so much for making math understandable!!
Good luck with your interviews! Let me know how they go.
Thanks for such a clear explanation of bootstrapping and confidence intervals. The two concepts do go together so that understanding bootstrapping makes confidence intervals and their interpretation easy to understand.
Thank you very much! :)
If I ever finish my PhD, I propably need to credit you for every knowledge I have about statistics. And I actually learned this stuff beforehand.
Good luck finishing your PhD! You can do it!!! :)
hey, did you finish it? how's life
Josh, you are genius. Finally i got the idea how t-test are made and confidence intervals and p-value relate to each other! And why one can simply check if "0" statistics belongs to conf int!
bam!
This saves the world, thank you so much
Thank you! I'm glad I could save the world! I thought only Spider-Man could do that. ;)
I am also a fan and I highly recommend the videos from StatQuest to student in my class.
Awesome! Thank you very much! :)
This video helped me to better understand the p-value besides the confidence intervals.
That _was_ good, very clearly related. The introduction of the term 't-test' threw me however.
One of best tutorials I watched on net paid or otherwise...
BAM! :)
Thanks so much. I have been working on hydrology for many years and finally I understood this valuable concept.
Glad it was helpful!
This is by far one of the best videos I've seen. Thank you so much!
Thank you! :)
I am trying to complete all ur videos
Nice! You're making great progress! :)
me too! this stuff is gold
@@samuelkellerhals5942 agree, it is good!
@@statquest double BAM!!
This was great. I’m taking and finding stats complicated but this broke down the basics of what it was supposed to. Thanks 🙏🏾
Glad it was helpful!
Im gonna pass my stat exam thanks to you, you explain it so well :'))))))
Good luck!!
I consider myself one of the most stupidest people on earth in learning stats. and yet here I understood the CI concept very well. a big fat thank you to you 😊.
Bam! :)
thanks for sharing. will use this example on my students for sure. will link to the video, of course
Thanks!
Your example also shows how backwards confidence intervals and p-values are.
You already assume a mean of ~26. But you end up calculating a p-value to make a probabilistic statement about the mean being lower than ~21 ... given samples from a distribution with a fixed mean of ~26.
YOU ARE THE BEST OUT THERE!
Thank you! :)
Dude, beautifully and simply explained!
Glad you liked it!
Mr. Josh Starmer's singing abilities has significantly advanced since year 2015.
Bam!
Thanks, confidence intervals seemed so tricky! Till now!!!
bam!
your videos are god sent
Thanks!
Thank you for explaining this like a normal person and not like you're teaching people who already know how to do it.
Thanks! :)
very good explanation. I liked your examples very much. Thanks you sir!
Thanks!
another exciting quest complete!
bam!
You are a godsend Josh!
Thank you!
02:12 I can not understand why keep on taking samples from original samples and calculated the mean, Shouldn't it to take samples from the original data, Not the Sample Data? Any one can reply me please
Here's another way to look at it: th-cam.com/video/Xz0x-8-cgaQ/w-d-xo.html
In real world you almost never know the exact population mean. You can sample the original data say 100 times more than Josh, but I can always oppose - why not 1000 times more - that would be more precise. And so on.
So there is no solution until you sample the whole population - but this is impossible.
That why the goal is - not getting the most precise assessment of population mean (it's impossible) but getting to know the boundaries where it lies with some % of confidence. And this is _possible_.
This is so simple and eloquent.
Thanks!
Thank you. It's crazy how nobody else seems able to explain this clearly
Thanks!
Can someone please help me understand 5:23 why p-value
First, if p-value < 0.025, then it is also < 0.05. Now, the reason I used 0.05 is because that is the most commonly used cut off for significance. In other words, we know that the p-value will be at least as small as it needs to be in order to obtain statistical significance.
@@statquest thank you Josh for replying! Love your videos, currently I’m preparing for Data Scientist interview, your videos help a lot!!
@@jasonsj BAM! Good luck!
Hi Josh, thank you for the nice video! One quick question, I learnt the interpretation of 95% confidence interval is 95% of confidence intervals will contain the true mean (i.e. if we have n=100 random samples of size 5, there are 95 confidence intervals will contain the true mean). It seems different from your explanation here?
It's the same. However, we are arriving at the confidence interval differently and we need to make sure we don't confuse a bootstrapped mean for a population mean. The interval that contains 95% of the bootstrapped means is a 95% CI, and thus, if we repeated the process a bunch of times, 95% of the intervals calculated that way will contain the population mean.
In 5:02 , shouldn't the p-value be 0.025? Given the Confidence Interval is both direction.
Sure. The point, however, is that 0.05 is the usual threshold for making a decision about the hypothesis. So as long as we are < 0.05, we will reject the hypothesis.
So if p value for a sample is < 0.05, does that imply that the sample is not a good representative of the population?
It suggests that the sample may come from a different population than the one you think you are collecting it from.
I'm curently speedrunnig all your videos
Go for it! :)
Insanely useful
Glad you think so!
Hi Josh! Great videos (I'm currently on a StatQuest marathon and it has been incredibly helpful!)
I have a question though. Could you explain the bit about p-value being less than 0.05 in case of the weights of female and male mice? Instinctively I understand that there's a statistical difference between the true means of these two but I'm struggling to relate it to the idea of p-value.
Thank you!
Same thing occurred to me!! If in a 95% confidence interval, the remaining 5% do not cover means right? If so, then how come its p-value is significantly different?
By definition p-value denotes a probability of (something other which is equally rare + something rarer than null hypothesis) happening. Here the null hypothesis is that the means for both male and female mice are from same population. But we already know that 95% confidence intervals of both male and female means don't coincide. So there is only one possibility left that less than 5% cases will have the possibility of their means coinciding. Which is why p-value is
In the example where you want to get the p-value for true mean less than 20, and the result is less than 0.05. Does that mean it's very unlikely that the true mean is less than 20? Thanks!
Yes!
Excellent, thank you so much!
Glad it was helpful!
I'm wondering if there's a correlation between this method and central limit theorem? Because if my understanding is correct, we can also construct a confidence interval using the latter.
The central limit theorem makes it possible to create confidence intervals for the estimate of the mean, but only the mean. In contrast, bootstrapping allows us to create confidence intervals for any statistic we want.
@@statquest Thanks! It makes so much sense
Total types of samples that we can take : 23C12 = 1352078, out of this we are asked to find sample mean of around 10,000 samples. Now, we can define confidence interval as : 95% confidence interval is just an interval that covers 95% of the above calculated sample means.
:)
Wonderful series.
Thankyou for sharing your knowledge.
"A 95% confidence interval is just an interval that covers 95% of the means." 😁
:)
Very clear! thank you
Glad it was helpful!
Hello Josh, @ 3:22 is it correct to say that 95% of all confidence intervals will contain the population mean? I am having a hard time understanding if this interpretation is the same as yours. Also, I am a bit confused about the bootstrapping. How do we construct the interval? How do we adjust the interval for different levels of alpha? Thanks again bro!
At 3:22 I say that when using bootstrapping a 95% CI is an interval that covers 95% of the (bootstrapped) means. Now, if we made a lot of 95% CIs using this method, then 95% of them would contain the population mean. For more details on bootstrapping, see: th-cam.com/video/Xz0x-8-cgaQ/w-d-xo.html
@@statquest Thank you for the quick reply. It really separates you from the rest.
At 3:22 -- should the confidence interval bar has some constraints? (eg, should be the shortest bar to cover 95% means, otherwise I can shift and prolong it to cover 95% )
Surprisingly, it can be any bar that covers 95% of the means.
@@statquest ah I see, here we are talking about CI itself, not the CI we using in N-distribution calculated by "std". (even in distributions CI can shift and change) Thank you!
@@stevequan7306 That's exactly right. There are different ways to calculate the confidence interval, and you end up with different intervals, but as long as you are covering 95% of the means (or at least doing that in theory), then you have a 95% CI.
@@statquest
Hi Josh, Hi Steve, I had the same question as Steve. But I don't understand the answer/explanation. Why can CI be any range? Shouldn't it be the shortest range that cover 95% of the mean values?
@@karannchew2534 Why should it be the shortest?
Why does the 95% CI select some means/values and not others? Does it need to be in the center? If so, how? I would suppose that if you force the mean of the interval to be the mean of all means, it would give you a CI similar to the ones you showed in the video.
Traditionally, we center the 95% CI over 95% of the means, but you don't have to do it that way. You just need to cover 95% of the means.
Hi, thanks a lot for the video. for step, how large is the original sample size? much much much bigger than 12? thanks in advance.
I think it needs to be 8 or more for bootstrapping to work.
You mean the world to me man.
Thanks!
Thank you so much Josh, I just watched your videos of standard error and confidence interval. Could you please verify if I understood it correctly?
95% confidence interval = mean of means ± 2 Standard Error
It depends on how you calculate it. If you are using bootstrapping, then your method is correct. If you are using a formula to approximate bootstrapping (so you are not using bootstrapping), then you have to appeal to the t-distribution (instead of the standard error). This is because for small sample sizes, the t-distribution is a little wider than a normal distribution, and that compensates for the fact that a small sample size means we have very limited knowledge of what is going on.
@@statquest you're a god sent Josh. Thank you 😄
This is so helpful. Thank you
Thanks! :)
Amazing! Thank youuuu
No problem 😊!
You are simply Awesome !! I have a doubt here though, Let's say the point estimate is the sample mean. We can repeatedly keep taking the sample means and then plot all these sample means in a histogram and we would observe a normal distribution called the sampling distribution of the sample means. The mean of this distribution would be a better estimate of the population mean and its standard deviation, called standard error would be the population standard deviation/sqrt (number of points in a sample). Won't the confidence interval(say 95%) range be (sampling distribution mean - 2 SE,sampling distribution mean + 2 SE) instead of (point estimate - 2 SE,point estimate + 2 SE)?
Why would we use the sample mean(point estimate) in calculating the confidence interval range? What if that particular sample mean was like an outlier in the sampling distribution of the mean? In that case, doing +/- 2*SE wouldn't be a good judge to measure population mean right?
The technical definition of a 95% confidence interval is that if we repeat the process a lot of times, calculating 95% CIs each time, 95% of the CIs we calculate will cover the true (population) mean. So, sure, sometimes we get outliers, and our CI is bad, but that is expected about 5% of the time we calculate a 95% CI.
@@statquest Thanks for responding, Josh ! Does this mean that while calculating the 95% CI, we are assuming that our point estimate (sample mean) is always 1.96 SD away from the population mean(mean of the sampling distribution) ?
no
@@statquest Thanks for responding again :). I'm having a lil bit of tough time connecting all the dots, sorry for the long questions !
If we are given the pop SD and use z stats to calculate 95% CI for mu, we say z=xbar-mu/(sigma/sqrt(n)) where z=1.96, xbar is the sample mean or point estimate and sample SD can also be computed. Based on the definition of z score, does this not mean that xbar is 1.96sd away from mu ? In any case, What is the intuition behind using this formula.
Thanks in advance !
the humor is great
Thanks!
A 95% confidence interval is just an interval that covers 95% of the means. This has cleared confidence intervals for me, without talking about the standard error and all other statistical jargon that's so confusing.
Quick question: What is the difference between confidence intervals and confidence levels?
Thank you !
Keep in mind that the 95% confidence interval is just the interval that covers 95% of the means created with bootstrapping. And I believe that confidence levels just refers to the "95%" or "99%" - 95 and 99 are the levels that can then be converted into intervals.
@@statquest yes that's it. Thank you.
Amazing explanation..!!
Thank you! :)
Beautiful.
Thanks!
But I don't understand, maybe I'm wrong. Perhaps you are giving a Bayesian approach? The frequentist approach would tell us that it is the frequency with which 95% confidence intervals calculated from many studies, would contain the true effect. The true value would be in the 95% confidence intervals.
Thank you in advance
This is the bootstrap approach to the frequentist CI. Since we are using bootstrapping, the interval covers 95% of the bootstrapped means and is equivalent to an interval that, if we repeated the experiment a lot of times and calculated the CI the same way, 95% of those CIs would cover the true mean.
@@statquest ❤️❤️❤️❤️❤️❤️. Thank you very much
How do I check hypothesis for individual distribution sampling?
I'm not sure I understand your question. Are you asking about the distribution of the samples? (like, are you asking about the whether or not the data come from a normal distribution?)
Josh, although you're crystal clear, i still don't get the following point: according to my interval i have a range of weights that can be considered an estimate of the true mean. But, now I get this 20 weight, and I know that it is out of my interval, so it's very unlikely that it represents a significative diference (it happened by chance). So what I do next? Discard this sample, and run another? What needs to occur so I say that yes, this value of 20 really show something that I need to pay attention?
And, you're saving my as...s with all these simple explained knowledge. I cannnot thank you enough. greetings from brazil ;)
You might need to learn about hypothesis testing to understand the value of the confidence interval. Here's the link: th-cam.com/video/0oc49DyA3hU/w-d-xo.html
95% C.I. is the area where 95% of the means are present but from where should I start drawing the line of C.I.( from 1st mean or 2nd or what)?
When you use bootstrapping, the 95% CI is any line that covers 95% of the bootstrapped means. So, take your pick! However, usually people center it over the estimated mean.
Does the number of random selections to calculate the bootstrap mean, from the sample need to equal the sample size as it does in your example? i.e. could you have chosen 8 random samples from selection and calculated the mean and bootstrapped mean and repeated this 10000 times?
The bootstrap sample is always the same size as the original sample.
Hey Josh, so if I build a confidence interval to the mean in a 95% confidence level, 95% of the bootstrapped means will be in the confidence interval, and there's a 95% chance the mean will be in that confidence interval?
95% of the bootstrapped means will be in the interval, but that doesn't mean there's a 95% chance that the interval covers the true mean.
@@statquest Ok, thanks!
it is brilliant. thanks!
Thanks!
Thank you sir!
You are welcome!
Any chance you could do a video on frequentist confidence intervals, based on the central limit theorem? Also, with the bootstrap method, is the interpretation that you're 95% confidence that the population mean is contained in the interval still valid? Thank you.
Confidence intervals always have the same interpretation. If we repeated the procedure to calculate the CI a bunch of times, 95% of them would overlap the population mean.
Thank you so much for the explanation! i have one question tho - could there be more that a single 95 interval for the example above and does it matter? how do you construct it? thanks!
Any interval that covers 95% of the bootstrapped means qualifies, but usually you select the one that is centered on the original mean.
great!! statquest apps will be a good platform
I think so too!
Hi Josh, a little confused about the p-value here as, if less than 0.05 is considered as less likely to reoccur then why are we considering variables with less than 0.05 as highly significant variables in the regression models?
A p-value < 0.05, means, in general terms, that the result is probably not due to random chance. Thus, when we do linear regression, a small p-value tells us the the relationship between the independent and dependent variable is probably not due to random chance.
@@statquest Thanks Josh 😊😊
Sir, I am looking for a Calculus series. Because I'm going for M.S. in Business Analytics.
3Blue1Brown has an excellent series on calculus. I also believe Khan academy has some good stuff.
3:11
Less the confidence level/interval i.e 80% rather than 90, the more meaningful our inference about the population?
It depends on the data and what you want from it. There's no one specific set of rules that fits all datasets.
Thanks for the video! Are confidence intervals always defined for distributions of the sample means (i.e. means obtained by boodstrapping)? Or can you also calculate them for one single experiment? Or the means of multiple experiments without bootstrapping?
Because of the central limit theorem, all means are normally distributed, so there is a closed form equation for all confidence intervals based on that and you don't need bootstrapping. In other words, you can calculate the CI with a single set of measurements. However, I believe the concept is easier to understand with bootstrapping.
@@statquest I see, thanks!
If all teachers could explain like Josh, more people loving statistics would be
Thanks!
One problem to discuss:
In this video, you said the probability that the "true" mean is in this area has to be < 0.05.
It was my definition of confidence interval before, but after some theoretical statistics courses, my definition has been refereshed. My statistics teacher told me that in this sentence, since neither the "true" mean nor that area is a random variable, this is in fact not a random event and give it a probability is logically wrong.
However, the theoretically correct definition (i.e., after hundreds of repetition, 95% of 95% CIs will include the population mean) is usually not informative by itself for researchers, so the definition in the video got to be more popular in the end.
I'm not sure whether you took all of it into consideration and then made that video, otherwise it may be better to review it if you remake this video in the future!
Please don't treat it as a negative criticism, just because you have released so many helpful videos, I think it would be meaningful if we can help to make them better!
At that time point, we're discussing calculating a p-value with the null hypothesis being H0 < 20. By looking at the confidence interval, we know that the p-value (which is a probability) will be < 0.5 and thus reject the null hypothesis.
Hi Sir, can you make series of videos for Bayesian inference & Bayesian credible interval ?
I hope to do that in the spring.
Excellent!!
Thank you! :)
Can you give some intuition of prediction interval? What is the difference between confidence interval and prediction interval?
PI estimates range of RVs or some statistics of RVs. Its limits are not random (are not drawn from samples). For example for standard norm distribution ~99% PI for RV = mu+-3*sigma. And we must _know_ mu and sigma. On the opposite CI estimates range of not random but exact value - the population parameter (mu for example) though we don't know it's exact value. Its limits are drawn every time from different samples so they are random.
these bootstrap means themselves are normally distributed, right ?
Means are always normally distributed.
@@statquest thanks for the reply, figured this out myself right after watched another video from you after 5 mins yesterday. 😃. You helped me to review my lifetime learned statistic courses in 50 mins and cleared many cloudy concepts in the past. Love you!
@@88skewer bam!
Hi Josh.
Sorry to bother you. I have a simple question. I always get confused by this. May be a dumb question. But want to get it clarified
Its about how probability is realised.
So, sometimes people say, for example in this case , it is said that 95% of the means lie in a certain confidence interval. Here its like speaking in terms of proportions . I understand this . Ok now lets take another statement. Lets take for reference in this video at 3.47, it states that " We know that anything outside of it occurs less than 5% of the time. So here , its speaking in terms of probability or percentage of chance.
So, here is my big confusion.. So , if 5% of the total points (here means) lie outside the confidence interval, how or why it can/is said that " at 3.47, it states that " We know that anything outside of it occurs less than 5% of the time"
Could you please explain me . Thanks in advance
I'm not really sure how to rephrase it. In this case, proportions and probabilities are equal. If 95% of the means end up in a region, then there is 95% chance that if we randomly selected one of the means, it would be within that region.
@@statquest thanks for your instant response. Hope you understood what my. confusion is....Kind of you
Thank you, Josh. Great video. However, I don't know how to calculate the confidence interval. Is it calculated through 2 times the standard deviation of the mean of the sample means?
There are lots of formulas for calculating confidence intervals. Conceptually, the easiest one to remember is bootstrapping, however there are lots of other formulas you can use. For details, see: www.statisticshowto.com/probability-and-statistics/confidence-interval/
@@statquest Thanks! I will check that out. Cheers
How do we calculate 95% cover from the Bootstrap means?
Josh, I love the line diagrams you use in your illustrations, how do you put these together?
For details on how I create my videos, see: th-cam.com/video/crLXJG-EAhk/w-d-xo.html
i'm a little confused, is it true that if you use bootstrapping of a sample that will only tell you with what confidence you can state the mean of that particular sample? wouldnt you need to know the population standard deviation to get the confidence intervals for a sample from the population?
i think i've worked it out, i wasnt adjusting the margin of error in accordance with the sample size ie changing the t/z value, that is when you sample is really small the confidence interval becomes massive to account for that. Either way i would still like to hear your answer if you have time, thanks.
youre a life saver
Thanks!
Very nice man
Thanks! :)
Thank you so much
You're welcome! :)
Dir Sir: I really enjoy watching your videos. Thank you very much! I still have one question regards to 'confidence interval'. Let's say that based on the sample of 12 female mice, we have got the 95% confidence interval is: 200grams +/- 30 grams. One interpretations is: any simple random sample of 12 female mice, its mean weight will be in the range of 170g to 230g, with 95% confidence. I guess if the sample size is 20 (female mice), I cannot say the sample mean will be in this range (170-->230g), with 95% confidence. Correct? (Because the sample size is not 12). Another question: I also intend to say, any random female mouse, its weight will be in this range(170-->230g), with 95% confidence. I cannot say that, correct? Thank you!
Unfortunately, the language surrounding confidence intervals is very tricky. A 95% Confidence Interval should be interpreted like this: "If I re-do the exact same experiment a lot of times and use the exact same method to calculate the 95% confidence interval, 95% of the intervals will cover the population mean". If you're not familiar with the concept of a "population mean", check out my StatQuest on Population Parameters: th-cam.com/video/vikkiwjQqfU/w-d-xo.html
@@statquest Thank you very much for your reply! It is very helpful. I think if the sample size is large enough(12 is not large enough), the sample mean(x-bar) could be treated as population mean(miu). If based on the sample statistics, the 95% confidence interval is 170g-->230g, indeed I can get the density curve and make this statement: the weight of a random female mouse will fall in this range, with 95% confidence. Is this correct?
@@xsli2876 Unfortunately, that statement is not correct. Again, the Confidence Interval only tells us that if we repeated the exact same experiment and used the same method to calculate the confidence intervals, 95% of the intervals would cover the population mean. In other words, the confidence interval tells us about the "mean" and not individual measurements. If you want to make a statement about an individual measurement, like "there is a 95% chance that the weight of a random female mouse will fall within this range", then you need look at the distribution (probably normal distribution), and find the range that 95% of the values fall in. That is the range you are interested in. In math terms, the 95% CI is usually the mean +/- 2 times the standard error. In contrast, for individual measurements, a region where 95% of the measurements fall is the mean +/- 2 times the standard deviation. If you want to learn more about the standard error vs the standard deviation, check out: th-cam.com/video/A82brFpdr9g/w-d-xo.html
@@statquest Thank you so much. I got it. For individual measurements, a region where 95% of the measurements fall is the population mean(miu) +/- 2 times the population standard deviation(sigma). In real life, we don't know miu or sigma. However, if we have one large sample, we may use the sample mean(x-bar) as the approximate population mean, the sample standard deviation(s) as the population standard deviation.
@@xsli2876 Yes that is correct. If the sample size is not very large, you can use a t-distribution (which has fatter tails than the normal distribution) to compensate for the uncertainty in your measurements of the mean and standard deviation. However, as your sample size increases, the t-distribution converges to the normal distribution and then then you can just use the sample mean and sample standard deviation.
Fantastic video :)
Thank you very much!
Is it always true that you don't need any other statistical tests for two distributions with non-overlapping confidence intervals?
Thanks!
Thank you for your wonderful video, here I have a question. When 95% CI do not overlap, we could say there is a significant difference between the two sample sets. I want to ask is we can conclude if the significant difference when the SD of two sets do not overlap, and how about SEM? Hope for your reply. :)
95% confidence intervals reflect the SEM, rather than the standard deviation of the raw data. For more details, see: th-cam.com/video/A82brFpdr9g/w-d-xo.html
What is this original sample?
I think it's just something I made up a long time ago.