Thank you so much for the video. It's far better than the others videos on TH-cam. This made easier to understand the permutation test. You are a very good teacher and explainer.❤️
this is a very good and clear explanation of the idea. I wish I could have watched this first, then half an hour of watching another one could be saved.
Excellent tutorial! I really like the approach to not only explain what a permutation test is but compare it to the t-test. I often struggle when people just present you another test amongst hundreds of other ones but then do not explain why this particular test could be better or worse than the other.
Thank you for this video! For some reason I just couldn't understand what permutation testing was about - turns out its very simple. Great demonstration.
Well done, but..: The assumption of a standard t-test is indeed to have random samples and random assignment, but - on the contrary to what is said here - a permutation test is also based on such assumptions. If there is a bias due to non-random samples and/or assignment, such bias will remain with a permutation t-test. So, why bothering with permutation tests? Actually, a standard t-test also has a number of other assumptions, i.e., the measured scores must follow normal distributions and the variances of the two samples must be equal. If this is the case, then the distribution of the t-value is known. It follows a Student distribution. This is not the case any more with a permutation test and the distribution of the test (here the difference between the two means) is estimated empirically as explained in the video, giving an estimate of the p-value. In other words, permutation tests can be a good choice (among others) if the traits measured is not following a normal distribution and/or if the variances of the two samples are not equal. In this respect, the way things are explained in the video are somewhat misleading.
Eric, thank you for your comment. I hope my video is not misleading or mistaken on the point you make. I don't think it is. The point, made by George Cobb (whose article I used to guide my video), is that the permutation test is a much better fit for the reality of the data set presented in the video. As you point out, the t-test has several important assumptions, such as a normal distribution with equal standard deviations. Cobb (2007) writes regarding the hypothetical example used in the video: "Notice that because the set of observed values is taken as given, there is no need for any assumption about the distribution that generated them." Hence, there is no assumption of normality or equal variances. The data were from a convenience sample, and not randomly drawn from a population. But, the people were randomly assigned to the treatment or control. Again, thanks for your comment - it gave me a good reason to go back and reread the Cobb article (it's listed in the references at the end of the video).
Thank you so much for very useful video. However I just want to clarify. I was confused with the 'repeat'. I initially thought it pertains to the permutation value which was for your example was 35. But later on you assigned 5000 counts on it. Do you just randomly chose amount of repeat? Thanks
Lloyd, I have a concept I'd really like to clarify before my exam next Wednesday. My prof, unfortunately, isn't capable of understanding the core of my question. If we are to rearrange the groupings but the order of the cluster, i.e., person A-G here, doesn't matter, are we not counting combinations? Why is it called a permutation test?
Is there anyway to use permutation for categorical data? For example if I have 1000 samples into 4 categories A to D in one population and another 1000 samples from population B, could I use permutation test to measure if Group A and Group B are significantly different?
Interesting question. I'm not really sure, but I don't see why not. I suggest you read the articles by Cobb and Ernst. The article by Ernst is much more technical in nature and may address your question directly.
Thanks for the great video and explanation In reference to: th-cam.com/video/GmvpsJHGCxQ/w-d-xo.html: One of the reasons you are getting a slightly larger p-value than the author (in the referenced paper) is because you are essentially conducting a two-tailed test by computing the absolute mean difference and comparing to observed mean difference of 9. In the paper, the author conducts a one-sided test and compares the mean differences to the observed mean difference of -9. Thus, the p-value obtained in the paper is a one-sided left-tailed p-value while yours is a two-sided p-value which is slightly higher. I hope this helps.
Thank you for this comment. This is very helpful. I intend to reread the referenced paper. If you are correct, that would explain the difference very well. Much appreciated! Lloyd
Yes, the formula is for number of combinations, not permutations. Practically it doesn't matter (though I agree that it is misleading to say that the formula calculates the number of permutations), you can see an answer regarding this here: stats.stackexchange.com/a/304804/229802
Here are the references shown on the final slide. I accessed the Ernst paper through the UGA library, but unfortunately I don't have copyright permission to share a link to that one. I hope you can access it through your library resources. Cobb, G. (2007). The introductory statistics course: A Ptolemaic Curriculum? Technology Innovations in Statistics Education, 1(1). Available Online: escholarship.org/uc/item/6hb3k0nz Ernst, M.D. (2004). Permutation methods: A basis for exact inference. Statistical Science, 19(4), 676-685.
Permutaion formula is n!/(n-r)! not n!/(n-r)!r! Permutation = arrangement . here order of arrangement matters Combination formula is n!/(n-r)!r! Combination = selection. here order doesnot matter
Thank you so much for the video. It's far better than the others videos on TH-cam. This made easier to understand the permutation test. You are a very good teacher and explainer.❤️
As a PhD. student, I have seen hundreds of videos related statistics and this is one of the excellent ones!
Thanks in advance!
Thank you so much for the video. It's far better than the others videos on TH-cam. This made easier to understand the permutation test. You are a very good teacher and explainer.❤️
This has been very well presented. For a non-statistican like me it has erased a lot of agony.
this is a very good and clear explanation of the idea. I wish I could have watched this first, then half an hour of watching another one could be saved.
Excellent tutorial! I really like the approach to not only explain what a permutation test is but compare it to the t-test. I often struggle when people just present you another test amongst hundreds of other ones but then do not explain why this particular test could be better or worse than the other.
Thanks Lloyd, great explanation for us non-statisticians
man, this video, man, man, man thank you!
Thank you for this video! For some reason I just couldn't understand what permutation testing was about - turns out its very simple. Great demonstration.
Bravo! Big Thanks for the best explanation ever!
Thank you for explaining permutation test so easy to understand!
Thank you so much! You have no idea how much this has helped me.
Nice job explaining the concept with this simple example. Thanks for creating this video.
Great job!! Thank you so much for explaining everything in a simple and straightforward way :)
Thanks Lloyd, the app is a great visualisation
easy way to understand and thank you so much
This was extraordinary 🎉
Now that's easy. Great program. Thanks Lloyd!
Thank you for the video. I have a question does the permutation test work with repeated data.
Interesting question. To be honest, I don't know. My hunch is that it does apply to any test of statistical significance. Thanks for your question.
Very helpful! Thank you!
Great video! Thanks!
Great explanation, much appreciated content!
Excellent simple explanation. Thank you!
Well done, but..: The assumption of a standard t-test is indeed to have random samples and random assignment, but - on the contrary to what is said here - a permutation test is also based on such assumptions. If there is a bias due to non-random samples and/or assignment, such bias will remain with a permutation t-test. So, why bothering with permutation tests? Actually, a standard t-test also has a number of other assumptions, i.e., the measured scores must follow normal distributions and the variances of the two samples must be equal. If this is the case, then the distribution of the t-value is known. It follows a Student distribution. This is not the case any more with a permutation test and the distribution of the test (here the difference between the two means) is estimated empirically as explained in the video, giving an estimate of the p-value. In other words, permutation tests can be a good choice (among others) if the traits measured is not following a normal distribution and/or if the variances of the two samples are not equal. In this respect, the way things are explained in the video are somewhat misleading.
Eric, thank you for your comment. I hope my video is not misleading or mistaken on the point you make. I don't think it is. The point, made by George Cobb (whose article I used to guide my video), is that the permutation test is a much better fit for the reality of the data set presented in the video. As you point out, the t-test has several important assumptions, such as a normal distribution with equal standard deviations. Cobb (2007) writes regarding the hypothetical example used in the video: "Notice that because the set of observed values is taken as given, there is no need for any assumption about the distribution that generated them." Hence, there is no assumption of normality or equal variances. The data were from a convenience sample, and not randomly drawn from a population. But, the people were randomly assigned to the treatment or control. Again, thanks for your comment - it gave me a good reason to go back and reread the Cobb article (it's listed in the references at the end of the video).
I have seen reports with permutation rank as well, what does that tell us about the permutation test?
Great explanation! It´s all very clear now!
This is simply perfect, thanks!
Thank you so much for very useful video. However I just want to clarify. I was confused with the 'repeat'. I initially thought it pertains to the permutation value which was for your example was 35. But later on you assigned 5000 counts on it. Do you just randomly chose amount of repeat? Thanks
EXCELENTE!! Muchas gracias
Fantastic explanation! Thank you!
Really awesome explanation!
Lloyd, I have a concept I'd really like to clarify before my exam next Wednesday. My prof, unfortunately, isn't capable of understanding the core of my question. If we are to rearrange the groupings but the order of the cluster, i.e., person A-G here, doesn't matter, are we not counting combinations? Why is it called a permutation test?
This helped a lot!
It seems the equation you provided for # of permutations is actually the equation for # of combinations?
Great video indeed
this is just great. thanks for this, the app is cute!! and the explanation is good
Thanks this really helped!
excelente video!!!
Thank you
Very nice. Thank you.
Is there anyway to use permutation for categorical data?
For example if I have 1000 samples into 4 categories A to D in one population and another 1000 samples from population B, could I use permutation test to measure if Group A and Group B are significantly different?
Interesting question. I'm not really sure, but I don't see why not. I suggest you read the articles by Cobb and Ernst. The article by Ernst is much more technical in nature and may address your question directly.
brilliant. thank you.
A perfect teacher.:)
Thanks for the great video and explanation
In reference to: th-cam.com/video/GmvpsJHGCxQ/w-d-xo.html: One of the reasons you are getting a slightly larger p-value than the author (in the referenced paper) is because you are essentially conducting a two-tailed test by computing the absolute mean difference and comparing to observed mean difference of 9. In the paper, the author conducts a one-sided test and compares the mean differences to the observed mean difference of -9. Thus, the p-value obtained in the paper is a one-sided left-tailed p-value while yours is a two-sided p-value which is slightly higher. I hope this helps.
Thank you for this comment. This is very helpful. I intend to reread the referenced paper. If you are correct, that would explain the difference very well. Much appreciated! Lloyd
Wouldn't the formula shown be referring to combinations and not permutations?
I have the same question.......anyone?
Yes, the formula is for number of combinations, not permutations. Practically it doesn't matter (though I agree that it is misleading to say that the formula calculates the number of permutations), you can see an answer regarding this here: stats.stackexchange.com/a/304804/229802
Could you link to the paper you're talking about, please?
Here are the references shown on the final slide. I accessed the Ernst paper through the UGA library, but unfortunately I don't have copyright permission to share a link to that one. I hope you can access it through your library resources.
Cobb, G. (2007). The introductory statistics course: A Ptolemaic Curriculum? Technology Innovations in Statistics Education, 1(1). Available Online: escholarship.org/uc/item/6hb3k0nz
Ernst, M.D. (2004). Permutation methods: A basis for exact inference. Statistical Science, 19(4), 676-685.
Thank you very much! I was able to access the Ernst paper through my university library.
Thanks!
Genius
Permutaion formula is n!/(n-r)! not n!/(n-r)!r!
Permutation = arrangement . here order of arrangement matters
Combination formula is n!/(n-r)!r!
Combination = selection. here order doesnot matter
The permutation formula is wrong, that's a combination formula
Thank you so much for the video. It's far better than the others videos on TH-cam. This made easier to understand the permutation test. You are a very good teacher and explainer.❤️