Hi, thanks for your comment. I'm not sure I fully understand your question, but if it is to do with how to select between different models, then it shouldn't matter whether you use R-squared or adjusted R-squared if you only have one variable in your model. Hope that helps! Thanks, Ben
Hi, My view would be to include both of the measures for transparency. You certainly won't be marked down for it. I would even err on the side of perhaps using R-squared rather than adjusted since it has a simple interpretation. Thanks, Ben
This was just a sort of video required for the kickstart of what seems to be a very interesting topic. Can you make a more elaborative video on it or can you share some source for further reading?
Super video, and excellent work in general, you are really doing us all a favor here! I would suggest to mention using a "hold out sample" as a means of investigating overfitting. In a sense, correcting for K in the way the formula suggests is a bit arbitrary, whereas the hold out sample gives a much cleaner way of determining if we are fitting the noise more than the signal. Perhaps introducing cross-validation at this point is beyond the scope, but it is just such a remarkably useful tool that it might be worth at least mentioning it.
Thanks for the reply, my lecturer told me that i can use R^2 for a 1 predictor model, i can use adjusted R^2 to compare the goodness of fit between 2 models if they have different number of independent variables
Thanks for the reply Ben, but if you use excel to perform a regression analysis for any 2 variables which has 1 predictor (or 1 independent variable) you will always get an adjusted R^2 which is smaller than R^2, and i don't know which one to use because they are different. I guess you can argue either way, since it only has 1 independent variable (k=1) u should use R^2, but since im taking sample from a population ( sample size =100) then i should use adjusted R^2 in my report
The example where adj-R^2 decreases from 0.50 to 0.40 is not a good example, as it assumes a sample size of about 5 or 6 observations, which no one will ever seriously use. Adjusted R^2 might be useful in small datasets. In normal datasets, with hundreds and thousands or more observations, the penalty is quite small, so you can end up adding dozens and hundreds of regressors before adj-R^2 will decrease, and then it will be difficult to justify the model. Also, if we are considering say 20 variables as potential controls, and we want to choose which to add using the adj-R^2 thumb rule, then statistically, even if all 20 are rubbish, we will end up adding say 10 of them (due of multiple comparisons problem). (I really love your videos, just this part bothered me)
good day sir, I just wanted to ask if an independent variable is not significant or does not have an explanatory power to the model but when removing it lowers the adjusted r-square what does this imply? so far the reason that i know the reason is because the t-statistic is greater than one. With this information, what can we infer?
should i use r squared or adjusted r squared for 1 variable model in your example it would be wages = constant + b1x edu im thinking since it only has 1 variable i should use r^2
In terms of linear algebra, R^2 is a measure of how far away the data vector is from the hyperplane formed by the regressors. As the number of regressors increases, the distance between the data vector and the hyperplane that lives in Rn must be less than or equal to the distance between the data vector and the hyperplane that lives in R(n-1). Alternatively, R^2 measures the proportion of variation in the dependent variable collectively by all the regressor. As you add more regressors, this fraction can't decrease.
if r squared tells us that a % of variation in y is explained by the variation in x my question is what about the adjusted r squared what does it tell us? and the F what does it tell us ? the sig. also what does it tell us. please i need answers for these questions. thanks
Adjusted R^2 helps us isolate the impact of say, X1 on Y, given that in the model there are X1, X2 and X3. If X3 is some unrelated, uncorrelatable variable which is assumed to affect Y, but it doesnt and yet it is included in themodel, as the no. of X's go up, r2 value too goes up, which makes us think that the effect is due to addition of those unrelated variables too.
the formula in my book says that adjR^2= 1 - ((1-R^2)(n-1)/(n-k)). ie it doesnt include the "-1" in the denominator that you showed in your equation. However, i noticed that you used a capital N rather than the lower case n which is used in my book.
That's because the k in your book equals (k+1) in this video. In your book, k equals the number of unknown coeffs, whereas in this video, k equals the number of regressors (and excludes the constant).
you could have started with what happens to r sqaure when we increase k, then go to adj R square to show how this problem is solved, because that is the root cause
Hi, thanks for your comment. I'm not sure I fully understand your question, but if it is to do with how to select between different models, then it shouldn't matter whether you use R-squared or adjusted R-squared if you only have one variable in your model. Hope that helps! Thanks, Ben
Never understood R2 adj until I watched this video. Thank you so much.
Hi, My view would be to include both of the measures for transparency. You certainly won't be marked down for it. I would even err on the side of perhaps using R-squared rather than adjusted since it has a simple interpretation. Thanks, Ben
This video is amazing! Really helped me understand the concept of Adj-R^2. Thanks
very clear explanation! Thanks
Hi, glad to hear that it helped! All the best, Ben
This was just a sort of video required for the kickstart of what seems to be a very interesting topic. Can you make a more elaborative video on it or can you share some source for further reading?
Super video, and excellent work in general, you are really doing us all a favor here!
I would suggest to mention using a "hold out sample" as a means of investigating overfitting. In a sense, correcting for K in the way the formula suggests is a bit arbitrary, whereas the hold out sample gives a much cleaner way of determining if we are fitting the noise more than the signal. Perhaps introducing cross-validation at this point is beyond the scope, but it is just such a remarkably useful tool that it might be worth at least mentioning it.
Thanks for the reply, my lecturer told me that i can use R^2 for a 1 predictor model, i can use adjusted R^2 to compare the goodness of fit between 2 models if they have different number of independent variables
Amazing Explanation .... Thanks for making life simpler...
Thanks for the reply Ben, but if you use excel to perform a regression analysis for any 2 variables which has 1 predictor (or 1 independent variable) you will always get an adjusted R^2 which is smaller than R^2, and i don't know which one to use because they are different. I guess you can argue either way, since it only has 1 independent variable (k=1) u should use R^2, but since im taking sample from a population ( sample size =100) then i should use adjusted R^2 in my report
The example where adj-R^2 decreases from 0.50 to 0.40 is not a good example, as it assumes a sample size of about 5 or 6 observations, which no one will ever seriously use.
Adjusted R^2 might be useful in small datasets. In normal datasets, with hundreds and thousands or more observations, the penalty is quite small, so you can end up adding dozens and hundreds of regressors before adj-R^2 will decrease, and then it will be difficult to justify the model.
Also, if we are considering say 20 variables as potential controls, and we want to choose which to add using the adj-R^2 thumb rule, then statistically, even if all 20 are rubbish, we will end up adding say 10 of them (due of multiple comparisons problem).
(I really love your videos, just this part bothered me)
Due to which restriction it is "N-1" in the numerator?
good day sir, I just wanted to ask if an independent variable is not significant or does not have an explanatory power to the model but when removing it lowers the adjusted r-square what does this imply? so far the reason that i know the reason is because the t-statistic is greater than one. With this information, what can we infer?
should i use r squared or adjusted r squared for 1 variable model in your example it would be wages = constant + b1x edu im thinking since it only has 1 variable i should use r^2
very nice explanation.can i say that'if the new variable x is complete rubbish,whick makes the adjusted r square decrease?'
Nice Video. However, my question is why would R square increase in the first place with use of additional variables.
In terms of linear algebra, R^2 is a measure of how far away the data vector is from the hyperplane formed by the regressors. As the number of regressors increases, the distance between the data vector and the hyperplane that lives in Rn must be less than or equal to the distance between the data vector and the hyperplane that lives in R(n-1). Alternatively, R^2 measures the proportion of variation in the dependent variable collectively by all the regressor. As you add more regressors, this fraction can't decrease.
This video doesn't explain what R-bar-sqd means. It just goes through the mechanics of what happens to it as "rubbish" variables are added.
very good explanation.
if r squared tells us that a % of variation in y is explained by the variation in x
my question is what about the adjusted r squared what does it tell us? and the F what does it tell us ? the sig. also what does it tell us.
please i need answers for these questions.
thanks
Adjusted R^2 helps us isolate the impact of say, X1 on Y, given that in the model there are X1, X2 and X3. If X3 is some unrelated, uncorrelatable variable which is assumed to affect Y, but it doesnt and yet it is included in themodel, as the no. of X's go up, r2 value too goes up, which makes us think that the effect is due to addition of those unrelated variables too.
the formula in my book says that adjR^2= 1 - ((1-R^2)(n-1)/(n-k)). ie it doesnt include the "-1" in the denominator that you showed in your equation. However, i noticed that you used a capital N rather than the lower case n which is used in my book.
That's because the k in your book equals (k+1) in this video. In your book, k equals the number of unknown coeffs, whereas in this video, k equals the number of regressors (and excludes the constant).
So in other words, _k_ *does* include the intercept, which is *why* you then subtract 1--k-1 does not include the intercept.
it would have been better if you showed how it is calculated using an example from a sample.
you could have started with what happens to r sqaure when we increase k, then go to adj R square to show how this problem is solved, because that is the root cause
Ma man! Ty
Legend