Very interesting session as always. 👏 I enjoy and learn a lot from your "thinking aloud". I particularly appreciate how you went about sequentially comparing the models, and was able to follow your steps. Keep up the great work! It's needed out here for sure. ✨ 👍
Thanks for that! Let me ask you something, I was running some mixed model and turns out that a have numerical predictor that has a non-linear relationship with my outcome variable. Thus, I used the function gamm from mgcv package. However, when I tried to use model.comparison from flexplot it didn't work 😢. Do you have any tips? I hope in the futuro I can do some of your courses, but no money at moment 😅
Thank you, Dustin, for another great video :)) A couple of comments: 1 - What I've learned, while doing a lot of model comparisons (lmer, glmer, but also gamm), is that BIC (and BF, which are based on BIC approximation) offer the strongest penalisation against complexity (thus often favouring simpler models, especially when N is large), whereas AIC is a more liberal metric, which often proposes you the most complex model. Sometimes this is very frustrating, especially when some of the parameters added in the most complex model turn out to be significant. They are, but BIC and BF still tell you "not enough, dude!". I don't think there are other, more clear-cut metrics, tho, maybe it just is what it is...what do you think, Dustin? 2 - When I compare mixed models varying in their fixed part, I refit them with ML (i.e., REML=F) prior to model comparison (they say REML is ok only for models varying in their random part). That's because, for some reason, the AIC and BIC values that I obtain in this way are different from those obtained via flexplot::model.comparison. I wonder why that happens...yes, flexplot refits the models with ML, but 1) for some reason, the values are different, 2) it does this in a few seconds, while rerunning the model with REML=F takes waaaay longer (and in my silly mind, longer means more accurate XD). I might be wrong, but I still wanted to point this out...maybe this could be improved? I hope these comments are helpful :)) Cheers, guys!!
Would using something like the `brm` function from the brms package help make our inferences? `brm` acts similarly to `lmer` but instead of getting single points, you get distributions. Aside from how cool it looks my gut feeling is that it kind of helps because you can see how stretched out the distribution is compared to your other effects. Do you think it makes a difference to use brm over lmer for mixed models? (assuming we don't care about how long it takes to run the functions)
my friend, brm is not merely "a function" from the brms package, it's the Bayesian way of fitting models! The fact that you "get distributions" for your parameters is not due to "a function", it's because Bayesian and frequentist statistics have two completely opposite ways when it comes to interpreting the data and parameters. Long story short: for frequentists, data is random and parameters are fixed; for Bayesians, data is fixed and parameters are random! That's why you have a whole posterior distribution for each of them, because parameters are random variables! As concerns the stretchedness of the posterior distributions, it depends on several factors: 1) stretchedness of the prior distribution (i.e., in Bayesian stats, you inform the model on where you think the parameter is, and you can be very precise or very vague about it; this will heavily affect the posterior distribution, 2) N, 3) multicollinearity (higher > more stretched posterior), and I'm sure I'm forgetting a lot of other reasons XD
Very interesting session as always. 👏 I enjoy and learn a lot from your "thinking aloud".
I particularly appreciate how you went about sequentially comparing the models, and was able to follow your steps.
Keep up the great work! It's needed out here for sure. ✨ 👍
Thanks for that! Let me ask you something, I was running some mixed model and turns out that a have numerical predictor that has a non-linear relationship with my outcome variable. Thus, I used the function gamm from mgcv package. However, when I tried to use model.comparison from flexplot it didn't work 😢. Do you have any tips? I hope in the futuro I can do some of your courses, but no money at moment 😅
Thank you, Dustin, for another great video :))
A couple of comments:
1 - What I've learned, while doing a lot of model comparisons (lmer, glmer, but also gamm), is that BIC (and BF, which are based on BIC approximation) offer the strongest penalisation against complexity (thus often favouring simpler models, especially when N is large), whereas AIC is a more liberal metric, which often proposes you the most complex model. Sometimes this is very frustrating, especially when some of the parameters added in the most complex model turn out to be significant. They are, but BIC and BF still tell you "not enough, dude!". I don't think there are other, more clear-cut metrics, tho, maybe it just is what it is...what do you think, Dustin?
2 - When I compare mixed models varying in their fixed part, I refit them with ML (i.e., REML=F) prior to model comparison (they say REML is ok only for models varying in their random part). That's because, for some reason, the AIC and BIC values that I obtain in this way are different from those obtained via flexplot::model.comparison. I wonder why that happens...yes, flexplot refits the models with ML, but 1) for some reason, the values are different, 2) it does this in a few seconds, while rerunning the model with REML=F takes waaaay longer (and in my silly mind, longer means more accurate XD). I might be wrong, but I still wanted to point this out...maybe this could be improved? I hope these comments are helpful :)) Cheers, guys!!
Great video!
Would using something like the `brm` function from the brms package help make our inferences? `brm` acts similarly to `lmer` but instead of getting single points, you get distributions. Aside from how cool it looks my gut feeling is that it kind of helps because you can see how stretched out the distribution is compared to your other effects. Do you think it makes a difference to use brm over lmer for mixed models? (assuming we don't care about how long it takes to run the functions)
my friend, brm is not merely "a function" from the brms package, it's the Bayesian way of fitting models! The fact that you "get distributions" for your parameters is not due to "a function", it's because Bayesian and frequentist statistics have two completely opposite ways when it comes to interpreting the data and parameters. Long story short: for frequentists, data is random and parameters are fixed; for Bayesians, data is fixed and parameters are random! That's why you have a whole posterior distribution for each of them, because parameters are random variables! As concerns the stretchedness of the posterior distributions, it depends on several factors: 1) stretchedness of the prior distribution (i.e., in Bayesian stats, you inform the model on where you think the parameter is, and you can be very precise or very vague about it; this will heavily affect the posterior distribution, 2) N, 3) multicollinearity (higher > more stretched posterior), and I'm sure I'm forgetting a lot of other reasons XD