As always, great video Mike. Just wondering why one might use this approach instead of mixed modelling (aka multi-level modelling) with a random intercept. Would the results be identical? Might this "dummy variable" approach be somewhat underpowered with so many (dummy) variables? Or are the CR_SEs adjusted in some way beyond the manual dummy variable approach?
Hi Tim, thank you for visiting and for your question. You are correct that a possible alternative approach to the analysis of this data would have been to use multilevel modeling with a random intercept. My objective in the presentation was to demonstrate a particular analytic possibility, and not so much to advocate for it over MLM given the research scenario I provided. In my own work, I often approach analysis of data such as this from a multilevel perspective. However, I also know there may be circumstances where a multilevel approach may be less than ideal and perhaps a fixed effects approach might offer some benefits. McNeish and Stapleton (2016) highlight the fact that in situations involving low sample size at the cluster level, analysis using MLM runs the risk of biased estimation of variance components and standard errors (for the fixed effects). As a solution, a possibility they offered was to perform MLM using REML estimation with the Kenward-Roger adjustment. On the other hand, if it was the case that my interest during an analysis is solely on modeling Level 1 predictors - i.e., modeling the effect of predictors on the outcome as it occurs solely within clusters (or subjects in the case of longitudinal data) - the fixed effects model does offer greater parsimony and does not require estimation of additional parameters (e.g., variance components) that may not be of substantive interest. In addition, this approach is a viable alternative for folks who may not have a background in multilevel modeling. There is not a lot of space here to lay out a full argument for when to use fixed effects versus MLM (and I'm sure there would be multiple points on which folks might disagree), so I thought I would leave you with the following references (see below) that I have found extremely helpful as I have been learning in this area. A major reason I really started investigating using the fixed effects model was due to students and researchers asking me what they could do when more idealized conditions (such as a large number of clusters) required for multilevel modeling are not present. It also provided me an option to provide for those with no MLM background (I'm a pragmatist at heart). I hope this is helpful to you! And be sure to check out the Powerpoint referenced in the video. Again, thank you for visiting and your question! McNeish, D., Stapleton, L. M., & Silverman, R. D. (2017). On the unnecessary ubiquity of hierarchical linear modeling. Psychological Methods, 22(1), 114-140. doi.org/10.1037/met0000078 McNeish, D. M., & Stapleton, L. M. (2016). The effect of small sample size on two-level model estimates: A review and illustration. Educational Psychology Review, 28(2), 295-314. doi.org/10.1007/s10648-014-9287-x McNeish, D. (2023). A practical guide to selecting and blending approaches for clustered data: Clustered errors, multilevel models, and fixed-effect models. Psychological Methods. Advance online publication. doi.org/10.1037/met0000620
@@mikecrowson2462 Hello Mike. Thanks for the articles. They are great resources and helped me bed down some key concepts I run into a lot with PhD students - the main one being a paucity of clusters in their data. It took me awhile to figure out which approach you were showing in SPSS, as the terminology across methodologists (including McNeish) varies. I can see that you are using a “clustered errors” approach (aka, cluster robust standard errors), as opposed to “fixed effects”. McNeish’s 2023 article was particularly well constructed on this issue and elucidated when and why you would use the various approaches for clustered (and longitudinal) data. I take on board your comment, that you can only fit so much into a short presentation, and so wanted to thank you for taking the time to send through these references.
Thank you so much for the help.
Great! Thank you so much for your help
You are very welcome! Thanks for visiting!
As always, great video Mike. Just wondering why one might use this approach instead of mixed modelling (aka multi-level modelling) with a random intercept. Would the results be identical? Might this "dummy variable" approach be somewhat underpowered with so many (dummy) variables? Or are the CR_SEs adjusted in some way beyond the manual dummy variable approach?
Hi Tim, thank you for visiting and for your question. You are correct that a possible alternative approach to the analysis of this data would have been to use multilevel modeling with a random intercept. My objective in the presentation was to demonstrate a particular analytic possibility, and not so much to advocate for it over MLM given the research scenario I provided. In my own work, I often approach analysis of data such as this from a multilevel perspective. However, I also know there may be circumstances where a multilevel approach may be less than ideal and perhaps a fixed effects approach might offer some benefits. McNeish and Stapleton (2016) highlight the fact that in situations involving low sample size at the cluster level, analysis using MLM runs the risk of biased estimation of variance components and standard errors (for the fixed effects). As a solution, a possibility they offered was to perform MLM using REML estimation with the Kenward-Roger adjustment. On the other hand, if it was the case that my interest during an analysis is solely on modeling Level 1 predictors - i.e., modeling the effect of predictors on the outcome as it occurs solely within clusters (or subjects in the case of longitudinal data) - the fixed effects model does offer greater parsimony and does not require estimation of additional parameters (e.g., variance components) that may not be of substantive interest. In addition, this approach is a viable alternative for folks who may not have a background in multilevel modeling.
There is not a lot of space here to lay out a full argument for when to use fixed effects versus MLM (and I'm sure there would be multiple points on which folks might disagree), so I thought I would leave you with the following references (see below) that I have found extremely helpful as I have been learning in this area. A major reason I really started investigating using the fixed effects model was due to students and researchers asking me what they could do when more idealized conditions (such as a large number of clusters) required for multilevel modeling are not present. It also provided me an option to provide for those with no MLM background (I'm a pragmatist at heart). I hope this is helpful to you! And be sure to check out the Powerpoint referenced in the video. Again, thank you for visiting and your question!
McNeish, D., Stapleton, L. M., & Silverman, R. D. (2017). On the unnecessary ubiquity of hierarchical linear modeling. Psychological Methods, 22(1), 114-140. doi.org/10.1037/met0000078
McNeish, D. M., & Stapleton, L. M. (2016). The effect of small sample size on two-level model estimates: A review and illustration. Educational Psychology Review, 28(2), 295-314. doi.org/10.1007/s10648-014-9287-x
McNeish, D. (2023). A practical guide to selecting and blending approaches for clustered data: Clustered errors, multilevel models, and fixed-effect models. Psychological Methods. Advance online publication. doi.org/10.1037/met0000620
@@mikecrowson2462 Hello Mike. Thanks for the articles. They are great resources and helped me bed down some key concepts I run into a lot with PhD students - the main one being a paucity of clusters in their data. It took me awhile to figure out which approach you were showing in SPSS, as the terminology across methodologists (including McNeish) varies. I can see that you are using a “clustered errors” approach (aka, cluster robust standard errors), as opposed to “fixed effects”. McNeish’s 2023 article was particularly well constructed on this issue and elucidated when and why you would use the various approaches for clustered (and longitudinal) data. I take on board your comment, that you can only fit so much into a short presentation, and so wanted to thank you for taking the time to send through these references.