Thanks for the great video and your very helpful book! May I ask - how exactly would the stata code look to test the "pre-treatment" period effects you are seeing and then plotting them?
The best video on event study analysis I have ever seen. Could you please also explain the DID estimator proposed by de chaisemartin and d'haultfoeuille (2020) in a separate video. I am struggling to understand the paper.
@@SouravDas-iw6ej no need to wait! Here's the video th-cam.com/video/hu2nDbnpALA/w-d-xo.html And here's the chapter section www.theeffectbook.net/ch-DifferenceinDifference.html#doing-multiple-treatment-periods-right
Hi mate. Not sure if you are still using platform, however, I have a question for you. For a job application assignment I need to us diff-in-diff to estimate the impact of a job training programme on log(employment). For context, in this hypothetical scenario, local government either roll out the programme (treatment group) or not (control group). I have data on log(employment) and log(population) for 3 periods. Two pre treatment periods (parallel trends assumption holds) and one post treatment period. My current regression looks like Δlog(employment) = time fixed effects + group fixed effects + DiD dummy + log(emp) + ε. Does it make sense to add an interaction term for each local government and the DID dummy to capture the heterogeneity in treatment effects for each treated unit?
I would probably make log(employment) the dependent variable and leave it off the right side, but your version works too I think. I'm assuming treatment is applied to all cases at the same time - if treatment is staggered then the typical TWFE setup doesn't work. If you add an interaction term for each local government, then in effect what you're doing is running a separate DID for each local government vs. the same control group. There's nothing inherently wrong with this, although keep in mind that your effective sample size for each of the DIDs will be much smaller (i.e. do you have enough sample to actually be able to look at each effect separately) and you now have *many* parallel trends assumptions to investigate - one for each treated government - rather than just one.
@@NickHuntingtonKlein Many thanks for the response, Nick. L(emp) on the right side was a typo and meant to be l(pop). And yes, its a one-off treatment, so static TWFE DiD works. I ended up dropping the interaction term based on what you said. However, I controlled for treatment and post treatment, and the DiD control was the interaction between the two. Anything inherently wrong with this?
Hello Nick, Thank you for your video. I would like to ask you about the event study design. My dataset is monthly panel between 2013 and 2018. My shock starts from Jan 2016. Should I put all time periods except the Dec 2015 in the event study? This approach makes my all post-periods insignificant and even pre-periods signficant, leading to pre-trend problem. As you guessed 60 coefficient might have created overparameterization in my regression. Some papers are going back to just 12 months before the shock even though their dataset lies to 24 or 36 months back. Does make sense for you? Best,
I don't think overparameterization should be an issue as long as you have plenty of observations per period / group (and if you don't, then this might not be the model for you). Going back farther gives you more data which is good, as long as you don't go back so far that the data no longer is representative of the period you're interested in. Significant pre periods does make parallel trends less plausible. That's a feature of the data though, not something that can really be fixed with modeling! You might try a method like HonestDID (Roth) which allows for parallel trend violations and lets you bound the effect.
Thank you @@NickHuntingtonKlein for your quick reply. I'm going to give a try to HonestDID. I have another question btw. In the monthly panel data, how should I control the seasonality? Does using firm x month + time FEs make sense for you? Or firm + month + time FEs instead?
Wold you have a R code for this? Also in the case I have the treatment happening in multiple time periods? Also just a few groups are treated each time so the treatment happen in different time points for each group of groups. I want to observe the effect in each city which in my case is each group of companies being treated in different time periods.
@@RightAIopen That's odd, those packages are for the exact situation you described. Why didn't they work? (Unless you mean that your treatments turn on *and then off* at different times for different groups).
@@NickHuntingtonKlein They do, but even if I consider forever treated I have another problem. The DID report shows as an effect of the treatment the "group" effect. However this groups are not the cities with companies, they are the cities treated in the same year. Although I specify that the ID is the city group I can not extract the city effect. The grouping of cities treated together doesn't hold much relevance for my study - the focus is more on the individual cities or companies themselves.
@RightAIopen oh I see, easy fix then. Just do a separate DID estimation for each treated city, dropping all the other cities already treated at the time. You'll be dropping a lot of data for each estimation so your standard errors will be bigger but this design works (and you're inherently doing something noisier by getting individual effects, so bigger SEs make sense). Or if you have enough pre-treatment data, synthetic control instead of DID would be better.
interesting explanation. Can you tell me if there is a way to observe the difference in effects between a treatment group and a control group using the difference-in-differences (DiD) method? For example, within the treatment group, I want to know if there are differences in the impact of treatment between men and women. Should I separate the samples by gender, or is there another method? Thank you.
@@hanifabd.43 Yep! Just use any of the methods for allowing a treatment effect to differ across some characteristic. The simplest of which would be just taking our basic DID setup and splitting the Treated indicator into two separate groups, Treated Men and Treated Women.
@@NickHuntingtonKlein so the model will be: Y_it = α + β1Treat_i + β2Post_t + β3(Treat_iPost_t) + β4Male_i + β5(Treat_iMale_i) + ε_it still need more time to understand how interaction terms works.
There are links to no-background-music versions in the description. People like the stock footage, it breaks things up, I don't really get it either though
Thank you, it is very succinct and helpful
Does anyone have any good papers to recommend that use this method? I'd like to see it in practice a few times!
amazing video
Thanks for the great video and your very helpful book! May I ask - how exactly would the stata code look to test the "pre-treatment" period effects you are seeing and then plotting them?
Thanks! See this section of the book
www.theeffectbook.net/ch-DifferenceinDifference.html#long-term-effects
The best video on event study analysis I have ever seen. Could you please also explain the DID estimator proposed by de chaisemartin and d'haultfoeuille (2020) in a separate video. I am struggling to understand the paper.
Thanks! No plans at the moment to cover that paper, but I do go over a few of the new estimators in my chapter including Callaway and Sant'Anna
@@NickHuntingtonKlein Great! I will be waiting for that
@@SouravDas-iw6ej no need to wait! Here's the video th-cam.com/video/hu2nDbnpALA/w-d-xo.html
And here's the chapter section www.theeffectbook.net/ch-DifferenceinDifference.html#doing-multiple-treatment-periods-right
@@NickHuntingtonKlein Wow! That's cool. Thanks a lot
Hi mate. Not sure if you are still using platform, however, I have a question for you.
For a job application assignment I need to us diff-in-diff to estimate the impact of a job training programme on log(employment).
For context, in this hypothetical scenario, local government either roll out the programme (treatment group) or not (control group). I have data on log(employment) and log(population) for 3 periods. Two pre treatment periods (parallel trends assumption holds) and one post treatment period.
My current regression looks like Δlog(employment) = time fixed effects + group fixed effects + DiD dummy + log(emp) + ε.
Does it make sense to add an interaction term for each local government and the DID dummy to capture the heterogeneity in treatment effects for each treated unit?
I would probably make log(employment) the dependent variable and leave it off the right side, but your version works too I think. I'm assuming treatment is applied to all cases at the same time - if treatment is staggered then the typical TWFE setup doesn't work.
If you add an interaction term for each local government, then in effect what you're doing is running a separate DID for each local government vs. the same control group. There's nothing inherently wrong with this, although keep in mind that your effective sample size for each of the DIDs will be much smaller (i.e. do you have enough sample to actually be able to look at each effect separately) and you now have *many* parallel trends assumptions to investigate - one for each treated government - rather than just one.
@@NickHuntingtonKlein Many thanks for the response, Nick. L(emp) on the right side was a typo and meant to be l(pop). And yes, its a one-off treatment, so static TWFE DiD works. I ended up dropping the interaction term based on what you said. However, I controlled for treatment and post treatment, and the DiD control was the interaction between the two. Anything inherently wrong with this?
@@CharlieDickens-j4m seems fine to me! Standard DID
@@NickHuntingtonKlein Legend. Thanks for the taking the time out of your day to reply!
Hello Nick,
Thank you for your video. I would like to ask you about the event study design.
My dataset is monthly panel between 2013 and 2018. My shock starts from Jan 2016. Should I put all time periods except the Dec 2015 in the event study? This approach makes my all post-periods insignificant and even pre-periods signficant, leading to pre-trend problem. As you guessed 60 coefficient might have created overparameterization in my regression. Some papers are going back to just 12 months before the shock even though their dataset lies to 24 or 36 months back. Does make sense for you? Best,
I don't think overparameterization should be an issue as long as you have plenty of observations per period / group (and if you don't, then this might not be the model for you).
Going back farther gives you more data which is good, as long as you don't go back so far that the data no longer is representative of the period you're interested in.
Significant pre periods does make parallel trends less plausible. That's a feature of the data though, not something that can really be fixed with modeling! You might try a method like HonestDID (Roth) which allows for parallel trend violations and lets you bound the effect.
Thank you @@NickHuntingtonKlein for your quick reply. I'm going to give a try to HonestDID. I have another question btw. In the monthly panel data, how should I control the seasonality? Does using firm x month + time FEs make sense for you? Or firm + month +
time FEs instead?
My outcome variables, I forgot to say, are the average wage and total sales
@@uguraytun I don't think that changes it
Thank you! @@NickHuntingtonKlein
Wold you have a R code for this? Also in the case I have the treatment happening in multiple time periods? Also just a few groups are treated each time so the treatment happen in different time points for each group of groups. I want to observe the effect in each city which in my case is each group of companies being treated in different time periods.
In R the Callaway and Sant'Anna method can be performed using the did package, or the Wooldridge method can be peformed using the etwfe package.
@@NickHuntingtonKlein Thank you so much for your reply and all those amazing videos. I tried both packages no success so far. Best
@@RightAIopen That's odd, those packages are for the exact situation you described. Why didn't they work? (Unless you mean that your treatments turn on *and then off* at different times for different groups).
@@NickHuntingtonKlein They do, but even if I consider forever treated I have another problem. The DID report shows as an effect of the treatment the "group" effect. However this groups are not the cities with companies, they are the cities treated in the same year. Although I specify that the ID is the city group I can not extract the city effect. The grouping of cities treated together doesn't hold much relevance for my study - the focus is more on the individual cities or companies themselves.
@RightAIopen oh I see, easy fix then. Just do a separate DID estimation for each treated city, dropping all the other cities already treated at the time. You'll be dropping a lot of data for each estimation so your standard errors will be bigger but this design works (and you're inherently doing something noisier by getting individual effects, so bigger SEs make sense).
Or if you have enough pre-treatment data, synthetic control instead of DID would be better.
interesting explanation. Can you tell me if there is a way to observe the difference in effects between a treatment group and a control group using the difference-in-differences (DiD) method? For example, within the treatment group, I want to know if there are differences in the impact of treatment between men and women. Should I separate the samples by gender, or is there another method? Thank you.
Sorry, I mean difference in effect within a treatment group.
@@hanifabd.43 Yep! Just use any of the methods for allowing a treatment effect to differ across some characteristic. The simplest of which would be just taking our basic DID setup and splitting the Treated indicator into two separate groups, Treated Men and Treated Women.
@@NickHuntingtonKlein thank you for your response. is it possible to use basic DiD model, then I include dummy for gender?
@@hanifabd.43 Not just a dummy by itself, but interacted, sure.
@@NickHuntingtonKlein so the model will be:
Y_it = α + β1Treat_i + β2Post_t + β3(Treat_iPost_t) + β4Male_i + β5(Treat_iMale_i) + ε_it
still need more time to understand how interaction terms works.
Feedback: If you want to teach people, refrain from using the background music (in a playlist), and remove the stock footage clips.
There are links to no-background-music versions in the description. People like the stock footage, it breaks things up, I don't really get it either though