-
-
Notifications
You must be signed in to change notification settings - Fork 38
/
27-changes-in-changes.Rmd
231 lines (184 loc) · 8.36 KB
/
27-changes-in-changes.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
# Changes-in-Changes
- **Introduction**
- The Changes-in-Changes (CiC) estimator, introduced by @athey2006identification, is an alternative to the Difference-in-Differences (DiD) strategy.
- Unlike traditional DiD, which estimates the Average Treatment Effect on the Treated (ATT), CiC focuses on the Quantile Treatment Effect on the Treated (QTT).
- QTT captures the difference between potential outcome distributions for treated units at a specific quantile.
- **Beyond Averages:** Policymakers often look beyond average program impacts, considering how benefits are distributed across different groups.
- **Job Training Example:** Two programs with the same negative average impact may be treated differently: one benefiting high earners might be rejected, while one benefiting low earners could be approved.
- **Traditional Methods' Limitations:** Methods like linear regression, which assume uniform effects, fail to capture important distributional differences.
- **QTEs' Advantage:** QTE methods are tailored for analyzing how treatment effects vary across different segments of a population.
- **QTE vs. ATE:** While QTEs provide detailed insights into distributional impacts, they also allow for the recovery of ATEs. However, ATEs are usually identified under weaker assumptions, making QTEs more suitable for exploring the shape of treatment effects rather than just their central tendency.
- **Key Concepts**
- **Quantile Treatment Effect on the Treated (QTT):** Difference in quantiles of treated units' potential outcome distributions.
- **Rank Preservation:** Assumes each unit's rank remains constant across potential outcome distributions---this is a strong assumption.
- **Counterfactual Distribution:** Estimation focuses on determining this distribution for the treated units in period 1.
- **Estimating QTT**
- CiC uses four distributions from a 2x2 DiD design:
1. $F_{Y(0),00}$: CDF of $Y(0)$ for control units in period 0.
2. $F_{Y(0),10}$: CDF of $Y(0)$ for treatment units in period 0.
3. $F_{Y(0),01}$: CDF of $Y(0)$ for control units in period 1.
4. $F_{Y(1),11}$: CDF of $Y(1)$ for treatment units in period 1.
- QTT is defined as the difference between the inverses of $F_{Y(1),11}$ and the counterfactual distribution $F_{Y(0),11}$ at quantile $q$:
$$
\Delta_\theta^{QTT} = F_{Y(1), 11}^{-1} (\theta) - F_{Y (0), 11}^{-1} (\theta)
$$
- **Estimation Process**
- **Counterfactual CDF:**
$$
\hat{F}_{Y(0),11}(y) = F_{y,01}\left(F^{-1}_{y,00}\left(F_{y,10}(y)\right)\right)
$$
- **Equivalent Expression:**
$$
\hat{F}^{-1}_{Y(0),11}(\theta) = F^{-1}_{y,01}\left(F_{y,00}\left(F^{-1}_{y,10}(\theta)\right)\right)
$$
- **Treatment Effect Estimate:**
$$
\hat{\Delta}^{CIC}_{\theta} = F^{-1}_{Y(1),11}(\theta) - \hat{F}^{-1}_{Y(0),11}(\theta)
$$
- **Equivalently:**
$\Delta^{CIC}_{\theta}$ is the difference between two QTE estimates:
$$
\Delta^{CIC}_{\theta} = \Delta^{QTE}_{\theta,1} - \Delta^{QTE}_{\theta',0}
$$
where:
- $\Delta^{QTT}_{\theta,1}$ = change over time in $y$ at quantile $\theta$ for $D = 1$ group.
- $\Delta^{QTU}_{\theta',0}$ = change over time in $y$ at quantile $\theta'$ for $D = 0$ group, where $q'$ is the quantile in the $D = 0, T = 0$ distribution corresponding to the value of $y$ associated with quantile $\theta$ in the $D = 1, T = 0$ distribution.
- **Marketing Example**
- Suppose a company implements a new online marketing strategy aimed at improving customer retention rates.
- **QTT:** The goal is to estimate the effect of the strategy on customer retention rates at different quantiles (e.g., median retention rate).
- **Rank Preservation:** Assumes customers' rank in retention distribution remains the same, regardless of the strategy---this assumption is strong and should be carefully considered.
- **Counterfactual:** CiC helps estimate how retention rates would have changed without the new strategy by comparing it with a control group.
- **References**
- @athey2006identification
- @frolich2013unconditional: IV-based
- @callaway2019quantile: panel data
- @huber2022direct
- **Additional Resources**
- Code examples available in [Stata](https://sites.google.com/site/blaisemelly/home/computer-programs/cic_stata).
## Application
### ECIC package
```{r}
library(ecic)
data(dat, package = "ecic")
mod =
ecic(
yvar = lemp, # dependent variable
gvar = first.treat, # group indicator
tvar = year, # time indicator
ivar = countyreal, # unit ID
dat = dat, # dataset
boot = "weighted", # bootstrap proceduce ("no", "normal", or "weighted")
nReps = 3 # number of bootstrap runs
)
mod_res <- summary(mod)
mod_res
ecic_plot(mod_res)
```
### QTE package
```{r}
library(qte)
data(lalonde)
# randomized setting
# qte is identical to qtet
jt.rand <-
ci.qtet(
re78 ~ treat,
data = lalonde.exp,
iters = 10
)
summary(jt.rand)
ggqte(jt.rand)
```
```{r}
# conditional independence assumption (CIA)
jt.cia <- ci.qte(
re78 ~ treat,
xformla = ~ age + education,
data = lalonde.psid,
iters = 10
)
summary(jt.cia)
ggqte(jt.cia)
jt.ciat <- ci.qtet(
re78 ~ treat,
xformla = ~ age + education,
data = lalonde.psid,
iters = 10
)
summary(jt.ciat)
ggqte(jt.ciat)
```
- **QTE** compares quantiles of the entire population under treatment and control, whereas **QTET** compares quantiles within the treated group itself. This difference means that QTE reflects the overall population-level impact, while QTET focuses on the treated group's specific impact.
- **CIA** enables identification of both QTE and QTET, but since QTET is conditional on treatment, it might reflect different effects than QTE, especially when the treatment effect is heterogeneous across different subpopulations. For example, the QTE could show a more generalized effect across all individuals, while the QTET may reveal stronger or weaker effects for the subgroup that actually received the treatment.
These are DID-like models
1. With the distributional difference-in-differences assumption [@fan2012partial, @callaway2019quantile], which is an extension of the parallel trends assumption, we can estimate QTET.
```{r}
# distributional DiD assumption
jt.pqtet <- panel.qtet(
re ~ treat,
t = 1978,
tmin1 = 1975,
tmin2 = 1974,
tname = "year",
idname = "id",
data = lalonde.psid.panel,
iters = 10
)
summary(jt.pqtet)
ggqte(jt.pqtet)
```
2. With 2 periods, the distributional DiD assumption can partially identify QTET with bounds [@fan2012partial]
```{r}
res_bound <-
bounds(
re ~ treat,
t = 1978,
tmin1 = 1975,
data = lalonde.psid.panel,
idname = "id",
tname = "year"
)
summary(res_bound)
plot(res_bound)
```
3. With a restrictive assumption that difference in the quantiles of the distribution of potential outcomes for the treated and untreated groups be the same for all values of quantiles, we can have the mean DiD model
```{r}
jt.mdid <- ddid2(
re ~ treat,
t = 1978,
tmin1 = 1975,
tname = "year",
idname = "id",
data = lalonde.psid.panel,
iters = 10
)
summary(jt.mdid)
plot(jt.mdid)
```
On top of the distributional DiD assumption, we need **copula stability** assumption (i.e., If, before the treatment, the units with the highest outcomes were improving the most, we would expect to see them improving the most in the current period too.) for these models:
| **Aspect** | **QDiD** | **CiC** |
|------------------------|-----------------------|-------------------------|
| **Treatment of Time and Group** | Symmetric | Asymmetric |
| **QTET Computation** | Not inherently scale-invariant | Outcome Variable Scale-Invariant |
```{r, eval = FALSE}
jt.qdid <- QDiD(
re ~ treat,
t = 1978,
tmin1 = 1975,
tname = "year",
idname = "id",
data = lalonde.psid.panel,
iters = 10,
panel = T
)
jt.cic <- CiC(
re ~ treat,
t = 1978,
tmin1 = 1975,
tname = "year",
idname = "id",
data = lalonde.psid.panel,
iters = 10,
panel = T
)
```