Evaluating a programme with a staggered rollout
Two groups introduce a fictional employment programme at different times. With a third group as a comparison, estimate the first two quarters of impact and see which assumptions make the result interpretable.
The programme and its comparison group
All values below are synthetic. The outcome is the employment rate among eligible residents. An early cohort enters the programme in quarter 4, a late cohort in quarter 6, and a third cohort remains outside the programme throughout. Each plotted series is a cohort average; there are no individual records.
| Quarter | Early | Late | Never treated |
|---|---|---|---|
| Q1 | 46 | 51 | 49 |
| Q2 | 47 | 52 | 50 |
| Q3 | 48 | 53 | 51 |
| Q4 | 51 | 54 | 52 |
| Q5 | 53 | 55 | 53 |
| Q6 | 55 | 58 | 54 |
| Q7 | 56 | 60 | 55 |
Calculate the effect for each cohort
Use the quarter before treatment as the baseline. For each post-treatment quarter, subtract the change in the never-treated cohort over the same calendar quarters from the change in the treated cohort.
- Early cohort, first quarter: (51 − 48) − (52 − 51) = 2 percentage points.
- Late cohort, first quarter: (58 − 55) − (54 − 53) = 2 percentage points.
- Second quarter: the early comparison is (53 − 48) − (53 − 51) = 3 points; the late comparison is (60 − 55) − (55 − 53) = 3 points.
Keep the cohort estimates visible. If you combine them in a real evaluation, choose weights that match your question and report the cohorts contributing at each time since rollout.
What would change the conclusion?
Interpretation depends on whether the treated cohorts would have followed the comparison trend without the programme. The pre-period is a useful check, but parallel pre-trends do not establish that assumption. An economic shock specific to one cohort, changes in who is counted, anticipation before rollout or spillovers into the comparison cohort would all make the estimate harder to interpret.
These group averages illustrate the arithmetic. They do not contain enough information to quantify sampling uncertainty. A real study needs appropriate unit-level or cluster-level information and uncertainty estimates that account for the rollout design.
Method reference: Callaway and Sant’Anna (2021), Difference-in-Differences with Multiple Time Periods. The data and arithmetic here are VCESR teaching materials.