Synthetic control methods

The single-treated-unit problem

Difference-in-differences needs a comparison group whose trend stands in for the treated group’s counterfactual. But many of the most consequential interventions arrive at the level of one aggregate unit: one county adopts a levy, one country reforms a tariff, one region pilots a health-financing change. With a single treated unit there is no averaging away of idiosyncratic shocks, and the choice of which untreated unit to compare against becomes both decisive and arbitrary — pick a different neighbour, get a different answer [1, 3].

Synthetic control replaces that arbitrary choice with a transparent, data-disciplined one: instead of selecting one comparison unit, build a weighted combination of several — the synthetic control — with weights chosen so that the combination reproduces the treated unit’s outcome trajectory and key predictors over the pre-intervention period [1]. No single county behaves like the treated county; a blend of three or four often does. The estimated effect is then the divergence between the treated unit and its synthetic twin after the intervention.

Two properties recommend the method beyond necessity. The weights are non-negative and sum to one, so the synthetic is an interpolation — it never extrapolates outside the range of the donor units — and the recipe is published: anyone can see that the synthetic is, say, 42 % County A, 27 % County B, and judge whether that blend is a sensible stand-in [1].

Building the synthetic

The donor pool. Candidate comparison units must be untreated, plausibly driven by the same structural processes as the treated unit, and free of their own large idiosyncratic shocks during the study window. Exclude units exposed to spillovers from the intervention — a neighbouring county absorbing diverted trade is not a valid donor — and units that adopted similar policies of their own. A smaller, well-argued donor pool beats a large indiscriminate one: every unit in the pool is a claim that it could plausibly have stood in for the treated unit [1].

The weights. Weights are estimated so the synthetic matches the treated unit’s pre-intervention outcomes and a set of predictors of those outcomes. The mechanics matter less to a commissioning evaluator than the discipline: the weights are fitted only on pre-intervention information, then frozen. Post-intervention data play no role in constructing the counterfactual — which is what separates the method from curve-fitting after the fact [1].

The credibility test. Everything rests on pre-intervention fit over a long pre-period. A synthetic that tracks the treated unit closely through years of ups and downs has demonstrated, out of sample after out of sample, that it moves for the same reasons. A synthetic that only roughly approximates the pre-period, or that fits a short pre-period, proves nothing — with few pre-intervention points, a close fit can be achieved by chance, and the post-intervention gap inherits that fragility [1].

Inference by placebo

With one treated unit, conventional standard errors have nothing to average over, so the method’s inference is permutation-based: run the identical procedure as if each donor unit had been treated, and compare. If the real treated unit’s post-intervention gap is large relative to the distribution of these in-space placebo gaps — especially among placebos whose pre-period fit was comparably good — the effect is unlikely to be an artefact of the method. In-time placebos apply the same logic on the other axis: pretend the intervention happened years earlier and confirm no effect appears where none should [1].

Report the placebo distribution graphically, not just a p-value: a spaghetti plot of all placebo gaps with the treated unit’s gap overlaid lets readers judge the claim with their own eyes.

Feasibility: when not to use it

Abadie’s own methodological survey is unusually direct about the contexts where the method should not be applied. The checks that matter [1]:

• Effect size vs volatility. If the outcome series is highly volatile, a real but modest effect will drown; synthetic control is for effects large enough to stand out from the noise the pre-period reveals.
• Pre-period length. A long pre-intervention window is required both to fit credible weights and to demonstrate the fit. Short pre-periods invite overfitting — the central caution for devolution-era county comparisons in Kenya, where consistent county-level series often begin only in the mid-2010s.
• Adequate donors. Enough untreated, unshocked, spillover-free units must exist, and the treated unit’s pre-period values must lie within the range the donors span — the no-extrapolation property is a feature only when interpolation is possible.
• No anticipation. If actors adjusted behaviour before the formal start date, date the intervention at the earlier point or the pre-period fit is contaminated.
• Aggregate data discipline. The method runs on unit-level aggregates (county revenue, regional coverage rates), so the series’ definitions must be stable across units and years — the routine-data caveats from the interrupted time series page apply with full force.

The relationship to difference-in-differences is close and worth stating in reports: synthetic control replaces DiD’s parallel-trends assumption with a demonstrated pre-period match, and the modern econometrics literature treats the two as members of one family, with hybrid estimators between them [2]. Where several treated units exist, that family — not the single-unit method — is usually the better home.

Development-sector fit

The method’s canonical applications — the economic cost of conflict in the Basque Country, California’s 1988 tobacco-control programme — are reviewed in Abadie’s survey, and their shape recurs in development practice: one jurisdiction adopts a policy the rest have not [1]. Devolved-government settings are the natural use case: one county introduces a health levy, a fee waiver, an agricultural subsidy, and the remaining counties supply the donor pool. National statistical series and cross-country databases play the same role for country-level reforms; the national M&E systems page in the systems cluster covers the institutional data infrastructure such analyses lean on. The binding constraints in these settings are honest ones: short consistent series, definitional drift across counties, and donor pools thinned by copy-cat policies — all detectable, all reportable, and all reasons to say “this question cannot yet be answered with this method” when they bind [1].

Checklist before you commit to synthetic control

• One (or very few) aggregate treated units; DiD with a defensible comparison group is unavailable.
• A long, definitionally stable pre-intervention series exists for the treated unit and all donors.
• The donor pool is argued unit by unit: untreated, no spillovers, no own shocks, no similar policies.
• The synthetic’s pre-intervention fit is close over the full pre-period, and the weights are published.
• Inference uses in-space and in-time placebos, shown graphically.
• The expected effect is large enough, relative to the series’ volatility, to be detectable at all.