Do Nudges Work? The Debate over Effectiveness and Reproducibility

Soft paternalism is often defended as a cheap and elegant way to improve behaviour. But how well it works at all is a question over which a sharp scientific dispute broke out in 2022.

The Mertens meta-analysis: nudging "works"

Mertens et al. (2022, PNAS), drawing on 447 effect sizes from 212 publications (n ≈ 2.15 million), found a significant effect of choice architecture: Cohen's d = 0.43 — "small-to-medium" (PNAS). The question seemed settled in favour of nudges.

The correction and the retracted data

But a formal correction to this meta-analysis soon appeared: the sample had included observations from a retracted paper and coding errors (PNAS Correction). The evidence base turned out to be less solid than it looked.

Zero after correcting for bias

Maier, Bartoš, and co-authors (2022, PNAS), using robust Bayesian meta-analysis (RoBMA), showed strong publication bias across all subdomains; once corrected for it, virtually no evidence of the effectiveness of nudges remains (PNAS). In other words, the literature may have systematically published "successes" and buried "nulls."

The gap between academia and nudge units

Independent confirmation came from the field. DellaVigna and Linos (2022, Econometrica), on 126 RCTs (≈23 million people) from the two largest American nudge units, found: the average effect in academic publications was 8.7 percentage points (+33% over control), whereas in real-world programmes it was only 1.4 pp (+8%) (Econometrica). The gap is roughly sixfold, and it is explained by selective publication and the low statistical power of academic studies.

The "one size fits all" myth

The spread of effects is enormous (in the meta-analysis, Cohen's d ranged from −0.69 to 3.08). This casts doubt on the idea of a universal nudge: what works in one country and cohort may fail, or even do harm, in another.

What is left of the evidence base

The dispute is methodological in nature and is not closed: the Mertens authors responded to the criticism. The cautious conclusion is not that nudges are useless, but that their effect is more modest and more context-dependent than the early wave of enthusiasm promised. This is an argument not against discussing soft measures, but against "one size fits all" generalizations.