Impact Evaluation and the Broader Evidence Agenda since the 2000s

In the quest to improve social and economic well-being with limited resources, timely and high-quality evidence on performance and outcomes is indispensable.

Policymakers, program managers, and funders need different types of evidence from multiple sources to inform the design, implementation, and evaluation of a policy, program, or intervention. One approach known as impact evaluation establishes the attributable net impact, making it uniquely well suited to answer the key question of what a policy or program is achieving.

The goal of impact evaluation is to rigorously identify the causal relationship between a program, policy, or intervention and its desired outcomes. Other kinds of evidence, including qualitative studies, monitoring data, and cross-sectional surveys, provide information on performance, coverage, and implementation considerations and, in turn, shed light on the causal pathways through which a policy or program affects outcomes. This information helps generate hypotheses and inform decisions about adjustments, imrovements, and future implementation strategies. As a complement, impact evaluation detects whether a specific policy or program leads to an observable change in outcomes and if it works better than an alternative approach or counterfactual. Impact evaluation methods can also generate crucial elements of cost-effectiveness analyses.

Despite all its potential uses for public policy, impact evaluation has also elicited concerns. Impact evaluation is often seen as too costly and time-consuming, bringing excessive attention to questions about specific interventions rather than underlying structural reforms to drive development outcomes; questions also have been raised about external validity and generalizability of findings (Bédécarrats et al. 2020; Cohen and Easterly 2010; Deaton 2020; Ogden 2017; Teele 2014; Vivalt 2020)

Over the past decade, a good deal of ink has been spilled on the topic of whether impact evaluation using experimental methods is a savior or a scourge in global development. After the 2019 Nobel Prize in Economics was awarded to Michael Kremer, Esther Duflo, and Abhijit Banerjee for their pioneering work in development economics, a raft of critiques and responses were written—many of which are included in the World Development special issue in 2020 on experimental approaches in development and poverty alleviation (Rodgers et al. 2020). Some of these discussions are among scholars and reflect academic debates about different methodologies. But others are centered on practical policy and resource implications: Do impact evaluations, which consume scarce capacities, time, and money, yield sufficient benefits that respond to the genuine needs of policymakers and result in reforms to policy and practice? If not, are there ways to increase the relevance and timeliness of impact evaluations, while decreasing the costs?

Against this background, the onset of the COVID-19 crisis in 2020 has underscored the need for high-quality, timely, and context-specific evidence—for both the effectiveness and the political credibility of the response. Beyond COVID-19, widespread calls to reckon with—and disrupt—the ways in which development research and institutions perpetuate inequities between and within countries and their development partners (Abímbólá et al. 2021; Erondu et al. 2021) have spurred reflection about the locus of power within research and evaluation communities.

These dynamics provide an opportunity to take stock of impact evaluation as a policy tool in and of itself, and to examine recent evolution within the field. This is also an opportunity to renew and broaden the bases of support for the evidence agenda. Doing so allows us to address concerns and identify the remaining challenges that limit the uptake of evidence by policymakers for real-world impact. By charting a renewed agenda for more useful, responsive, and relevant impact evaluation that elevates the perspectives of government policymakers and other evidence users around the world, we can reinvigorate policy commitment to, and funding for, rigorous evaluation. With more and better funding, we can harness the full value of impact evaluation to improve lives through improved policy decision making.