Recommendation 4: Strengthen Evidence Use through New Incentives and Structures
National, multilateral, and bilateral policymakers make thousands of consequential decisions everyday about how to invest their resources.
In theory, they seek to maximize improvements in living standards and outcomes as part of the moral core of public policy (Levine 2017). In reality, shifting institutional and policy priorities, bandwidth limitations, and a focus on inputs deemed (but not known) to be effective make it difficult to learn about and achieve better outcomes (Gaarder and Bartsch 2015). Organizational leaders rarely express or model sufficient commitment to learning and evidence use, and individual performance is often measured by approval and disbursement as opposed to actual results, which are typically available only after staffing changes between project cycles. At times, speed and confidence are rewarded over efforts to ask and answer questions about actual results. Further, COVID-19 has placed massive stress on officials’ abilities to collect, share, and use data and has laid bare just how insufficient existing evidence cultures often can be. More robust systems and incentives are needed to institutionalize the generation and use of rigorous evidence.
The most successful mechanisms to integrating evidence (including evidence from impact evaluation) into day-to-day operations will necessarily vary between different institutions. Yet across all bureaucracies, aligned incentives are an indispensable ingredient. The lack of institutional incentives and consistent signals from leadership impede evidence use in development institutions. Further, most government policymakers choose not to purchase or use rigorous evidence. Financial and professional rewards could generate demand, motivate capacity strengthening, and increase accountability for measurable impact. Depending on the institution, this might involve new budgetary requirements for evaluation strategies, changes to individual performance assessment criteria, redesigned results agreements, or legal mandates, as in the case of the US Foundations for Evidence-Based Policymaking Act of 2018 (Evidence Act). As policymakers increasingly put in place environmental and social safeguard policies, the use of evaluation could be institutionalized as a kind of economic safeguard to ensure the use of appropriate tools to determine whether projects have their intended impacts and whether they should be adjusted or scaled up or down. The rate of return is immense: a $1 million impact evaluation could save hundreds of millions in mistargeted or ineffective spending.
External actors can help fortify these functions through supportive partnerships and nudges for accountability. Some funders, such as the Hewlett Foundation, focus on these areas, but current funding levels are inadequate to building—and sustaining—systems and partnerships to withstand the tests of time and turnover. COVID-19 has shown that when organizations help officials use evidence to solve pressing problems, governments are more likely to demand—and in some cases purchase—more evidence. But this requires upfront demand generation. Many evidence-to-policy partnership organizations focus on capacity strengthening to build on existing interest and talent. Some organizations first work with governments to quickly respond to short-term evidence needs as an entry point to spurring longer-term interest in commissioning impact evaluation evidence or complementary data sources. For example, 3ie partnered with the Philippines National Economic and Development Authority and Australia’s Department of Foreign Affairs and Trade on a multiyear Philippines Evidence Program, in which 3ie and the Philippine government collaboratively commission impact evaluation of major government reforms and service delivery interventions prioritized by select departments.
In addition to external partnership initiatives, in-government or semi-autonomous evidence units—such as the Department of Planning, Monitoring and Evaluation in South Africa, MinEduLAB in Peru, and the Health Intervention and Technology Assessment Program in Thailand — provide value to broader institutional evidence cultures. Government agencies, evaluation units, and “arm’s-length” research and advisory bodies play a central role in commissioning and undertaking research studies and in facilitating engagement with civil society organizations and other external partners. Many of these units also provide capacity-strengthening support to build awareness about evaluations as a potential source of information and to help decision makers better understand how to interpret and act on data and research findings.
Built with Shorthand