Essay 19


Should Foreign Aid for Education Focus Exclusively on Raising Second-Grade Test Scores?

By
Justin Sandefur

Test Scores: Adobe Stock

Introduction

Girin Beeharry's recent essay in the International Journal of Educational Development is, more than anything else, a manifesto. The core empirical premise of this manifesto is that education systems in much of the developing world are failing. Millions of kids are going to school for multiple years and emerging functionally illiterate and innumerate. But Girin’s goal is not to rehash well-known statistics about the global "learning crisis," or even to diagnose the causes for that failure. He wants to put forward a broad vision of how international donors to global education can escape the mess they’re in.

This manifesto has three pillars. First, donors should prioritize foundational literacy and numeracy— essentially, test scores in 2nd grade—above all other education goals. This focus on early-grade reading and math has the dual merits of being instrumental to advancing other loftier goals, and inherently egalitarian, inasmuch as it “raises the floor” of minimum educational outcomes. Second, national governments and international organizations should invest in a global regime of standardized testing to monitor progress on this goal. And third, aid donors must be held accountable for improving those test scores.

Girin’s zeal for the cause of early-grade learning, and his bluntness about who is failing and how, make his essay more compelling than most white papers in the aid sector. He names names, or at least organizations. And as the founding director of the Bill and Melinda Gates Foundation's work in global education, when Girin addresses officials from the World Bank, UNESCO, or USAID, he’s speaking to friends and professional colleagues. He believes in their collective enterprise, understands the bureaucratic constraints they face, and exploits that understanding to propose concrete ways they could, by his metrics, do better. To be transparent here, I should note that many of the organizations Girin calls out, both for praise and criticism, are Gates Foundation grantees, as is the Center for Global Development, where I work. Having recently stepped back from his main Gates role, one senses that Girin feels liberated to speak his mind, making the essay refreshingly candid.

Stated so nakedly, Girin’s proposal to reorient international aid for education around primary-school test scores is sure to provoke opposition in many quarters. So I think it's important to note some of the intellectual traps that Girin's essay avoids, and that separate this piece from some of his potential allies in the push for a focus on testing and accountability—what critics have labeled the "Global Education Reform Movement" or GERM.

  1. First, Girin’s call for greater accountability is not code for blaming teachers for educational failures. So often, the moral panic in conservative circles that "our children aren't learning" transitions seamlessly into a denunciation of lazy teachers and the nefarious influence of teachers’ unions. In contrast, the call for accountability here is explicitly about accountability within institutions like UNICEF or the Global Partnership for Education, and not a call to deploy carrots and sticks against teachers.
  2. Nor should Girin’s call for better data on learning outcomes be read as a plea for more high-stakes testing. If anything, there is reason to believe that a focus on high-stakes exams at the end of primary or secondary school may detract from the focus on early-grade learning advocated in his essay.
  3. The focus on monitoring and accountability at the system level allows for—though Girin’s essay does not elaborate—a more sophisticated view of how education systems operate, beyond mechanistic policy levers subject to simple cost-benefit analysis. The piece avoids the gimmicky “solutionism” that plagues the sector. Girin harbors no illusions that if we just identify what works through rigorous research we can magically will it into being. Indeed, the piece is rather pessimistic—perhaps too much so for my own tastes as a researcher!—about the role for more impact evaluations to improve educational performance.

But while the essay avoids wedding itself to any specific set of education policies, Girin’s rather monomaniacal focus on test scores as the proper goal of education system strikes me as a bit too narrow. His focus on low-cost pedagogical innovations gives too little weight, in my view, to the potentially higher returns to basic investments in easier-to-implement things like early-childhood development, free secondary schooling, and school meals. And its approach to governance of the international aid system can feel a bit, well, Gates-esque, in prioritizing a technocratic agenda over democratic consensus.

To counter those tendencies, I want to recommend three additional principles—beyond prioritizing, monitoring, and imposing accountability for learning outcomes—that global actors in education might consider.

Look for policies that are hard to get wrong, not ones that are hard to get right

Over the past half century, developing countries have dramatically improved their literacy rates, converging gradually to the levels of rich countries, and achieving far higher literacy rates at given levels of economic development than in decades past. They did this primarily by expanding access, an area where there's still some low-hanging fruit to be harvested: after all, in 2018 completion rates for lower-secondary school in low-income countries averaged just 40 percent.

Figure 1. Estimated literacy rate among adult women by birth year and region

Note: horizontal axis shows year of birth. Source: Le Nestour, Moscoviz, and Sandefur (2021) based on DHS and MICS data.

Note: horizontal axis shows year of birth. Source: Le Nestour, Moscoviz, and Sandefur (2021) based on DHS and MICS data.

Getting that number to 100 percent won’t be cheap. It requires building secondary schools, hiring teachers, slashing fees, and repeating all the stuff we did to get the world tantalizingly close (but not quite) to universal primary enrollment. The point is, we know how to do it. The kinds of policies required constitute shovel-ready investments that can bring us closer to universal literacy and numeracy while reaping huge economic and social returns.

My concern is that Girin’s proposal would essentially rule out the kinds of investments that have gotten us this far. The manifesto tells us to deemphasize raising education budgets, to resist the push for free secondary school and even pre-school, and to turn our attention to improving learning in early primary.

While I agree it would be nice to get more (learning) for less (money and time in school), this ignores what’s worked historically in favor of what hasn’t.

Unlike expanding schooling, scaling up successful pedagogical reforms to improve learning outcomes region- or nation-wide has proved very difficult almost everywhere. When the World Bank research department summarized the lessons of development economics for policymakers, one of their core lessons was simply that "implementing successful small interventions at scale is hard.” The first example cited comes from an NGO program to increase primary-school test scores in Kenya which my coauthors and I studied, and watched fall apart, as the government took it to scale.

That’s not an isolated example. When researchers have gone out looking for successful programs to improve reading scores in the developing world through pedagogical innovations, they find a striking negative relationship between the scale of the program and the size of the effect (Crawfurd and Le Nestour, forthcoming).

Meanwhile, there are already feasible, rigorously tested, scalable policy alternatives to improve literacy and numeracy outcomes that are actually hard to get wrong. These are policies that even the most fragile states with limited implementation capacity have rolled out successfully. Make school affordable and convenient, and get more kids in school for more years. Provide free school meals to get more kids in school paying more attention. Increase instructional hours by extending the school day.

Yes, these things can be expensive. But the returns are high, and they’ve proven hard to get wrong, while improving pedagogy has proven hard to get right.

Let countries choose: there’s no technical basis to force countries to invest in second-grade pedagogy instead of, say, universal preschool or free secondary school

Girin’s essay expresses frustration with the "partnership structure" of the Global Partnership for Education, which, it contends, leads to a lack of focus. Girin laments that GPE's ability to prioritize the foundational literacy and numeracy indicator in the UN’s list of SDG targets (i.e., target 4.1, indicator 4.1.1) is compromised by the partnership’s need to attend to all 10 targets under SDG 4. He concludes with a fairly explicit call for donor countries to reduce their focus on "greater collaboration" and turn instead toward accountability for outcomes.

Developing country governments could be forgiven for reading this as a call to reduce their voice in multilateral decision-making.

Compare the governance structure of GPE to the World Bank, which receives higher praise in the essay, and has received more money from the Gates Foundation. As a conduit for foreign aid for education, GPE provides developing countries and civil society with the most direct oversight and control, though UNICEF is arguably comparable. The World Bank's International Development Agency (IDA) lags behind, and at the opposite extreme, of course, are bilateral aid agencies where rich countries call all the shots.

Figure 2. Board representation of rich and poor countries across different aid channels

This is reflected in how these various donors work on the ground. While GPE is forced to reach consensus among various stakeholders and follow country priorities, other donors have much more liberty to impose top-down solutions, especially in the poorest countries with the least bargaining power.

Developing countries ought to be able to choose whether they want to invest in expanding pre-schools or secondary schools, teacher training or performance bonuses, reducing school-based violence or increasing university scholarships. The goal of the global aid architecture should be to facilitate those democratic experiments, not to stifle them.

I don’t want to overstate my case. During our roundtable discussion of his essay at CGD, Girin noted that if he were to write it over again, he’d start from the country perspective, rather than an agenda for global actors. He also mentions the idea of a coalition of the willing to attack the challenge of early-grade reading and math. It’s hard to argue with that voluntary approach, so long as external assistance is not conditioned on recipients adhering to donors’ policy priorities.

First do no harm: Test scores come second to keeping kids safe, and we’re failing at that

While it is disturbing to hear that millions of kids go to school every year without being exposed to the basics of literacy and numeracy, it is perhaps more disturbing to contemplate what they are exposed to.

In a survey of Zambian students age 13-15, the WHO found that over half had experienced physical violence in the past year, and roughly a third had experience sexual violence in their lifetimes. Figures were lower, but not nearly as low as you'd hope, in Namibia, Swaziland, Uganda, and Zimbabwe.

Mind you, nobody really knows how much of that abuse is happening in schools, because we rarely ask. More recent data from the PISA-for-Development surveys found about 12 percent of Zambian girls reported sexual harassment by a teacher or school staff member in the past four weeks, as calculated by my colleagues Lee Crawfurd and Susannah Hares.

Underreporting is almost certainly a major problem here, and methods to elicit confidential responses to such sensitive questions have shown mixed results. But for the most part, nobody is asking. There is no routine system of data collection on physical or sexual abuse in schools in most developing countries. Unlike for foundational literacy and numeracy, there is no UNESCO monitoring effort to track the number of kids being raped by teachers. And there is no multimillion-dollar World Bank effort to develop new measurement tools to figure out whether children are being physically and sexually abused in school, or to study what we can do to improve child protection.

Prioritizing foundational literacy and numeracy to the exclusion of all else in education poses real trade-offs.

Admittedly there’s a potential tension here between promoting a child protection agenda and my previous point about respecting country priorities. But if foreign donors are ever going to use financial leverage to shape domestic policy priorities, then the protection of basic human rights for children feels like a much more compelling basis for aid conditionality than the promotion of test scores over enrollment as the proper metric of educational performance.

What works, what matters, and what people want aren’t always the same, and education policymakers face big trade-offs

In advocating such a hard line, Girin’s proposal goes further than I think most aid donors can or should go in focusing exclusively on early-grade reading and math programs, and casting aside other priorities like child protection, early-childhood development, school feeding, and so on. My view is slightly less pessimistic than Girin’s about the current path we’re on. Educational investments over the past half century have yielded incredible returns. We should not despair about them, or hesitate to advocate for more money for school systems as they currently exist.

But despite my quibbles, Girin’s provocative essay is a welcome departure from much of the stale platitudes that pass for debate in the global education sector. Many senior officials at donor agencies are quite accustomed to preaching about the need for more investment in early-childhood development in one meeting, and the benefits of free secondary schooling or vocational training in the next. Trade-offs go entirely unacknowledged.

Girin's manifesto calls us to confront these trade-offs head on, and hold ourselves accountable for those choices. His impatience with the current pace of progress, and insistence on laying down practical steps forward, has become a nagging voice in the back of my head. I hope his essay has the same effect on others.