Girin Beeharry’s essay on the pathway to progress on Sustainable Development Goal 4 is thoughtful and thought provoking. But it left me somewhat uncomfortable. While I agree with much of it, I cannot fully share the belief in “performance monitoring” and “accountability” as the way forward. My discomfort has to do with how Girin’s conclusions could be used, and misused, when the debate moves from the global to the national and the local levels. My reflections are less concerned than Girin’s with the global aid architecture and more with in-country realities. There are three reasons for this.
First, my own experience, mainly at UNESCO’s International Institute for Educational Planning. For some 10 years, I have led a team in charge of supporting countries with the preparation of education sector plans and related documents. My research has focused on school supervision, decentralization, and the functioning of the educational administration, three themes that are strongly influenced by national contexts. I still remember my first contact with the global aid community, 35 years ago, as a secondary school teacher in the Caribbean. Upon the advice of the World Bank, the ministry of education decided to lengthen the school day, a measure that to us teachers was in no way an answer to the profound daily problems we faced.
Second, I believe that the impact of the global education architecture on Foundational Literacy and Numeracy (FLN) and similar goals is limited. Sustained change demands commitment and action by national and local actors.
Third and maybe most important, I fear that the proposals to improve the functioning of the global architecture will seep into and distort national discussions on educational improvement, where the challenges are very different. While performance measurement and accountability may be priorities for global organizations, they form a very incomplete part of what is needed at local level.
My reflections are based on my experience, or rather on my interpretation of my experience (therefore limited and biased); my reading of literature; and discussions with colleagues and friends, including Girin himself.
SDG 4 or FLN: Do we have to choose?
Let me start with what may appear, but should not be, controversial. Girin’s essay argues that there is an almost unsolvable conflict between the imperative to respect the broad global mandate that SDG 4 represents, and the need to prioritize, because this mandate is impossible to realize. Girin’s priority is FLN. But this argument is built on a misinterpretation (by some actors of the “global aid architecture”) of SDG 4.
SDG 4 is not a global education plan, and it is even less a blueprint for a national education plan. There are several reasons for this. SDG4 is not accompanied by a comprehensive set of strategies that can lead to its achievement, nor by a detailed estimate of total cost and available funding. Countries are simply too different for any single action agenda to be relevant. More importantly, insisting on the need to address all SDG 4 targets at the same time may lead to a status quo, for two reasons. First, countries will spread their scarce resources over the whole sector, with such little depth that it makes no difference. Second, as Girin notes, when a policy has to respond to the desiderata of all different interest groups, it risks responding mainly to those whose voice is loudest, whose power is strongest.
Denying that SDG 4 is a global plan, does not make this Goal less valuable, rather to the contrary. As a policy or a plan, SDG 4 can easily be put aside as unachievable, and therefore without credibility. The role of SDG 4 is different. It presents a long-term vision for the development of a learning system. As such, it is a source of inspiration that can bring people together and create “enthusiasm,” a resource that is all too scarce among many stakeholders, who have lost belief in the possibility of improvement.
It is therefore eminently possible to ascribe to the long-term vision that SDG4 presents and to design policies and plans that have a much more selective set of priorities. I recognize, as someone who has worked with many ministries of education on their education plans, that when sector-wide plans fail to choose a select set of priorities, they can become an instrument for the status quo rather than a force for reform. That is, in aiming to change everything, often nothing changes.
The global aid architecture and the lack of change
Girin’s essay goes a step further: it proposes a focus on one single priority, namely FLN, and sees this as a strategy to improve the functioning of the global aid architecture. He argues (i) that this focus allows for a clear priority, (ii) with an indicator that is both measurable and actionable, and (iii) for which aid agencies can be held accountable. On each of these three elements, I have several reflections.
A single priority that is not one
Girin presents several strong arguments for the selection of FLN as the priority for countries characterized by learning poverty. Although I agree with him, the strongest argument is also one that shows the near impossibility of selecting a single priority. This is the argument that a focus on FLN allows for the identification and examination of different system dysfunctions, which explain the low levels of learning. Indeed, “learning poverty” has many causes. They may include, without any order of priority: the scarcity and/or poor quality of early childhood education; the inappropriate classroom practices of early grade teachers; the incapacity of parents to demand better school performance; the unavailability of basic teaching and learning materials; the lack of awareness among teachers of where they need to improve; the ineffective school support and supervision structure; the unsatisfactory performance of teacher training institutions; and so on. In other words, choosing FLN as a policy priority allows for a sharper focus in a discussion on relevant strategies. However, the decision on which plan/program to adopt to achieve FLN still involves difficult choices between potentially conflicting strategies, and a discussion of the role of different sub-sectors. Such a plan/program may not be as wide as some sector plans, but it will always include a diverse set of strategies, aiming to effect different elements of the education system.
One issue that I am hesitant about is who decides that FLN will be the priority. Ideally, national authorities should do so. But Girin mentions several valid reasons why many do not. From my own experience working with ministers of education and their staff, this is not because they have an elitist vision from a privileged position, but because they have identified other challenges as more urgent: the almost total absence of technical and vocational education or the utterly ineffective governance of the system. Undoubtedly, that choice is in part an expression of their own social position, but that does not render their choice valueless. What may be more surprising is that teachers and parents are not clamoring for a focus on FLN. I will come back to that in a moment. In such a context, can this priority be imposed by the international community? This may be the implication of Girin’s suggestion that we should only work with countries already persuaded of the need to prioritize FLN. This is evidently not a preferred scenario, not so much because of an uncritical respect for “national sovereignty,” but because an imposed priority is hardly a genuine one, and may be respected in appearance but not in action. Ideally, the choice of priorities will result from a search in which all stakeholders participate, guided by “evidence” on successful programs, by the experiences of the different actors (which is also evidence), by their opinions, and their interests. The international community, as one of these stakeholders, can bring convincing arguments and build a coalition in support of FLN.
A frightening indicator we can afford to disregard
One apparently strong argument for the focus on FLN is the very low levels of learning, illustrated by several references to the learning poverty indicator. The existence of a single indicator that allows for easy measurement of its achievement is seen as an advantage. I cannot fully agree. There are two major risks with an emphasis on this one indicator.
First, when it is easier to manipulate an indicator than to change the behavior this indicator intends to measure, there will be a strong temptation towards manipulation. This is not unique to education nor is it prevalent only in developing countries. The phenomenon is so well known that it has its own law, Campbell’s Law, which states: “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.” Diane Ravitch’s excellent book The Death and Life of the Great American School System presents schools’ gaming of the tests as an almost unavoidable response to “test-limited” management and refers to multiple reports of systematic cheating on tests in school. It is not ill-mannered to suggest that schools and teachers in education systems, where regulation is weak and such test results are high stakes, will be tempted to change results; this would actually be the intelligent reaction. It can be argued that externally administered standardized tests are difficult to manipulate. This may be true, but it emphasizes an essential worry: these tests are not integrated into the evaluation practices of the education system. As a result, they are low stakes. They lead to neither rewards nor sanctions for schools, teachers, or students; they provoke no action on their behalf. This helps explain why many teachers and parents do not consider FLN as a priority. They are simply not aware of the severity of the learning crisis. It is too easy to say that exams cannot be trusted and therefore should be disregarded when every administration spends significant efforts to run them and when they remain the main tool for social mobility. It is politically, technically, and institutionally much simpler to organize an external assessment than to reform the exam system, but the latter is more important. Now (with some simplification), several countries are faced with a conundrum: exams are high stakes and actionable, but distrusted by much of the international community; external achievement tests are trusted, but considered of little importance by those who need to change their practice or whose voices need to be strengthened.
The second risk is that an indicator of FLN offers a very simplified picture of a very diverse and complex situation. The simplification is threefold:
- There are many different factors that help explain if children have learned or not. FLN is the result of these factors. Learning improvement demands that these factors undergo change. An FLN indicator does not inform us at all about these factors.
- The distinction between a child who is foundationally literate and numerate and one who is not is not clear-cut. Comparison with equivalent health-related indicators (neo-natal, infant, and under-5 mortality) is instructive: mortality sadly bears no discussion. The statement that “nearly 9 of 10 children aged 10 in sub-Saharan Africa are not able to read with comprehension” reflects a number of choices made by those who measure, and does not tell us anything about the level of learning of the 90 percent. This is not to argue that we are not faced with a serious crisis, but that the depth and extent of this crisis (and the brightness of the rays of hope…) are important elements in a policy debate.
- The third point is the most important for me. The FLN indicator presents an average, but in reality, this average does not exist. Each school is different and unique. Of course, no national indicator can represent this diversity, but this is particularly problematic in efforts to improve learning. Test results inform us about the state of the system, but change in learning depends more on what happens in the school and in the classroom than through a system-wide reform, especially in countries where such systematic efforts do not reach all schools and classrooms. The implication is that each school needs support to design its own improvement strategy, based on its present situation. An FLN indicator, even more so if it is based on a sample survey, usefully guides system reform, but does not provide the essential school-relevant knowledge.
While an FLN indicator is concrete and measurable enough to highlight if action is needed, it is not helpful to define which actions are needed where. I have little doubt that Girin is aware of the limits of the FLN indicator, but my worry remains that, with so much attention and energy going to this one indicator, less energy (and funding) is available for a more comprehensive indicator system.
Let me add here that we should be careful in transplanting experiences from a country that has improved FLN to other countries. Undoubtedly, such experiences are sources of inspiration. However, in many cases this success is not the result of a set of strategies that can become a universal reform package, but rather of the relevance of these strategies to a specific context. It is not the strategy but its appropriateness to the context that makes for success.
The question of accountability: A different tragedy of the commons?
The question of the accountability, or lack of it, of the international community is a pertinent one. Two principles and one more practical question should guide any discussion on accountability. The first principle: there should be a balance between professionalism, autonomy, and accountability. A genuine professional deserves autonomy, and this autonomy should be accompanied by accountability. The second principle: I can only be held accountable for something over which I have control. The practical question: to whom am I accountable? Who can hold me accountable?
The first principle poses no problems for the global education community: most of us are genuine professionals, with the qualifications, competencies, specialized knowledge, resources, and sense of service that allow us to make a difference. Most of us work with significant autonomy. Therefore, we can and should be held accountable, as individuals and as agencies. The second principle is more complicated. The global community does not have control over the achievement of FLN. Undoubtedly, it influences this in different ways, through funding, advice, and technical support. However, FLN is the result of actions by many different groups, some with more direct influence than the global community. In such a scenario, where several groups need to contribute to achieve a single result, it is easy to escape accountability as others can almost always be blamed (usually, blame is shared).
This raises, of course, the question: for what can the global education community and its members be held accountable, in our joint effort to achieve FLN? I would suggest, as a minimum, two elements. First, the choice of strategies and programs that we promote, or fund, or implement. We can be asked to demonstrate that this choice is made through careful and well-argued reflection, which refers to relevant evidence, on how these strategies and programs contribute to achieving FLN. This is fully under our control. Second, the success of these strategies and programs, namely: how far have their intended objectives (at least at the “outcome,” not only at the “output” level) been achieved? We should be fully transparent in the methodological aspects of the evaluation, in its findings, and in how we intend to change our practice in function of these findings. While we do not always fully control the outcomes of a program, we cannot judge a program only by its intentions; we need to look at their actual effects.
Who can demand such accountability? Girin suggests that we hold ourselves accountable. I am not fully convinced, but I do not have a better alternative to propose. I am not convinced because even a humble individual finds it difficult to examine her or his own performance and accept responsibility for mistakes. Organizations, who are working in a competitive environment, and are staffed by strong-minded experts, may be less prone to do so. (I do recognize that there are exceptions, with effective evaluation or oversight services in several organizations, but I have not seen many examples of their reports leading to profound changes in practice.) In an ideal world, the demand for accountability would come from those who are the beneficiaries of the FLN-focused programs, students and their families. But that ideal world is far away. So, in the meantime, the answer is probably a combination of different approaches: to reinforce internal accountability mechanisms; to strengthen the existing global fora that aim to hold the international community accountable; and to continue the long and slow struggle of strengthening the voice of the unheard.
The weakness of accountability is unfortunate, especially because it limits the opportunities for learning by the global community from mistakes and successes, but I do not want to dramatize this. The existence of an accountability system is not the only incentive to undertake work of good quality that benefits students and their societies. There are other incentives that guide us. Some are institutional, such as professional development or the promotion of a culture of joint learning, while others are individual, including a sense of duty and a sense of service.
The global architecture and the local reality
I finish my reflections with four points that I rather think Girin will agree with. They are not contradictory to what he wrote, and they are in part inspired by discussions with him.
FLN is a useful and a legitimate priority for many countries. However, the choice of FLN as a priority does not imply that the problem of ambitious plans and competing priorities is solved. There will always be a need for an internal policy dialogue on the choice of appropriate strategies, on the funding of competing programs. This is potentially very useful. An “outcome harvesting” evaluation of IIEP’s support to Jordan and Guinea with the preparation of their sector plans demonstrates that this internal process, with government leadership and in a participatory spirit, is not only well appreciated but has also led to some significant changes, including more robust government funding and better coordination between ministries.
“Performance measurement and monitoring” are essential to management, to learning, and to improvement. But the emphasis needs to shift in two ways.
- Deepen ownership and awareness of relevant measurements, not so much among the global community, but at national and local levels, where change in action is most necessary. I would be very surprised if many Malian teachers actually realize that their teaching is so weak that only 2 percent of early learners master FLN. And when they are confronted with this datum, their understandable reaction may be one of disbelief. Too many other signals (the performance of other teachers; the acceptance by many parents; their continued employment; pass rates and exam results) paint a less dark (though not necessarily a rosy) picture. And if ever they recognize their weakness, many are at a loss to know how to improve, which brings me to my second shift.
- Move the balance of our efforts and our funding from “measurement” to “learning for improved action.” Develop system-wide responses that support local actors and allow appropriate local leeway (more framing when local capacities are weak; more autonomy when they are strong). Support the search by districts and by schools for a reform package that is appropriate to their situation. Listen to global advice; learn from experiences elsewhere; but keep in mind that every global certainty can be disproven by a local reality.
Balance accountability with support. The above principles about accountability play out very differently at the district and school levels, where, with a crude simplification, the situation can be summarized as “little professionalism, little autonomy, little accountability.” Strengthening only one of these elements makes things worse, especially if that one element is “accountability” or “autonomy.”
Finally, the fact that many education staff and many teachers have lost belief that they can make a difference is arguably the deepest challenge to be solved. Without disregarding the severity of the learning crisis, we need to find sources of optimism. We have to think of the teachers who, against the odds and abandoned by the system, continue to go to school and work hard for their students. We have to support those ministry officials who, almost on their own in an often sclerotic and demotivating environment, work beyond the call of duty for a better future for their country. Indicators do not capture the full reality, and we have to believe that there is still enough energy, willpower, and determination to make for lasting change.