Changing indicators to change the world: evidence from the ground.


Why the way we measure progress on education matters.

The building of a school next to an IDP camp in the Teso region. Research participants said they have the physical walls but not the teachers and teaching materials needed to provide their children with quality education.

The building of a school next to an IDP camp in the Teso sub-region. Research participants said they have the physical walls but not the teachers and teaching materials needed to provide their children with quality education.

A widely acknowledged success of the current MDG framework is the creation of strong incentives for the governments of developing countries to achieve progress on the agreed goals. Governments want to receive international praise and increased AID flows associated with the implementation of good pro-poor policies. Government performance is assessed on the achievement of the MDGs targets associated to the goals. If we take the example of the second goal “Achieve universal primary education”, the target is to “Ensure that, by 2015, children everywhere, boys and girls alike, will be able to complete a full course of primary schooling”. The key indicator used is the Net enrolment rate in primary education, complemented by the number of pupils enrolled in grade 1 who reaches grade 5, and literacy rates. This post examines the implications of the incentives generated by the current MDGs indicators. It does so by considering the critical views of those living in poverty collected through the COMPASS 2015 research.

Before exploring the perspectives of COMPASS 2015 participants regarding progress in education, it is important to highlight why the choice of indicators is so important. According to Nobel Prize, Amartya Sen, choosing indicators means choosing the aims and ends of society. There is a powerful sport metaphor that helps understanding the relevance of this point. The body of a person will radically change according to the targets that he has to achieve. A sprinter will develop a body full of muscles to reach a fast speed; a sumo wrestler will develop a heavy fat body that gives him mass to push the competitor out of the ring. Equally, the development objectives that we set through indicators deeply shape our society.

Let’s assume that a government is genuine, has no intention to manipulate data and wants to comply with the internationally-set goals. What is the most effective way for a government operating under serious resource constraints to achieve high enrolment rates (being this the key target through which their efforts are evaluated)? Of course, they would build everywhere the cheapest structures, send the minimum amount of teachers per school, with the minimum teaching equipment possible. Without clear indicators for quality, this is what tends to happen. Progress on MDGs is almost exclusively measured quantitatively (have a look at the 2012 MGD progress report).

During the COMPASS 2015 research, the concern for the poor quality of education and the need for monitoring progress in education in qualitative terms with the participation of citizens is unanimously emerging from communities in different continents. For instance, in Uganda, everyone agrees on the importance of introducing Universal Primary Education and Universal Secondary Education – policies which made the country loved by the international donor community. However, the rural communities engaged by the COMPASS 2015 described large overcrowded classes without teachers, furniture and teaching materials. They argued that without enough trained teachers, packing dozens of students in a room is a waste of their time, particularly when the students could contribute to the livelihoods of the family by farming. An Ugandan participant questioned: “What are they learning? We are losing a generation if we run schools with 3 teachers. It is very dangerous to have half-educated people who believe they are educated”.

Achieving high enrolment rates is undoubtedly important, but the quality and relevance of what is taught is equally important. However, investing resources in good education doesn’t improve MDGs education statistics. A research assistant who has been working with children with special needs emphasised how these children have been completely excluded from the MDGs. There is a need for trained and capable teachers and additional resources to ensure that these children may be able to fully develop their potential and become fully part of the society. Unfortunately, what counts is just the enrolment rate – that kids stay in schools, not what they learn. In terms of the quality of learning, how can students in a rural under-resourced school receive comparable quality education if their peer in better urban schools also receive food and have better facilities? “How can you learn with an empty stomach?”, effectively summarised a participant. The disadvantage is consolidated rather than challenged through the school system which fails to open opportunities for all citizens.

The same message came from Bolivia, where participant reflecting upon the MDGs emphasised the need to measure the quality of education by involving civil society in monitoring education progress. There is a wide consensus on this issue across countries. During a research session in the Zimbabwean city of Mutare, a policeman sent by the government to monitor the research activities joined the discussion expressing his concern for the deteriorating conditions of public education and the increase need for those who can afford to hire private tutors.

An additional element to the debate was added by the research in Mindanao (Philippines). Participants argued that governments should also engage with the citizens in thinking what is the model of education and what are its aims. Should education help students to gain access to colleges which open the door to long-term international emigration? Or should education provide the skills to build successful and sustainable livelihoods in local communities? The current model is focused on the objective of producing labour for export. Some students and their families are happy and choose emigration as a deliberate strategy but often it is the only strategy if education does not equip students with what they need to develop a viable livelihood in their community. Moreover, this focus has led to perverse effects. In the community of Rogongon, the school has a computer lab but the machines are idle and are getting rotten since the area has been without electricity for more than a year, and even if they had electricity there are no other computers in the area, where IT skills may not be very relevant to local life. In the community, the local school has started a unique attempt to introduce a year of sustainable agricultural practice at the end of high school to teach how to use effectively the fertile land with appropriate technologies. The good news is that the government has shown interests in this experimentation.

To conclude, evidence from the ground in three different continents supports the need to include indicators that measures quality in the new post-2015 framework and change the current system of incentives for governments. The measurement of quality can happen only with the involvement of citizens. Such involvement may also lead to a deeper national and local reflection on the role of education in the development of the country. These insights from participatory research have implications beyond education; the need for radically different sets of indicators can be extended to other areas such as health care. How we measure quality and how we measure the fact that government response reflects the needs and aspiration of citizens remain central questions for a post-2015 framework. The High Level Panel’s call for a ‘data revolution’ and ‘better accountability in measuring progress’ at the end of their last meeting in Bali is therefore very important. However, those living in poverty are very clear about the need to go beyond better data, statistics and ‘disaggregation’ and include citizens’ participation in monitoring the quality of education through different indicators and defining collectively the purpose of education in society.

Tags: , , , , , , , , , , , ,

2 Responses to “Changing indicators to change the world: evidence from the ground.”

  1. Paulous Says:

    Universal primary education and secondary education were introduced in Uganda but the extent to which they are enabling the students to be what they want to be and do is the question the people are grappling with. One can say universal education has not been able to enhance important trained capabilities for the poor people

  2. Do we need the Multidimensional Poverty Index (MPI) for post-2015? | Serpents and Doves: A development policy blog Says:

    […] A government may work strategically on the cheaper deprivations to move the number of people needed beyond the cut-off for specific indicators. Moreover, a large component of the Global MPI is based on input indicators regarding education, such as, school attendance and years of schooling which simply measure that fact that a number of pupils are put in a room for a number of years. As we’ve learnt from the MDGs, to improve performance in such indicators it is enough to force people into poor buildings they call schools (threatening withdrawal of other type of support, e.g. CCTs, for non-attendance); increase class size; or to reduce the teachers/students ratio. Therefore, improvement in these indicators may not be connected with changes in the poverty of people, not even in terms of accumulation of human capital. In fact, none of these indicators measure what people living in poverty across the globe felt they needed. Recent participatory research by CAFOD revealed how the quality of education, adequate facilities and staff for students with disabilities were seen as priorities for people. Research participants thought they were wasting their children’s time by placing them in poorly equipped and understaffed school…. […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: