Abstract
This paper proposes an extension to the recently introduced learning poverty measure by the World Bank. We argued that the World Bank approach only allows for estimation of learning poverty incidence and that it is crucial to go deeper in measuring depth and severity of learning poverty. The proposed extension is demonstrated using a case of Nigeria and will show that more insight on the education system can be learnt from it. In addition, the perverse incentive that is possible with concentrating only on incidence of learning poverty is not present under the new proposed extension.
1 Introduction
The recently proposed learning poverty line by World Bank (2019) represents a major milestone in the quest for better understanding of global learning crisis and for measuring progress on Sustainable Development Goal 4 (SDG 4). The learning poverty line is defined by the ability to read and understand a simple story at age of 10. Similar to the impact of income poverty line of a dollar per day, the new approach should deliver improvement in quality and quantity of data on learning and help generate evidence-based policy to address chronic learning crisis.
In this paper, we propose an extension to make an already good approach better. The learning poverty captures the extent of deprivation based on the proportion of children at age 10 (corresponding to end of primary education) that are not learning. This is similar to income poverty incidence measured by poverty headcount ratio. While estimating the poverty incidence is a good starting point, extending the poverty measurement to account for the depth and severity of learning deprivation can complement and strengthen the information on education system contained in the poverty incidence.
The key idea behind the suggested extension is illustrated in Figure 1. The graph shows learning evolution in three hypothetical countries from Grade 2 to Grade 6. Under the assumption that no child is out-of-school, the percentage of children not learning at Grade 6 gives the learning poverty incidence. However, education curriculum introduces children to basic reading and comprehension from Grade 2. The progression in learning from Grade 2 to Grade 6 will vary across countries. In a highly efficient education system (Country A), all children attained the required learning proficiency at Grade 2 and they maintain the impeccable performance through to Grade 6, implying that there is no learning poverty. In a less perfect system (Country B), many students donot immediately meet the defined learning threshold, but they improve considerably overtime and recorded no learning poverty at Grade 6. Country C depicts the case of flat learning curve in which there is no significant improvement in learning across grade levels.
Source: Authors Illustration
In this illustration, the learning poverty incidence in the first two countries is the same, but the education system in Country A is arguably superior to B. A quality education system is characterized by the expediency to which teaching combined with other educational inputs translates into learning outcomes (The United Nations Children’s Fund, UNICEF, 2000). Country A exemplifies an ideal education system where no child is left behind at any grade level. In reality, many education systems might not follow this utopian process; but the farther away from this ideal state, the less effective is the system. In essence, education system in Country A is better than in Countries B and C, and Country B is likewise better than in Country C. The key argument here is that looking at learning poverty at Grade 6 alone provides an incomplete picture, but by extending the analysis to all relevant grade levels, a more in-depth insight into the education system and process across countries is revealed.
Akmal (2018) made a similar point using the idea of a learning “S” curve. Specifically, she argued that in an ideal system, the steepest part of the learning curve will concentrate in the grade level where the concept is taught. Tracing the learning progression across grade level in this ideal system will look like an S-curve. However, she finds that learning curve is linear in many developing countries, suggesting a constant increase in mastery taking place. According to her, deviation from S-curve is an indication of structural challenges in education system, as teachers are teaching to a wide distribution of students’ skills in each grade. Learning progression will be hard under such education system.
In the illustration in Figure 1, Country A is used to represent the S-curve or the utopia state and deviation from it indicates the extent of the learning poverty and quality of education system. In essence, examining the evolution of learning will give a much complete picture of a country’s education development.
2 Brief Review of World Bank Learning Poverty
World Bank learning poverty encompasses both quality and quantity of education. The quantity of education refers to access and this is captured by the number of out-of-school children. The quality of education is the learning outcomes resulting from schooling and it is measured by the number of children below the learning poverty line.
Formally, this is measured by:
LP = [(BMP) x (1-OOS)] + [ 1 x (OOS)]
Where LP is the learning poverty; BMP is the share of children at age 10 (the end of primary) who read at below the minimum proficiency level. OOS is share of children of primary school age, all of which are regarded as being below the minimum proficiency level.
This approach has several advantages. First, it is easy to understand, therefore provide a clear goalpost for education stakeholders to collectively work towards. Second, its simplicity makes it easy to measure as well as replicate across countries. This enables identification of individuals and countries that are being left behind on the SDG 4. Third, the approach has no conceptual ambiguity and it is consistent with the existing literature on learning and quality of education (World Bank, 2017; Oye, Pritchett, & Sandefur, 2016). Yet, despite these advantages, the proposed extension in the next section will ensure a more complete picture of education system.
3 Proposed definition and measurement of depth and severity of learning poverty
3.1 Definition of concepts
The poverty incidence as computed by World bank can be considered the starting point in looking at learning poverty. This gives the proportion of children aged 10 that are below the poverty line. However, looking at the evolution of learning from where the reading and comprehension concept is first introduced up till the end of primary education will account for the depth of learning poverty. We define depth of learning poverty as the proportion of children that are below learning poverty line from the earliest grade at which reading and comprehension is introduced and up till the final grade in primary school.
Implicit in the above definition of depth of learning poverty is that equal weight will be attached to performance irrespective of grade level. Specifically, our depth of learning poverty is a simple average of poverty incidence from earliest grade the concept is introduced to the end of primary education. However, it can be argued that poor performance at higher grade level suggests more weakness in the education system. At the earlier grade when the concept is presented, it is expected that not all children will have full grasp of the new concept. But in an effective education system, the children left behind can be specifically targeted such that relationship between learning poverty and grade levels will be downward sloping and not flat. To reflect these circumstances, it is possible to weight learning poverty, with penalty attached to poor performance at higher grade levels. The severity of poverty is therefore defined as a weighted average of learning poverty across grades.
3.2 Measurement of learning poverty
We illustrate the idea behind the proposed formula for measuring depth and severity of learning poverty using the Nigerian Education Data Survey (NEDS, 2015). The NEDS assesses student’s literacy proficiency from pre-primary education to Grade 9 and also document the number of out-of-school children at each age-grade level. Primary education in Nigeria ends at Grade 6. The literacy assessment evaluates ability to read and comprehend simple sentences about everyday life and the same assessment is administered at all grade levels. The NEDS in a way illustrates the type of data requirement for measuring incidence, depth and severity of poverty, although World Bank-led learning poverty requires a more comprehensive and detailed learning assessment (ability to read and comprehend a story). According the Adeniran, Ishaku and Akanni (2019), the reading and comprehension assessments administered in NEDS is first introduced at Grade 2, based on the Nigerian education curriculum. Based on this dataset, we summarize basic minimum proficiency (BMP) between Grades 2 and 6 and percentage of children out-of-school (OOS) for the corresponding age level in Figure 2.
Source: Nigeria Education Data Survey Report (2015)
Using the information in Figure 2, the learning poverty at each grade level can be estimated using the formula:
LPi = [(BMPi) x (1-OOSi)] + [ 1 x (OOSi)] (2)
Where LPi is the learning poverty for grade i, and i ranges from 1 (earliest grade) to k (last grade); BMPi is the share of children at grade i who read at below the minimum proficiency level. OOSi is share of children at age equivalent to grade i that are not in primary school, all of which are regarded as being below the minimum proficiency level.
A key difference with equation 1 is that the estimate in equation 2 is based on each grade level. To align this approach with the World Bank estimate, the learning poverty for grade 6 will be estimated using average out-of-school children for all grades. The estimated learning poverty is shown in Figure 3. For ease of exposition, we will henceforth refer to graph showing learning poverty across grades as learning poverty curve.
On the learning poverty curve shown above,
Learning Poverty Incidence = LP5 = 59% (3)
This means 59% of the children aged 10 are unable to read and comprehend simple sentences. This illustrates that Nigeria is indeed facing a learning crisis. It is also expected that incidence of poverty in Nigeria will be worse if a more stringent learning assessment recommended by the World Bank is applied.
However, if we are interested in understanding depth of the crisis, we have to look at the entire area below the learning poverty curve. Formally, this is given as:
The formula takes cognizance of the fact that poverty incidence at last grade will be computed differently from the rest of the grades. The estimated poverty depth shows that 71.4% are below the poverty line per grade level. This shows that learning crisis is on much larger scale than what is depicted by the poverty incidence.
Finally, extending this approach to measure severity of learning poverty means adding weight to account for the expected improvement in performance at a higher-grade level. The ideal weight is a subjective issue, but any proposed weight must be monotonically increasing with grade level. We proposed an exponential weight of i/k. In the 6-tier grade level that we are working with, this gives a weight of 0.2 for earliest grade and 1 for the last grade. This meets the monotonicity condition. The average of the weighted learning poverty gives the severity of learning poverty. Formally, this is computed by:
Again, we take cognizance of the fact that learning poverty at the last grade will be computed differently from the rest of the grades. The estimate for poverty severity indicates that 79% of children per grade are below the poverty line. In the final analysis, the basic ideal in estimating incidence, depth and severity of learning poverty is similar to the Foster, Greer and Theorebeck (FGT) approach used in income poverty. Similar to FGT approach, estimating incidence, depth and severity of learning poverty gives a complete picture of the state of education deprivation.
4. Other benefits of the proposed extension
The extension of learning poverty to capture depth and severity in education outcomes retain all the characteristics and advantages of the World Bank approach. However, it will require data beyond learning assessment at age 10 or just end of primary education. The learning assessment will start from the grade at which the reading concept is introduced until the end of primary education.
However, there is an advantage even in estimating incidence of learning poverty with complete grade level assessments. World Bank (2019) already alluded to the comparability problems in the data used for calculating its learning poverty. One aspect of the comparability problem is that learning assessment is administered to different grades (mostly Grade 4, 5 or 6), which means poverty incidence will be overestimated in some countries and underestimated in others. The other aspect of the comparability problem is that education system differs both within and across countries. In some countries, at age 10, children are expected to be in Junior Secondary School if in private school, and Grade 6 if in public school. In as much as curriculum differs across countries, what it means to be in a particular grade will differ across countries.
Using complete learning assessment from all grade levels can resolve some of these challenges. One instance of this is illustrated in Figure 4. We assumed that due to data unavailability poverty incidence is estimated in Country A at Grade 5 (at age 9) and in Country B at Grade 6 (age 10), even though the two countries operate a 6year primary education structure. Let us further assume that there is learning assessment data from Grades 1 to 5 in Country A and from Grades 1 to 6 in Country B. The result in Figure 4a shows that learning poverty incidence is higher in country A than B, based on data available for highest grade level. However, if we consider Figure 4b where learning progression is observed between Grades 1 to 5, it is possible to predict poverty incidence at Grade 6 for Country A. The dotted line indicates the predicted learning poverty incidence in Country A and it coincided with the performance rate in Country B.
Furthermore, focusing on poverty incidence alone and using it as the basis for donors’ intervention can create perverse incentive. For example, if donors promise to provide grants or development supports for education on the basis of improvement in learning outcomes based on poverty incidence, countries can respond by being more stringent on promotion to grade level where the learning assessment will be conducted. This could be done by ensuring only those that have already reached or about to reach the poverty line are considered for promotion.
Alternatively, there might be concentration on programmatic interventions on English language (or any other language of assessment) curriculum or at Grade 4 and 5 in order to improve performance. Under these scenarios, learning outcome will improve but with only a superficial change in policy. However, by using assessment data across grade levels, this perverse incentive does not arise. Moreover, if the donors choose to still focus only on poverty incidence, complementing such information with poverty depth and severity will help identify when and where the learning outcomes is real or a mirage.
5 Conclusion
As Pritchett (2019) argued, the main aim of the learning poverty should center on systemic approach to education sector reform rather than a programmatic intervention. However, if learning assessment reveals only incidence of learning poverty rather than experiences across the entire system (incidence, depth and severity), it is not implausible to assume policy intervention will be programmatic. The approach illustrated in this paper provides a broader and more complete picture of an education system’s effectiveness and will not generate the perverse incentive to improve score and not system.
References
Adeniran, A. P., Ishaku, J. and Akanni, L. (2019), “Is Nigeria Experiencing a Learning Crisis: New Evidence from Curriculum-based Learning Assessment”, Center for the Study of Economies of Africa (CSEA) Working Paper. Available on: http://cseaafrica.org/wp-content/uploads/2019/10/Is-Nigerian-Experiencing-a-Learning-Crisis-New-Evidence-from-Curriculum-matched-Learning-Assessment-1.pdf
Akmal, M. (2018), “Are We Wrong About the Right Way to Organize Schooling?”, Center for Global Development, Available on: https://www.cgdev.org/blog/are-we-wrong-about-right-way-organize-schooling
Oye, M., Pritchett, L., & Sandefur, J. (2016). Girls’ Schooling Is Good, Girls’ Schooling with Learning Is Better. Education Commission, Center for Global Development, Washington, DC.
United Nations Children's Fund. (2000). Defining Quality in Education. New York: UNICEF.
World Bank (2017). ‘World Development Report 2018: Learning to Realize Education’s Promise’. Washington, DC: World Bank.
World Bank (2019). ‘Ending Learning Poverty: What will it takes” Washington, DC: World Bank.
Lant Pritchett (2019), Tackling Education Poverty with System-wide Improvements, Research on Improving System of Education (RISE) Blog, Available: https://www.riseprogramme.org/blog/tackling_education_poverty
