Careful monitoring and observation of each component of the TaRL approach is an important aspect of successful TaRL implementation. In this post, we reflect on how the TaRL Africa team, along with country TaRL teams in Nigeria and Côte d’Ivoire, identified challenges with the TaRL assessment tool and used this information to tweak it. The new assessment tool is being tested in 2020.
How was the original tool developed?
The numeracy assessment tool used for TaRL programmes was originally conceptualised by Pratham in India and later adapted by country teams in Africa, with support from the TaRL Africa central team, to suit particular contexts.
What’s in the original tool?
The original tool used in Nigeria and Côte d’Ivoire comprised of two sections:
- Number recognition: Tasks consisted of recognising numbers from 1 digit to 3 or 4 digits. First, children were asked to identify 1-digit numbers, if they could do this correctly, they were asked to identify 2-digit numbers and then, if identified correctly, they were asked to identify 3-digit numbers and so on.
- Operations: 2-3 sums for each operation – addition with carryover, subtraction with borrowing, multiplication, and division. Children were asked to complete all operations.
In TaRL programmes the facilitator recorded each child’s highest number recognition level and recorded which operations the child could complete (writing either “can do” or “cannot do”). Children were then grouped based on their number recognition performance, as seen below.
Group 1: Beginner + 1-digit number recognition
Group 2: 2-digit number recognition
Group 3: 3-digit number recognition
Why did the tool change?
From field observations of the country teams undertaking government partnerships in Zambia, Nigeria, and Côte d’Ivoire and accumulated experiences of partner organisations and the TaRL Africa central team across contexts, it was found that the original numeracy tool suffered from a range of issues:
Time: The assessment process required each child to attempt all operations, making assessments time-consuming, especially in contexts where the child-facilitator ratio was more than 50. Observers noticed that many children were accustomed to solving operations using circles/tally marks, further extending the time needed for assessment. Let’s take the example of 57 multiplied by 9. A child would draw 57 circles/sticks on the paper 9 times and count all the circles/sticks again. Similarly, for 46 divided by 3, a child would draw 46 circles/sticks, count them again making groups of 3. This meant that facilitators spent a lot of time assessing children.
“Observers noticed that many children were accustomed to solving operations using circles/tally marks, further extending the time needed for assessment.”
Data recording: Since a child was marked on 5 competencies (number recognition and 4 operations), the facilitator needed to record 5 ticks per child, which meant that facilitators were more likely to make errors. Furthermore, the number recognition section was progressive (recording only the highest-digit number) while the operations section was successive (facilitators were required to record the child’s ability to complete each operation), requiring facilitators to be mindful of this switch when recording data. Often facilitators forgot to mark children on some operations which led to data totalling errors.
Grouping: The data emerging from many of the countries implementing TaRL in Africa revealed that the majority of the children are able to recognise at least 2-digit numbers but very few are able to solve operations. For example, 64% of the total children involved in the TaRL pilot in Borno, in northeast Nigeria, could recognise 2-digit numbers at baseline, but only 18% could solve an addition problem and only 4% could solve division. Since the grouping of children for a TaRL class was based on number recognition, most children were placed in group 2 or 3 (indicated in the grouping recommendations above). As classes started and children progressed in number recognition, they shifted to higher groups without necessarily making as much improvement in solving operations. This led to children with varying skills in mathematics operations being grouped in the same class. Sometimes this made it challenging for the facilitators to plan class sessions.
Given these challenges, the TaRL Africa central and country teams began to think – could the assessment tool be adjusted to cut down the time needed for assessment without compromising the ability to group children by level? They leveraged information from classroom observations, as well as the Annual Status of Education Report (ASER) survey assessment tool to create an adjusted assessment tool.
What’s in the new tool?
The new tool consists of 1-digit and 2-digit number recognition tasks followed by subtraction with borrowing and division problems. The tasks are completed progressively (moving from one-digit to two-digit, then from subtraction to division). The child is only asked to do a higher level task only if they are able to solve the previous task.
Each child is marked only at the highest learning level. Children are grouped based on the following recommendation:
Group 1: Beginner + 1-digit number recognition levels
Group 2: 2-digit number recognition level
Group 3: Subtraction + Division levels
Since this tool is progressive and does not require the facilitator to administer all tasks, it is much quicker and data recording is simpler. A drawback of this tool when compared to the old tool might be that the facilitator is not able to understand children’s progress in addition and multiplication and makes the assumption that they are proficient in these operations if they qualify the subtraction and division levels, respectively. Ongoing monitoring and classroom observations will help us to understand whether this is a problem, if it is we will need to come up with a solution.
Comparing the old and new tools
| Old tool and grouping | New tool and grouping | |
| Time | More time consuming | Less time consuming |
| Data recording | More room for error | Less room for error |
| Grouping | Different learning needs in operations within a group | Similar learning needs in operations within a group |
| Understanding progress | Able to see progress in all operations | Able to see progress only in subtraction and division |
Next Steps
Both tools have pros and cons. The suitability of the tool is contingent on the context in which TaRL is implemented. For example, if the classes are small, time is not a big constraint. If one facilitator is managing all learning level groups within one class, then grouping and progression needs to be considered accordingly. The new numeracy tool will be used in a pilot in Kano, northwest Nigeria, and was selected by government master trainers in the extension of the pilot in Côte d’Ivoire. As we receive feedback on the new assessment tool from these two pilots, the TaRL Africa team will continue to learn from implementing partners and iterate on different versions of the numeracy tool.
