Given both the varying life experiences of students, how can we do our best to make sure literacy and numeracy assessments are fair and valid for all?
There’s been plenty of discussion in the media recently about the trial of the New Zealand Qualifications Authority’s revamped assessments for literacy and numeracy. Different groups have expressed concern about pass rates, test design, and our overall capabilities in these areas.
For some time, both international and national testing has indicated there is inequity in achievement in literacy and numeracy, particularly between different socio-economic and ethnic groups. However, this type of testing does not always give us a reliable picture of what our students know and can do.
Why not? There is a range of challenges in assessing these skills, especially when a one-size-fits-all approach is used – as is the case when assessments are nationally or internationally administered.
READ MORE: * Reading the signs: Why literacy rates are falling * We have to face the truth – our kids aren’t good at writing
For literacy and numeracy skills to be useful, people need to be able to apply them to help solve problems and answer questions they come across in their lives. The lives of people in our country, including our secondary school students, vary hugely and so they will have differing background experiences to draw from and slightly differing literacy and numeracy needs.
Think about the differences in the lives and background knowledge of children in a South Island town, in big city suburbs, or growing up on a farm, and the different languages and cultural groups in Aotearoa.
Research tells us that an ideal assessment should be timely, fair, valid, credible, and enhance learning and wellbeing. Given the diverse lives of children, it can be challenging to craft test questions that ensure all these factors are present for everyone. If they aren’t, students might not be able to show everything they know and can do.
The diversity of life experiences also makes it difficult to design an assessment that students see as relevant to their lives and worthwhile to achieve. Other assessment considerations, particularly for national assessment, include efficiency and cost.
All this makes it hard to assess achievement accurately and to know which students are capable of the content being assessed and which are genuinely struggling.
Further complications arise in relation to questions that acknowledge and draw from te ao Māori, as knowledge, priorities, stories, and so on can differ by iwi. Research is helping us understand more about how cultural ways of being and doing can be at play in ensuring assessment is suitable. Recent research into assessment that is culturally sustaining for ākonga Māori has emphasised manaakitanga (relating to care, respect, hospitality) and wānanga (sharing knowledge).
Then there’s also the question of who does the assessment.
Assessment can be most valid when the person doing the assessing can be near the person being assessed and can observe and ask questions. Think, for example, of watching someone bake a cake – if the person doing the watching knows how to bake a cake, they can see and judge whether the person they are watching knows what to do and has the necessary skills.
Similarly, an expert in kapa haka can observe and assess the quality of a kapa haka performance, an electrician can watch an apprentice carry out an electrical task and assess whether it has been completed safely and appropriately, and a basketball coach can observe how a player does a lay-up shot and assess if they are using a suitable technique.
Knowledge of the person being assessed and their skills helps inform the assessor’s judgment of competence in each of these situations. The relationship between the person being assessed and the assessor can also help ensure the assessment is carried out in a way and location that enable that person to be sufficiently comfortable and confident to show their capabilities.
People suitable for assessing literacy or numeracy are those who have great literacy or numeracy expertise and know the person whose literacy or numeracy is being assessed – in a school example, it makes sense that this is the student's teacher of these particular skills.
However, while teachers may be best placed to make a valid assessment of their learners' literacy or numeracy, there is another factor at play – especially for high stakes assessment such as NCEA. We would need to know that every teacher is making the same judgments about a student's capability, and it can be difficult to generate public confidence in this idea.
In general, a simple and relatively cost-effective way to ensure public confidence in assessments is using the same assessment for everyone. But given both the varying life experiences of students, how can we do our best to make sure assessment is fair and valid for all?
This is the nub of what officials at the Ministry of Education and NZQA are currently grappling with. If there were easy answers to this dilemma, they would already be being used.
The 2022 pilot of the new assessment tools will have given great learning – learning likely to be in areas fairly typical when new assessment tools are trialled, such as how long students take to complete the assessment, implications for student familiarity with online testing, and considerations of terminology and use of contextual information around the literacy and numeracy tasks.
Such information will be useful for informing next steps. It was a pilot and this is the purpose of pilots – to try things out, find what works well, find what doesn't, and modify for a fit-for-purpose tool and more fair and valid outcomes. This is crucial because literacy and numeracy skills open doors to satisfying work and pleasure activities and help everyone manage their lives and contributions well.
