Report of the ICME15 survey on challenges and perspectives of mathematics assessment

This article provides a short summary of the work of one of five survey teams commissioned by the organisers of ICME15. The findings were presented at the congress held in July 2024 in Sydney, Australia. Survey Team 1 worked on “Challenges and perspectives of mathematics assessment.” The team leader was Kaye Stacey, and the members were Yuriko Yamamoto Baldin (Universidade Federal de São Carlos, Brazil), Kim Koh (University of Calgary, Canada), Ruhama Even (The Weizmann Institute of Science, Israel) and Ross Turner (The Australian Council for Educational Research, Australia).

Assessment has very many forms, with many players involved, and it is conducted for multiple purposes, so this survey could consider only a small part. The chosen focus is the teaching-learning-assessment cycle as handled by classroom teachers at any level of mathematics. The assessment is of mathematical knowledge and the intended audience for the report from the assessment is the teacher and/or the learner with the purpose of informing future teaching or learning. To distinguish assessment from the monitoring of students’ understanding that teachers carry out continuously, we require some formality to the process (e.g., by recording results). Summative assessment for credentials or for evaluation of education systems or to provide information for next year’s teacher is beyond the scope of this survey.

The survey consisted of five separate components. We begin with formative assessment (FA) which sits right at the heart of classroom teaching. Ruhama Even presented results from a systematic survey (following PRISMA guidelines) of literature published from 2017 to 2023 on teachers’ use of assessment for informing short-term instructional decisions. The literature search focused on two lines of inquiry with which this topic is associated in the mathematics education literature. One is formative assessment (FA), which involves three key teaching competencies: (1) eliciting evidence of student learning, (2) interpreting evidence of student learning, and (3) responding or acting on evidence of student learning. The other is the extended approach to teacher noticing (TN), which involves three similar competencies: (a) attending to the details in children’s strategies, (b) interpreting children’s understandings, and (c) deciding how to respond (or actually responding). Of an initial 2990 papers, 105 papers from 10 top-tier mathematics education sources were found to meet the criteria. About a quarter of them involved international collaboration. Although the similarity of FA and TN competencies suggests a close connection, the review reveals a surprising disconnection. Authors who write about FA rarely link their work to TN and vice versa. The number of papers on TN increased steadily over the period, clearly outnumbering FA. Additionally, the survey identified an important absence: teachers’ responses and acting on evidence of students’ learning are rarely studied in real classrooms.

Ross Turner provided a sweeping survey of how learning progressions (LPs) are used in assessment. He exemplified this by sharing his experience in working on the mathematics LP developed by the Australian Council of Educational Research (ACER).⁠¹⁠https://learning-progression-explorer.acer.org LPs have attracted considerable recent attention among education researchers, practitioners and policymakers. Research on LPs forms part of a long tradition of scholarship to describe trajectories for teaching and learning mathematics, especially the order in which concepts and skills are learned and mastered as well as common states of partial knowledge along the way. Research also explores how such descriptions of learning can guide and inform education. LPs vary in their scope, from a focus on a single concept or skill, through to attempts to describe learning across the whole domain of mathematics. There are different approaches to their development. For example, different ways of using learner data and expert input and different measurement models are used. There are also different uses intended, from tools to support formative assessment to descriptions to guide curriculum development.

The work at ACER is grounded in measurement principles and leverages extensive experience with described reporting scales for assessments. The ACER mathematics LP describes the key milestones in mathematical learning from the early years through to the end of compulsory schooling. It is designed to assist with curriculum design and syllabus content (including internationally in relation to the United Nations Education 2030 agenda); to help identify the current knowledge of individual learners; to describe and report learning progress of individuals; to guide teachers in the selection or development of suitable teaching and learning resources; and also to support teacher professional learning. Action research is continuing on these fronts. The long-term challenge is for LPs to offer support to teachers to implement their craft in such a way that better treats individual learners at their point of need, rather than as members of a group who ‘should’ be at a certain point in their learning.

Kim Koh presented the findings of a systematic literature review on assessment of 21st century competencies for mathematics and higher-level mathematics thinking skills and competencies (together labelled MTSC). Curricula from around the world now highlight the importance of supporting students to develop a deeper mathematical understanding with competencies such as critical thinking, creativity, collaboration, and communication, and to help them become competent, confident, and creative users and communicators of mathematics. Teachers are urged to adopt assessments that provide authentic experiences for all learners. Inquiry-based learning and modelling approaches are also deemed to be affording students with the opportunity to develop competencies. Yet, questions remain about how students’ competencies are assessed and what form assessments can take.

The review included empirical studies published from 2018 to 2023 focusing on assessment of MTSC for K–12 students and preservice teachers. From an initial 2489 papers, 87 were found to meet the criteria. The review identified the skills and competencies that were assessed, and the tools and strategies used. Approximately half the studies were published in research journals specifically focused on mathematics education. Quantitative, qualitative and mixed methods were well represented. More than half involved students in grades four to nine and more than half involved more than one content area. A third of articles published after the pandemic addressed online assessment. A first outcome of the study was to clarify the construct of MTSC, observing that its components should neither be treated as atomized competencies nor a ‘laundry list,’ and that there are both cognitive and affective competencies. Since 72% of studies focused on cognitive skills, Koh calls for more research into innovative assessment of students’ socio-emotional skills and motivation, considering them simultaneously in classroom-based formative and authentic assessments.

Yuriko Yamamoto Baldin reported on a recent online survey on assessment in teacher education. There were 38 respondents from seven Latin-American countries (Brazil, Ecuador, Mexico, Venezuela, Costa Rica, Peru and Columbia) and from Portugal and Spain. Nearly all the respondents work in initial or continuing teacher education and most undertake mathematics education research. Designing and using assessment that attends to 21st century demands is especially significant to several countries in the region with recent curricular reforms and where COVID-19 changed teaching and learning dynamics. The survey examined changes in national assessment policy, how assessment (especially FA) is treated in teacher education, and guidance on assessment for teachers using active learning, problem-solving, or modelling methodologies. The concerns that most respondents shared will be used as the basis for further study.

Most respondents noted the lack of cohesion between curriculum demands and the reality of classroom practices. They urged better preparation for teachers to understand practices for FA, and to plan assessment instruments. They especially noted the need to link FA to the active teaching and learning methodologies essential for students to achieve 21st century competencies. Respondents generally believed that it was not productive to use external large-scale assessments with uniform ranking criteria that do not take local cultural, political and educational contexts into account. These put pressure on national education policies, often without much analysis of the roots of the difficulties.

Finally, Kaye Stacey provided a brief overview of some of the major changes to assessment within the teaching-learning cycle that are already being implemented using new technologies and some that seem just around the corner. The changed conditions of school and university education imposed by the pandemic greatly accelerated these changes. The work of Michael Obiero Oyengo (Maseno University, Kenya) was one case presented. By using a computer-algebra based assessment tool, he is providing students in extremely large classes with regular and immediate mathematically-detailed feedback on their own work and as many opportunities for practice as they wish. Most students access this with mobile phones. Such formative assessment has never previously been possible and may change the mathematics learning experience of millions of students around the world. Further details relating to all these contributions are available from the author and survey team members.

Emeritus Professor Kaye Stacey was the leader of the five member ICME15 survey team on assessment. In addition to assessment, her interests centre around mathematical thinking and problem-solving, the school curriculum and new technologies. She is author of many articles and books in mathematics education, reporting research and for teacher professional development. She was Foundation Professor of Mathematics Education at the University of Melbourne and holds several awards including ICMI’s Emma Castelnuovo Medal. k.stacey@unimelb.edu.au

1. ¹

   https://learning-progression-explorer.acer.org