Since the end of last century, international large-scale assessment studies have been providing systematic information about countries’ education results. The news they convey regarding European education are not at all reassuring and have worsened with school closures and the disruption due to the recent Covid-19 pandemic. The size of the education losses and the distance of most European countries to other areas of the world, namely South East Asia, are often overlooked. A simple statistical analysis shows most European students are about one to two school years behind their peers in most advanced countries and regions, and the gap is increasing. These facts will negatively impact European workforce, economic competitiveness, and development. They will most likely also negatively impact mathematics higher education and research in our continent. To recognise the problem is the first step to cope with it.⁠1⁠This short article incorporates parts of a short report the first author wrote for the EMS Executive Committee shortly after PISA 2022 results were released on 5 December 2023. The note benefitted from some comments and suggestions from the EMS Education Committee. In any circumstance, the two authors assume fully responsibility for the statements and views here expressed.

This short article incorporates parts of a short report the first author wrote for the EMS Executive Committee shortly after PISA 2022 results were released on 5 December 2023. The note benefitted from some comments and suggestions from the EMS Education Committee. In any circumstance, the two authors assume fully responsibility for the statements and views here expressed.

International education studies results

Every four years since 1995, the IEA (International Association for the Evaluation of Educational Achievement) is organising the TIMSS (Trends in Mathematics and Science Studies) survey involving dozens of countries and assessing 4th and 8th graders in mathematics and science. The IEA later developed other studies for assessing other education areas and skills, e.g., reading. In 2000, the OECD (Organization for Economic Cooperation and Development) started the PISA (Programme for International Student Assessment) studies, regularly assessing 15-year-old students in many dozens of countries. PISA assesses mathematics, reading, science, and another selected rotating domain for each wave. It is held every three years, but the pandemic delayed the last one, which was held in 2022. We thus have a time series of more than 20 years, which allows us to compare performance and evolution of countries. Both PISA and TIMSS also report many data on students’ background and teachers and school variables. PISA, TIMSS and most other international survey results are reported on a scale with average 500 points and standard deviation 100. However, due to long-term declines, the current average no longer corresponds to 500 points. Some 690 000 students took the PISA assessment in 2022, representing about 29 million 15-year-olds in the schools of 81 countries and economies. PISA students are aged between 15 years 3 months and 16 years 2 months at the time of the assessment, and they have completed at least 6 years of formal schooling. Using this age across countries and over time allows PISA to consistently compare the knowledge and skills of individuals born in the same year who are still in school at age 15, despite their diversity. They can be enrolled in any type of institution, participate in full-time or part-time education, in academic or vocational programmes.

The OECD publishes very detailed analyses on PISA results. At this moment, there are already five volumes of the technical report (2000–2022) and four volumes on related PISA 2022 subjects [⁠6⁠ OECD, PISA. (2000–2024) ]. The microdata will be accessible in a few months’ time, but the available data and reports are already very rich and very detailed. Table ⁠1⁠ shows the average PISA mathematics literacy scores for selected European, North American and South East Asian jurisdictions.

Evolution of mathematics literacy in European, North American, and South East Asian selected jurisdictions. Data from OECD (2021–2023).

Figure 1. PISA 2022 (a) and TIMSS 2019 (b) mathematics literacy scores in selected European, North American and Asian countries (see Table ⁠1⁠ for country names). The vertical axis is the rank of the countries adjusted for the figure height. Canada’s value in TIMSS is the average of the Ontario and Quebec provinces. 🅭🅯 CC BY 4.0

PISA 2022 findings showed a major drop in knowledge and skills in Europe as measured by the OECD PISA tests. This drop is larger in mathematics (-18.8 score points) than in the other main PISA areas, i.e., reading (-14.2) and science (-4.6). The fall in European results is in line with that in North American countries, but larger than what happened in many other countries and regions. It is particularly visible that some countries and regions, namely Singapore, Macao, Japan, Taiwan, and Korea scored significantly higher than all other countries/economies in mathematics (575 to 536 points), and outperformed all other countries and economies. Overall, these South East Asian countries were able to sustain or even increase their 15-year-olds math literacy (see Figure ⁠1⁠). Another 17 countries also performed above the current OECD average (472 points), ranging from Estonia (510 points) to New Zealand (479 points). Equally striking is that Finland, which was considered by some educational currents to be a model country, continued its decline, initiated in 2006. Finland scored 484 points, below Slovenia and the Czech Republic, for instance. In Europe, Estonia (510) is maintaining its place above all other European countries. It is now clearly established that the pandemic took a toll on European mathematics learning, contradicting some optimistic early assessments. But the pandemic does not explain everything. Comparing countries with the same number of lockdown days, we notice that many European countries show regresses that are not solely explained by the pandemic.⁠2⁠OECD, PISA 2022 Database, Tables I.B1.2.1, I.B1.2.2, I.B1.2.3, I.B1.4.42, I.B1.4.43, I.B1.5.4, I.B1.5.5 and I.B1.5.6 [⁠7⁠ OECD, PISA 2022 Results (Volume I). (2023) , pp. 29–30].

Figure 2. Trends for PISA (a) and TIMSS (b) mathematics literacy in selected European, North American and South East Asian jurisdictions (see Table ⁠1⁠ for the list of countries used in each block). The “b” coefficient is the linear slope of the lines (score points/year). Colour bands and the 95% confidence intervals for the regression lines are shown, calculated from OECD and IEA data. 🅭🅯 CC BY 4.0

It is also of direct interest for mathematics education to look at TIMSS results, in particular at the TIMSS results for 8th graders in mathematics. These TIMSS country results are highly correlated to the PISA ones. It is thus not surprising that most European countries are not performing well and that Singapore, Taiwan, Japan are far ahead of European countries. Figure ⁠1⁠ (b) illustrates the gap between countries in South East Asia and Europe for math literacy, measured in TIMSS 2019.

It is also now clearly established that the learning losses in European countries are not a minor and temporary issue. Year after year, for longer than a decade, the decline is consistently shown in PISA and TIMSS surveys and should be seriously tackled (see Figure ⁠2⁠).

Long-term trends

Decline in PISA results in mathematics have been almost constant for European countries along the last decade. Decline from 2018 to 2022 happened in all European countries, with an average drop of 18.8 points (see Table ⁠1⁠ and Figure ⁠2⁠). For reference, it is usually considered that a decline of 20 to 30 points corresponds to a loss of a school year instruction. We can also estimate, for instance, that Singaporean students, who are scoring 575 points, are about three to almost four school years cognitively ahead of European students, who scored on average 480 points. This difference in not inevitable, as the decline of student results is not inevitable. Some countries such as Singapore, Taiwan, and others are showing a steady long-term improvement. These global results represent a serious warning to European education, science, competitiveness, and mathematical research.

More recently, OECD released the results for the fourth domain surveyed in this PISA wave: Creativity. The results are also important for mathematicians. It is many times said that South East Asian countries are ahead in school mathematics because they concentrate on rote fact memorisation and rote procedural practices. They would lack creativity, which is a major drive for technological innovation and economic competitiveness. Results for the creativity category are very interesting in this regard. Singapore is ahead of all other countries, both in school mathematics and in creativity as measured by PISA. Other Asian countries are also ahead in both categories. Furthermore, data show a significant correlation between countries’ results in these two areas. Figure ⁠3⁠, retrieved from OECD publications, is very illuminating.

Figure 3. Mean creative thinking and mathematics performance. OECD Notes: Only the 64 countries and economies that implemented the creative thinking cognitive test are shown. A student’s relative performance in creative thinking is defined as the residual obtained upon a cubic polynomial regression of the student’s performance in creative thinking over his or her performance in mathematics or reading. The regression is performed at an international level, pooling data from all countries and economies that participated in the creative thinking assessment. Source: OECD, PISA 2022 Database, Tables III.B1.2.1. and III.B1.2.4.⁠3⁠https://stat.link/o12ktl ©

Why should mathematicians be worried

There are several reasons why we should be worried by these results. Primarily, there are issues concerning the opportunities provided for future generations. No one should be satisfied when their nation’s youth is not having the best education possible. We, as European citizens, should be worried about our youth’s future. Secondly, there are economic and social issues. We now know from several statistical and econometric research projects that human capital formation, as measured by PISA and other standardised measures, has a decisive impact on the economic foundations and development of countries. See, e.g., [⁠4⁠ E. A. Hanushek and L. Woessmann, The knowledge capital of nations: Education and economics of growth. The MIT Press, London (2015) ] for a thorough review. Thirdly, there is one seemingly egotistic reason. As professors and researchers, a suboptimal mathematics education means suboptimal and harsh class teaching, fewer talented students, fewer graduate students, and less productive research [⁠3⁠ N. Crato, Math curriculum matters: Statistical evidence and the Portuguese experience. Eur. Math. Soc. Mag. 124, 49–56 (2022) ]. Although one can dismiss these concerns saying what matters for professional mathematics is simply a small pool of good minds, this pool becomes smaller and smaller in Europe as worse results on country averages reduce the availability of high achievers. None of these motives, though, is actually egotistic. Decades of research have highlighted the importance of both success in basic skills and success in preparing talented people, as one factor reinforces the other and both contribute to economic and social development. As Hanushek and Woessmann summarise in [⁠4⁠ E. A. Hanushek and L. Woessmann, The knowledge capital of nations: Education and economics of growth. The MIT Press, London (2015) , p. 64], “achieving basic literacy for all may well be a precondition for identifying those who can reach ‘rocket scientist’ status” and “a large pool of those with basic skills may be an efficient way to obtain a large share of high performers.”

Overall, the decline in math literacy among European students poses a significant threat to scientific progress, economic development, educational quality, societal well-being, and the future of research and academia in Europe. Addressing this issue is crucial for sustaining innovation, competitiveness, and a well-informed and capable society.

And now, what?

The problems with mathematics education in Europe have been known for quite some time, and PISA and TIMSS surveys are constantly showing red flags about our performance. To improve the situation, there are a couple of fundamental basic steps already, proven to work elsewhere [⁠1⁠ N. Crato, Curriculum and educational reforms in Portugal: An analysis on why and how students’ knowledge and skills improved. In: Audacious Education Purposes, pp. 209–231, Springer, Cham (2020) , ⁠5⁠ J. Marôco, Portugal: The PISA effects on education. In Improving a Country’s Education [2], pp. 159–174, Springer, Cham (2021) ] and that can be recommended after the last PISA wave [⁠2⁠ N. Crato (ed.), Improving a country’s education: PISA 2018 results in 10 countries. Springer, Cham (2021) ]:

  • Follow the modern scientific evidence about focused, progressive and coherent curriculum, direct instruction methods and student evaluation.

  • Promote more rigorous and demanding stable curricula, better structured and sequentially coherent, and significant.

  • Increase student awareness of mathematical knowledge importance and applicability.

  • Pay special attention to equity in education, not only equity between genders, but also work against the fact that socio-economic and other circumstances still hinder the education achievement of disadvantaged students.

  • Follow with special attention students that are staying behind and provide them content knowledge tutoring, instead of lowering curricular standards.

  • Equally follow with special attention students that are in the highest performing levels and provide them with opportunities for further deepening their knowledge.

  • Better evaluate the status of mathematics education in our countries through high-stakes and low-stakes frequent and rigorous student assessments.

  • Press for a more complete and rigorous initial teacher training in mathematics, stressing the need for the topics they will need to teach at the elementary and secondary school levels.

Nuno Crato is a research professor of mathematics and statistics at CEMAPRE, ISEG, University of Lisbon. He is also the president of Iniciativa Educação, and member of the Conseil Scientifique de L’Éducation Nationale of the French government. ncrato@iseg.ulisboa.pt João Marôco is a full professor of statistics and research methods at ISPA – Instituto Universitário. He has been extensively researching international large-scale assessments (ILSAs) and their impact on educational policies. jpmaroco@ispa.pt

  1. 1

    This short article incorporates parts of a short report the first author wrote for the EMS Executive Committee shortly after PISA 2022 results were released on 5 December 2023. The note benefitted from some comments and suggestions from the EMS Education Committee. In any circumstance, the two authors assume fully responsibility for the statements and views here expressed.

  2. 2

    OECD, PISA 2022 Database, Tables I.B1.2.1, I.B1.2.2, I.B1.2.3, I.B1.4.42, I.B1.4.43, I.B1.5.4, I.B1.5.5 and I.B1.5.6 [⁠7⁠ OECD, PISA 2022 Results (Volume I). (2023) , pp. 29–30].

  3. 3

    https://stat.link/o12ktl

References

  1. N. Crato, Curriculum and educational reforms in Portugal: An analysis on why and how students’ knowledge and skills improved. In: Audacious Education Purposes, pp. 209–231, Springer, Cham (2020)
  2. N. Crato (ed.), Improving a country’s education: PISA 2018 results in 10 countries. Springer, Cham (2021)
  3. N. Crato, Math curriculum matters: Statistical evidence and the Portuguese experience. Eur. Math. Soc. Mag. 124, 49–56 (2022)
  4. E. A. Hanushek and L. Woessmann, The knowledge capital of nations: Education and economics of growth. The MIT Press, London (2015)
  5. J. Marôco, Portugal: The PISA effects on education. In Improving a Country’s Education [2], pp. 159–174, Springer, Cham (2021)
  6. OECD, PISA. (2000–2024)
  7. OECD, PISA 2022 Results (Volume I). (2023)

Cite this article

Nuno Crato, João Marôco, Should mathematicians worry with PISA and TIMSS math results?. Eur. Math. Soc. Mag. 133 (2024), pp. 42–47

DOI 10.4171/MAG/210
This open access article is published by EMS Press under a CC BY 4.0 license, with the exception of logos and branding of the European Mathematical Society and EMS Press, and where otherwise noted.
Back to Issue 133