Math curriculum matters: Statistical evidence and the Portuguese experience

1 Introduction

In a well-known dialogue from the pen of the mathematician-turned-writer Lewis Carroll, Alice asks the Cat: “Would you tell me, please, which way I ought to go from here?” “That depends a good deal on where you want to get to,” said the Cat. “I don’t much care where –” said Alice. “Then it doesn’t matter which way you go,” concluded the Cat.

Any reasonable and mature person understands it perfectly: With clear goals, it is easier to progress. Any business- or management-oriented person understands it perfectly: When we set up goals, plans, and monitor a schedule, we progress faster. Any experienced teacher worried about student’s progress understands it perfectly: With clear curricular goals and planned lessons, students advance faster and in a consistent way.

However, this Alice dialogue is a paradigm of what happens frequently in educational debates. More often than not, curricular goals are derided in favour of generic “skill development”. “Curricular flexibility” is a pretext for not conforming to generic curricular goals. The so-called deep understanding is often a pretext for eschewing the assessment of curricular attainment.

In this paper, I will describe a positive experience in the Portuguese education system along the years 2003–2015, namely from 2011 to 2015, and its backlash after 2016.

2 A push for quality over quantity

Throughout the twentieth century, Portugal struggled against abackwards education environment. In 1970, almost 18 % of the population was still illiterate, 66 % of the 15-year-old hadn’t completed any level of formal education, and only 0.9 % of the total population had a higher education degree.⁠¹⁠Pordata, www.pordata.pt, consulted 23 July 2021.

The progresses made during the last decades of the twentieth century are extraordinary. Following the general improvement of economic conditions after the 1960s, the euphoria of a baby boom, a newly installed democracy, and the arrival of European structural funds, we witnessed a school expansion and a complete change of the country. In 30 years, illiteracy dropped from 18 % to 9 %, the percentage of 15-year-olds without any level of formal education dropped from 66 % to 9.2 %, and the fraction of the population with higher education raised from 0.9 % to 8.4 %.⁠²⁠Here and in the following description, I draw heavily from my previous paper on the Portuguese experience [⁠1⁠ N. Crato, Curriculum and educational reforms in Portugal: An analysis on why and how students’ knowledge and skills improved. In Audacious Education Purposes, edited by F. Reimers, Springer, Cham, DOI 10.1007/978-3-030-41882-3_8 (2020) ].

All these successes were essentially quantitative, i.e., they democratized education, expanded the school system, and increased schooling years. But did they bring youngsters to a reasonable level of literacy? A debate about the quality of education began dividing the country.

Shocking news arrived late, with the first international large-scale assessments (ILSA). In 1995, TIMSS showed Math 4th grade Portuguese students at the bottom of the scale, with only two countries, Iran and Iceland, behind⁠³⁠TIMSS (Trends in International Mathematics and Science Study) is a large-scale assessment designed to inform educational policy and practice by providing an international perspective on teaching and learning in mathematics and science. TIMSS is a project of the International Association for the Evaluation of Educational Achievement (IEA) and is directed by the TIMSS International Study Center at Boston College in collaboration with a worldwide network of organizations and representatives from the participating countries.. In 2001, as PISA⁠⁴⁠PISA is the Organization for Economic Co-operation and Development (OECD) Programme for International Student Assessment. Started in 2000, every three years, it now tests 15-year-old students from almost the entire world in reading, mathematics, and science. The tests are designed to assess how well students master key subjects to be prepared for real-life situations in the adult world. 2000 results were released, Portugal saw its students’ score below participant countries average and much below the OECD average in all three areas (literacy, mathematics, and science).

During the ensuing years, the debate continued and took many shapes. Reforms in the school system were at times contradictory and served different purposes, but until 2015 they essentially went in one direction: to pay more attention to the results [⁠5⁠ J. Marôco, Portugal: The PISA effects on education. In Improving a Country’s Education: PISA 2018 Results in 10 Countries, edited by N. Crato, Springer, Cham, DOI 10.1007/978-3-030-59031-4_8 (2021) ]. Several changes propelled the attention to education outcomes.

In 2001, the government was legally forced to release school exam outcomes that previously were hidden from the public. This was a game changer, as it increased public awareness of the diversity in schools’ quality and put pressure on schools and teachers to improve education results. In 2003, a new minister established exams for mathematics and reading at the end of compulsory schooling (then the 9th grade). In 2005, the ministry established priority of mathematics and reading subjects and developed special plans to support teaching of these disciplines. In 2010, a new minister introduced in Portugal the first learning standards, following an Anglo-Saxon lead. These standards did not replace the existing curriculum; they simply provide it with a clearer structure.

In Portugal and other countries with a highly centralized system, the curriculum is usually subject-based and essentially consists of a set of official documents called programs (“programas”), which detail the topics to be covered in each school discipline or subject. Standards typically organize the course contents sequentially, highlighting the learning goals, and the achievement level desired for each content. They refrain from pedagogical recommendations and favour the setting of detailed learning outcomes.

The first learning standards appeared in 2010 and 2011. Although they represented a progress with respect to the vague programs in place at that time, they still included pedagogical recommendations mixed up with learning outcomes; they still didn’t clearly highlight knowledge goals, and they still were vague in some areas.

By December 2010, PISA 2009 results were published and showed an important improvement in all PISA areas. Some analysts stressed the importance newly instituted 9th grade exams may have had, while others emphasized the role of the new policies of increased attention to results in basic subject areas. In my opinion, both points are correct.

A financial and political crisis exploded in early 2011. Portugal was coming to grips with the most serious financial crisis of its recent history. In June 2011, elections were held, a new majority was formed, and a new prime minister was chosen. I was then appointed as an independent minister. Budget was as tight as possible.

Against this background, the education policy had to be very clear and focused. From the start we decided that we should try to “do better with less”, i.e., we should focus on the quality of education.

3 To push for quality under budgetary constraints

We can group the main reforms put in place from 2011 to 2015 into five areas. I still believe the most important one was the setup of a clear and demanding curriculum, allowing a rigorous assessment of students. I will develop this idea in the next section, briefly summarizing now the main reform areas.

First, curriculum. As I will describe better in the next section, we designed an increasingly demanding and structured curriculum.

Second, assessment. When standards related to the curriculum are well defined and well structured, it is possible to align assessment with these standards. This way, assessment can be more rigorous and act as a reference to teachers, students, and parents. The introduction of rigorous, frequent, and varied assessment tools was a crucial part of the 2011–2015 reforms.

Third, a plan for success promotion. In parallel to striving for higher academic standards, we devised a series of measures to improve students who trailed behind and, at the same time, to allow more advanced students to thrive by pursuing some of their specific interests. These measures were set out as early as 2012 in a special law,⁠⁵⁠“Decreto-Lei 176/2012 de 2 de agosto.” and complemented by regulatory legislation that made compulsory the support to students with academic difficulties.

Fourth, school autonomy with incentives tied to students’ improvement. After 2012, we developed a complex system to increase resources allocated to schools as they proved to be able to improve students results with these additional resources. Autonomy allowed schools to freely use their resources to put in place the promotion-of-success measures previously referred to.

Fifth, parallel offers and vocational tracks. As compulsory schooling was extended from 9th to 12th grade, vocational high-school tracks became a choice for the three years of senior high school.

4 Everything starts with the curriculum

There are many definitions of curriculum, starting from a more restricted one, which usually understands curriculum as the specification of what is intended to be taught and learned in academic terms, to more general ones, in which the very generic purposes are considered, and methods and materials are included (see, e.g., [⁠3⁠ A. V. Kelly, The curriculum: Theory and practice. 6th ed., Sage (2009) ]). For our purposes, we do not need a very precise definition, and will use the common restricted version just outlined.

The characteristics we intended and believed to have essentially succeeded in having in the curriculum are the following.

First, the set of courses offered from 1st to 12th grade should prioritize the commonly accepted essential subjects and add complementary subjects such as information technology and the sorts. Among the essential subjects, we may highlight reading, literature, grammar, and writing; arithmetic, geometry, algebra, and basic probability and statistics; country and world basic history; geography; sciences, physics, chemistry, biology, earth sciences; arts and basic art history.

Second, the foundational subjects such as reading and elementary mathematics should receive special attention. All subjects are important, but some have precedence along the school years and priority, as they are essential to a civilized life and to progress in the studies.

These two characteristics seem trivial and indisputable, but unfortunately this is not true. Many educational currents abhor the idea of separated subjects and would like to constantly mix all subjects. These educational currents particularly reject the idea of having foundational subjects.

Third, curricula should be organized into different subjects, with internal coherence regarding their fields. This seems obvious, but postmodern currents have attacked the idea of organized knowledge. From their point of view, organized subject knowledge is reductive and should be abandoned.

However, organized field knowledge is the way humans found to progressively understand reality. Obviously, when we look at nature, we do not have a biological phenomenon developing outside of a physical world and outside a given planet climate, to name just a few related areas. But these spheres of phenomena have differences and the way we cope with reality is exactly by partitioning it and by studying piece by piece parts of the moving world.

Transmission of school knowledge adds another constraint: knowledge communication should be facilitated by breaking complex concepts into simpler ones and by organizing them progressively. One of the most important findings in modern educational psychology is that comprehension operates through a narrow channel of working memory which only supports a limited cognitive load (see, e.g., [⁠4⁠ P. A. Kirschner and C. Hendrick, How learning happens: Seminal works in educational psychology and what they mean in practice. Routledge (2020) ]).

Fourth, each subject needs an internal coherence. This is most obvious for mathematics, but the same applies to all subjects. As an example, consider learning a foreign language. Obviously, there are many possible progression paths. But if a teacher starts by introducing her/his students to basic day-to-day vocabulary and a few verbs related to everyday house life, she or he cannot immediately assign readings related to foreign travel. As another example, consider the study of Brazil’s independence. To understand the basic forces behind the independence, a student must first be introduced to Brazil’s colonization, King D. João VI’s escape from Lisbon to Rio de Janeiro in the aftermath of Napoleon’s invasion of the Iberian Peninsula, the long settlement of the crown court in Rio, the development of this city, and the return of the king to Portugal 14 years later. This interdependence can be studied in many ways, but if we want students to have a basic understanding of history as an evolution of trends and not as collection of facts, the study must have an internal coherence.

In mathematics, all this is even more relevant, as mathematics is a hypothetical-deductive discipline. In mathematics there are many ways of establishing logic sequences, but the interdependence is crucial. We usually start with definitions that set up the ground for construction of a theory, then explore basic properties, and establish mathematical facts by logical deduction.

One can be more rigorous or less rigorous, according to the students’ level and the purpose of the specific course (basic algebra for vocational training is different from college preparation algebra). One can stress the formal aspects of theorem proving or stress the computational aspects of a particular topic. But in all cases, the development of topics must be internally coherent with the definitions formulated at the start and the various facts established previously.

To give an elementary example, let us consider the definition of the logarithm. One can start with exponents,

or with the integral

or proceed formally, e.g., define the logarithm as a strictly monotonically increasing function $L:R^{+}\to R$ with the following property for all $x,y\in R^{+}$:

Now, suppose a teacher asks a student to prove that $\log (xy)=\log x+\log y$. The expected answer differs according to the definition the class is using. In case they are using the last described definition, the correct answer would simply be “it is the definition”.

Mathematics is a structure. By developing only “mathematics experiences” without a structured curriculum, our students would perceive mathematics as a series of disperse facts, or even of disperse tricks. They would hardly learn mathematics. At best, they will acquire a superficial knowledge of mathematical concepts.

Fifth, curriculum should be organized in a progressive way, translating the internal coherence and logic of the discipline. There are of course many equally valid alternative sequences, but the curriculum should present at least one of these possible sequences. A teacher may choose a slightly different path at her or his own risk, but this requires a deep knowledge of the subject, a reasonable experience, and some art. Curricula are tasks for expert groups.

In Portugal, we redesigned the curricula by forming expert groups that included university professors of the subject (professional mathematicians, or historians, or biologists, …), experienced teachers, and experts in the science of teaching, namely educational psychologists. This departed with the recent usual practice of having curricula designed by so-called “educational experts”, i.e., educationalists with no solid knowledge of the subject areas and often with a deep ideological non-scientific slant.

Sixth, curricula should be ambitious, knowledge rich, envision a deep understanding of the concepts and procedures, and help the grasp of the structure of the different subjects. This departs from the recent practice of trying to develop skills without knowledge, as if the ability to apply or develop a mathematical practice could be achieved without any particular substantive knowledge.

Seventh, curricula should be translatable into assessment and auxiliary materials, namely textbooks. This means that curricula cannot be vague, otherwise they would be useless for guiding teachers, textbook authors, exam authors, and even families involved in helping their children.

This last point means we are seeing curriculum as the centre of a pedagogical coherence: textbooks and other materials, assessment, including standardized testing, all these should be coherently aligned with the curriculum, turning this set of tools into a coherent instrument for education improvement.

5 After the continuous improvement from 2003 to 2011, a sharp progress from 2011 to 2015 and a decrease after 2016

Figure ⁠1⁠ shows the results of PISA assessments in all three areas along the seven PISA waves that started in 2000 and had the last survey in 2018.

I have already described the main forces that led to this great progress along the years. In essence, after the 2000 PISA shock, all governments paid increased attention to students’ results and searched for ways of supporting the basic disciplines development and assessment. Two important factors for improvement played a role: one is the introduction of 9th grade math and reading exams, which led to the sharp increase in 2009, the other is the improvement in the structure of the curriculum, with better standards accompanied with additional standardized assessment in 4th and 6th grades. Later, one cannot help noticing a visible decrease in 2018 results.

In the preface to the PISA 2018 report, one can read a surprising praise to Portuguese education achievements. It’s worth quoting it: “given the fact that expenditure per primary and secondary student rose by more than 15 % across OECD countries over the past decade, it is disappointing that most OECD countries saw virtually no improvement in the performance of their students since PISA was first conducted in 2000. In fact, only seven of the 79 education systems analysed saw significant improvements in the reading, mathematics, and science performance of their students throughout their participation in PISA, and only one of these, Portugal, is a member of the OECD.” This is what the former secretary-general of the OECD had to say.

As a Portuguese citizen and a former minister of education, I can only be elated at this acknowledgement. We conclude that my country is the only OECD country that has constantly improved in PISA. But as an observer, I must be surprised.

Portugal has been improving its results from to 2000 to 2015. However, 2018 is exactly a date when we witnessed a decline: a statistically significant decrease in science, a visible downturn in reading, and a stagnation in mathematics. Why is this precisely the time when we deserve such a public praise from the OECD at its highest level? One can speculate and read on it politically coloured messages, but these statements are surely misleading and similar to frequent misleading references to another country’s education policies, namely Finland.

To place the PISA assessments into perspective, it is useful to have a global panorama of PISA math results. Figure ⁠2⁠ presents the evolution of PISA scores for an illustrative set of countries.

Singapore has been at the top, along with a couple of Asian countries and regions, such as South Korea and Macao. For a few years, Finland was the only European country that was close to those top performing countries. Now, it is Estonia that is at the top of European results. Countries such as Portugal and Spain are close to the OECD average. Most rich and middle-income countries have been at a similar level⁠⁶⁠Detailed data for PISA and other international surveys are readily accessible at the International Explorer of the National Center for Education Statistics: https://nces.ed.gov/surveys/international/ide/. An overview can be found in [⁠2⁠ N. Crato, Improving a country’s education: PISA 2018 results in 10 countries. Springer, Cham, DOI 10.1007/978-3-030-59031-4 (2021) ].. Given the cultural differences between Europe and East Asia, it is natural that Europeans look up to countries such as Finland.

If we want to understand what brought the Finnish system to the comparatively high-performance levels it enjoys in the European panorama, we should look to what Finland did in the past, and not to what Finland is doing now. Given the general laudatory praises Finnish education receives in the western press, it will be probably come as a surprise to the reader the fact that Finland has been declining continuously since 2006 in all three main areas.

This reality can be seen in Figure ⁠3⁠, from the OECD itself, which reveals declines of 41 points in mathematics, 27 points in reading, and 41 in science. To put things into perspective, experts usually estimate that a 30-point change is roughly equal to the difference between two successive school years. We would conclude that in the last 10 or 12 years, Finnish education got worse. Roughly: middle school 15-year-old students now know about the same as 14-year-old students knew in 2006. It is highly debatable whether recent Finland education innovations are an example to follow!

However, many educationists praise current lax programs, multidisciplinarity over disciplinarity, phenomenon-based learning and discovery learning, i.e., praise the changes in Finnish education that are concomitant with its decline (see, e.g., [⁠9⁠ G. H. Sahlgren, Real Finnish lessons: The true story of an education superpower. Centre for Policy Studies, Surrey, UK, www.ifn.se/media/0irhqejv/2015-sahlgren-real-finnish-lessons.pdf (2015) , ⁠8⁠ A. Saarinen, Equality in cognitive learning outcomes: the roles of educational practices. Doctoral dissertation, University of Helsinki, https://helda.helsinki.fi/handle/10138/320436 (2020) ]). We should instead praise the demanding teacher training and teacher selection, the evaluation system, and the valorization of disciplinary knowledge that prevailed during the last decade of the 20th century and the first years of the 21st century – exactly what many educationists abhor, but exactly what raised Finland to its extraordinary level [⁠7⁠ T. Oates, Finnish fairy stories. Cambridge Assessment, Cambridge, UK, www.cambridgeassessment.org.uk/Images/207376-finnish-fairy-stories-tim-oates.pdf (2015) ]. And do not take this in the wrong sense: Finland’s is still a great education system by western standards.

All these points are important when we discuss any national education policy. It is not only the Finnish system that is at stake; it is any other country in the world that would like to learn from international experiences.

I obviously know better the Portuguese situation and history, so I return to this country’s experience and now review TIMSS 4th grade math results.

Unlike PISA, which surveys general application skills of 15-year-old students in three main areas and every three years, TIMSS is grade based and more tied to the curriculum. Every four years, it surveys the 4th and 8th grades, both in mathematics and science. Portugal joined TIMSS in 1985 and got terrible results. Only in 2012 it re-joined TIMSS. We are particularly interested in math, as the general purpose of this article is to understand the evolution of this discipline. Additionally, up to 4th grade, sciences are not a particular focus of elementary school.

As we can see in Figure ⁠4⁠, Portuguese students progressed sharply up to 2015. This global picture highlights how the country was able to improve its education system during the last decades. The year 2015 is particularly notable since Portuguese students outperformed their Finnish counterparts. This is partly due to the continuous decrease in Finnish results we can also observe in the graph, but it is nonetheless notable that a medium performing country as Portugal could surpass a still high performing European country such as Finland.

Unfortunately, we also observe a significant decrease in 2018, when the results were even lower than in 2011.

Having described the main factors that led to a remarkable improvement in Portuguese education, specifically in mathematics, it is also important to interpret the decline in results observed both in PISA and TIMSS from 2016 onwards.

The education contexts in 2011 and 2016 could hardly be more different. In 2011 we were building upon a general progress with a better focus on the results, with increased curricular rigour and more evaluation. We were deeply overwhelmed by a financial crisis and the mood of the country was to fight to improve all results, even if with less resources.

The 2011–2015 ministerial team always stressed that we were building upon previous results, but we needed to improve the curriculum and to have better assessment [⁠6⁠ A. Morris, Student standardized testing: Current practices in OECD countries and a literature review. OECD Working Paper Series No. 65 (2011) ].

The curriculum was improved along the orientation described in the previous section. To be specific, we focused on the fundamental subjects, namely reading and mathematics, we better structured each discipline’s program, and we designed standards that established a clear and measurable progression within each subject.

Assessment was improved by creating an evaluation institute⁠⁷⁠Instituto de Avaliação Educativa, IAVE, created by Decreto Lei nº 102/2013, de 25 de julho. with greater autonomy and technical expertise on modern statistical assessment methods, namely item response theory. Assessment was made more frequent and more stable, so that student outcomes could be compared from year to year.

In 2016, the country was coming out of a difficult period and the new parliamentary majority changed course. In education, two of the four standardized assessment moments were abolished. The entrance exam for new teachers was also abolished. The new government directly and publicly attacked previous orientations that were deemed as “elitist”, “unrealistically ambitious”, and “too content focused”, avoiding the “competencies” approach. Instead of focusing on subject knowledge and subject coherence, the ministry then adopted a focus on multidisciplinarity and practical skills and competencies, and eschewed assessment, which was classified as “narrow” and “detrimental to the socioeconomic unfavoured classes”.

Later, even after PISA results were made public, the government abolished all programs and standards, replacing them with vague and ill-structured “Essential Learnings” (“Aprendizagens Essenciais”).

The country’s education mood changed completely, both in administrative orientation and public perception. It is clear that this change is associated with the immediate fall in education outcomes.

6 Inequality and fairness

One of tenets of the opposition to an ambitious and structured curriculum and to standardized assessments is the faulty idea that ambition and evaluation harm students coming from less privileged classes [⁠6⁠ A. Morris, Student standardized testing: Current practices in OECD countries and a literature review. OECD Working Paper Series No. 65 (2011) ]. It is worthwhile to examine what happened to low-performing students with the 2016 change in education policy.

It is clear that the high-performers fraction of the students moved into a favourable direction along the years up to 2015, i.e., this fraction increased, but changed course after this year. Although at a much lower rate, it kept the positive movement for mathematics, following the general trend in PISA. As mathematics is a highly cumulative discipline, it is possible that mathematics scores kept more momentum than science and reading.

It is also clear that the low-performers faction of the students moved into a favourable direction along the years up to 2015, i.e., this fraction decreased, but changed course after this year. Although at a lower rate, it kept the positive movement for mathematics, following the general trend in PISA. Again, the mathematics momentum slowed down the relapse.

PISA assesses the general applicable knowledge of 15-year-old, and it is less tied to the curriculum than TIMSS. It is worthwhile to see what happened with 4th grade students as we moved from 2011 to 2015 and 2019. The mathematics momentum I just referenced above for students who have been in school for about nine years would surely be less influential. Results in Table ⁠2⁠ are also very interesting.

In TIMSS, the results are unequivocal. The fraction of high performers increased from 2011 to 2015 and decreased from 2015 to 2019. The fraction of low performers decreased from 2011 to 2015 and increased from 2015 to 2019.

What happened is what economists call a “natural experiment”. There are two groups of students in this table. In the first group, students have entered 1st grade in 2011 and have been assessed in 2015 at the end of the first cycle of the elementary school, which in Portugal ends with the 4th grade. In the second group, students have entered 1st grade in 2015 and have been assessed in 2019. In order to understand why one is dealing with two fundamental blocks, one has to be aware that school in Portugal formally begins in 1st grade, when pupils are about 5 or 6 years of age. With the 4th grade ends what is considered the first cycle of elementary schooling, when pupils are about 10 years old. At this moment, students are assessed and most of them progress to a different school, to study at a level which corresponds roughly to what some countries classify as middle school.

This table clearly shows that students from the first group had a much superior learning experience than those from the second group. As these groups do not overlap, except for a very rare number of flunked students, this table complements Figure ⁠2⁠. It is now clear that first-group students not only progressed in the mean, but also their fraction of high performers increased while their fraction of low performers decreased. For the second group, exactly the contrary happened.

What happened is really a natural experiment. The first group of students entered school in a motivating atmosphere of rigour, curricular ambition, and assessment. They had a more demanding curriculum and better structured standards. They knew that they would have a standardized assessment in 4th grade, at the end of this cycle. The second group of students began school in a relaxed atmosphere of less rigour, no curricular ambition, and no assessment. They had a less demanding curriculum and no structured standards. They knew that they would not have a standardized assessment in 4th grade. The results are clear.

While the new policy obsessively dismissed knowledge ambition and assessment as policies that would harm the less privileged students, the reality is that, on the contrary, leniency harms those with a weaker background.

As sometimes economists say, policies should be judged by their results, and not by their stated goals. This is such an occasion.

Acknowledgements I thank the editor and an anonymous referee for very helpful suggestions. This work was partially supported by Project CEMAPRE/REM – UIDB/05069/2020 financed by FCT/MCTES through national funds.

Disclaimer

For about a decade, I have been serving on the Portuguese Mathematical Society boards (2000–2011) and have been its president (2004–2010). Later, I was appointed minister of education and science of the Republic of Portugal and served as an independent in the government for a whole term (2011–2015). The results I describe in this paper encompass these periods.

Nuno Crato is professor of mathematics and statistics at ISEG, University of Lisbon. His research focus on stochastic models, time series applications, and statistical data-based evaluation of policy measures, namely education. He has published extensively, and has also worked on education policy analysis. He was president of the Portuguese Mathematical Society (2004–2010) and director of the International Institute of Forecasters (2016–2020). From 2011 to 2015, he was the minister of education and science of Portugal. A prolific science writer, some of his books have been translated and published in various countries, including the US, UK, Italy, and Brazil, namely his Figuring It Out (Springer 2010). He has received prizes from the European Mathematical Society (2003) and the European Union (2007). He was awarded Commander (2008) and Great-Cross (2015) of Prince Henry Order and Great-Cross (2022) of Public Instruction Order of the Portuguese Republic. ncrato@iseg.ulisboa.pt

1. ¹

   Pordata, www.pordata.pt, consulted 23 July 2021.

2. ²

   Here and in the following description, I draw heavily from my previous paper on the Portuguese experience [⁠1⁠ N. Crato, Curriculum and educational reforms in Portugal: An analysis on why and how students’ knowledge and skills improved. In Audacious Education Purposes, edited by F. Reimers, Springer, Cham, DOI 10.1007/978-3-030-41882-3_8 (2020) ].

3. ³

   TIMSS (Trends in International Mathematics and Science Study) is a large-scale assessment designed to inform educational policy and practice by providing an international perspective on teaching and learning in mathematics and science. TIMSS is a project of the International Association for the Evaluation of Educational Achievement (IEA) and is directed by the TIMSS International Study Center at Boston College in collaboration with a worldwide network of organizations and representatives from the participating countries.

4. ⁴

   PISA is the Organization for Economic Co-operation and Development (OECD) Programme for International Student Assessment. Started in 2000, every three years, it now tests 15-year-old students from almost the entire world in reading, mathematics, and science. The tests are designed to assess how well students master key subjects to be prepared for real-life situations in the adult world.

5. ⁵

   “Decreto-Lei 176/2012 de 2 de agosto.”

6. ⁶

   Detailed data for PISA and other international surveys are readily accessible at the International Explorer of the National Center for Education Statistics: https://nces.ed.gov/surveys/international/ide/. An overview can be found in [⁠2⁠ N. Crato, Improving a country’s education: PISA 2018 results in 10 countries. Springer, Cham, DOI 10.1007/978-3-030-59031-4 (2021) ].

7. ⁷

   Instituto de Avaliação Educativa, IAVE, created by Decreto Lei nº 102/2013, de 25 de julho.

References

1. N. Crato, Curriculum and educational reforms in Portugal: An analysis on why and how students’ knowledge and skills improved. In Audacious Education Purposes, edited by F. Reimers, Springer, Cham, DOI 10.1007/978-3-030-41882-3_8 (2020)
2. N. Crato, Improving a country’s education: PISA 2018 results in 10 countries. Springer, Cham, DOI 10.1007/978-3-030-59031-4 (2021)
3. A. V. Kelly, The curriculum: Theory and practice. 6th ed., Sage (2009)
4. P. A. Kirschner and C. Hendrick, How learning happens: Seminal works in educational psychology and what they mean in practice. Routledge (2020)
5. J. Marôco, Portugal: The PISA effects on education. In Improving a Country’s Education: PISA 2018 Results in 10 Countries, edited by N. Crato, Springer, Cham, DOI 10.1007/978-3-030-59031-4_8 (2021)
6. A. Morris, Student standardized testing: Current practices in OECD countries and a literature review. OECD Working Paper Series No. 65 (2011)
7. T. Oates, Finnish fairy stories. Cambridge Assessment, Cambridge, UK, www.cambridgeassessment.org.uk/Images/207376-finnish-fairy-stories-tim-oates.pdf (2015)
8. A. Saarinen, Equality in cognitive learning outcomes: the roles of educational practices. Doctoral dissertation, University of Helsinki, https://helda.helsinki.fi/handle/10138/320436 (2020)
9. G. H. Sahlgren, Real Finnish lessons: The true story of an education superpower. Centre for Policy Studies, Surrey, UK, www.ifn.se/media/0irhqejv/2015-sahlgren-real-finnish-lessons.pdf (2015)