PISA and TIMSS are like apples and oranges

How successful are Australian students? Based on the results of the last two Programme for International Student Assessment (PISA) tests, Premier Bob Carr argues that our students are the best in the world.

The results of the first PISA test, sponsored by the OECD to test 15-year-old students in areas such as literacy, were released late 2001 and Australian students were ranked among the best performers.

The second round of the PISA test results were released early December last year. Out of the 41 countries in reading literacy Australia was ranked number 2; in scientific literacy we were ranked number 4; and in mathematics literacy Australian students came 5th.

Based on PISA, it appears that Premier Carr has got it right. Unfortunately, such is not the case. A week after the PISA results were released, the results of a second international test became public. Unlike the PISA test, the results of the Trends in Mathematics and Science Study (TIMSS), undertaken by the International Association for Evaluation of Educational Achievement (IEA), shows a much bleaker story. In fact, rather than ranking with the best, Australian students performed poorly, worse still, we were overtaken by a number of countries that we had outperformed in previous TIMSS tests.

Australian year 4 maths students were beaten by 15 other countries and 13 countries outperformed our year 8 maths students. Even worse, on comparing the results of the 2004 TIMSS tests with earlier TIMSS tests, Australia’s results remained static while students’ results in the USA, England and New Zealand dramatically improved.

Which test carries more weight? The first thing to note is that PISA adopts many of the progressive education fads such as fuzzy maths and whole language that have bedevilled Australian education. TIMSS, on the other hand, measures more traditional classroom content on the assumption that students need to master essential learning such as demonstrating an understanding of fractions and decimals and the relationship between them.

Similar to Australia’s progressive approach, PISA argues that learning should not be restricted to right and wrong answers where students are made to learn a set body of knowledge. Instead, education must deal with clichés like real-life problems and life-long learning.

As a result, as noted in the OECD’s Executive Summary of the 2000 PISA results, the traditional view of literacy is redefined, in the “edubabble” much loved by those committed to education fads, as:

Knowledge and skills that reflect the current changes in curricula, moving beyond the school-based approach towards the use of knowledge in everyday tasks and challenges.

Based on PISA, it appears that education in Australia is on the right track and that standards are high. A closer view of the PISA test shows that the opposite is the case as the test, when compared to TIMSS, is substandard and flawed.

Firstly, even though the reading test is about literacy, students are not penalised for faulty spelling, grammar and punctuation. If Australian students were corrected the majority would have failed.

To quote from the ACER’s Australian report: “Errors in spelling and grammar were not penalised in PISA … It was the exception rather than the rule in Australia to find a student response that was written in well-constructed sentences, with no spelling or grammatical error.”