Some PISA commentary leans away from truth

As expected, Australia’s recently released results in the Programme for International Student Assessment (PISA) tests are causing a good deal of media interest and response.  Unfortunately, though, not all of what is being written meets the ‘truth test’.

Take the statement from the Grattan Institute’s Ben Jensen that it is wrong to argue, as I have, that the PISA results are proof that “a particular school sector is superior to others or that more money should go to a specific sector” (‘Money wars hold back on schools’, The Australian, December 11).

Jensen is wrong.  If superior performance is defined in terms of PISA test results then the overwhelming evidence is that the Catholic and independent school sectors outperform the government sector.

The ACER’s analysis of the 2012 PISA test results concludes that the Catholic and independent school sectors outperform the government school sector.  And contrary to what non-government school critics like the Australian Education Union like to argue schools in the government sector, except for selective secondary schools, are generally ranked third as measured by other academic indicators like Year 12 results, completion rates and tertiary entry.

It is true that the ACER analysis of the latest PISA results  argues that there is little, if any, difference between the three school sectors on the adjusted figures when the authors conclude, “When school-level socioeconomic background is also taken into account, the differences in performance across school sectors are not significant”.

The problem with the ACER’s conclusion, though, is that there is a good deal of research proving the opposite is the case.  A second ACER paper titled  LSAY Briefing Number 9, instead of questioning the impact of school sectors on results, notes that some school sectors achieve stronger results compared to others when it states, “Multivariate analysis indicates that the differences between school sectors remain significant after making a statistical allowance for the influence of other factors”.

In relation to tertiary entry results the paper goes on to observe, “After making allowances for differences in the social and academic composition of schools, there was an average difference of six percentage points between government and non-government schools results”.

It goes without saying, especially for those who have taught Year 12, that a difference of six points is significant as it can decide whether a student gains entry to his or her first preference for tertiary study and the career of his or her choice.

Gary Marks is an Australian researcher, currently at the University of Melbourne, who has written extensively over the last 10 to 15 years analysing the performance of different school sectors based on international test results, tertiary entry and Year 12 results plus completion rates and the ability of schools to help underperforming students perform better than otherwise might be expected.

A 2001 paper titled LSAY Research Report Number 22, co-authored by Marks and in relation to the ability of schools to improve student performance as they move from Year 9 to Year 12, concludes that “achievement growth was greater among Catholic and independent school students than among government school students”.

When analysing the impact of school sector on tertiary entry scores the paper also states, “higher proportions of students attending Catholic and independent schools gain higher ENTER scores compared to students at government schools”.

In relation to how successful school sectors are helping students do well at Year 12 and gain tertiary entry Marks, in a 2004 paper titled School sector differences in tertiary entrance: improving the educational outcomes of government school students, is unequivocal when he writes, “So a variety of studies using different sources of data all show substantial sector differences in university entrance”.