Some PISA commentary leans away from truth

As expected, Australia’s recently released results in the Programme for International Student Assessment (PISA) tests are causing a good deal of media interest and response.  Unfortunately, though, not all of what is being written meets the ‘truth test’.

Take the statement from the Grattan Institute’s Ben Jensen that it is wrong to argue, as I have, that the PISA results are proof that “a particular school sector is superior to others or that more money should go to a specific sector” (‘Money wars hold back on schools’, The Australian, December 11).

Jensen is wrong.  If superior performance is defined in terms of PISA test results then the overwhelming evidence is that the Catholic and independent school sectors outperform the government sector.

The ACER’s analysis of the 2012 PISA test results concludes that the Catholic and independent school sectors outperform the government school sector.  And contrary to what non-government school critics like the Australian Education Union like to argue schools in the government sector, except for selective secondary schools, are generally ranked third as measured by other academic indicators like Year 12 results, completion rates and tertiary entry.

It is true that the ACER analysis of the latest PISA results  argues that there is little, if any, difference between the three school sectors on the adjusted figures when the authors conclude, “When school-level socioeconomic background is also taken into account, the differences in performance across school sectors are not significant”.

The problem with the ACER’s conclusion, though, is that there is a good deal of research proving the opposite is the case.  A second ACER paper titled  LSAY Briefing Number 9, instead of questioning the impact of school sectors on results, notes that some school sectors achieve stronger results compared to others when it states, “Multivariate analysis indicates that the differences between school sectors remain significant after making a statistical allowance for the influence of other factors”.

In relation to tertiary entry results the paper goes on to observe, “After making allowances for differences in the social and academic composition of schools, there was an average difference of six percentage points between government and non-government schools results”.

It goes without saying, especially for those who have taught Year 12, that a difference of six points is significant as it can decide whether a student gains entry to his or her first preference for tertiary study and the career of his or her choice.

Gary Marks is an Australian researcher, currently at the University of Melbourne, who has written extensively over the last 10 to 15 years analysing the performance of different school sectors based on international test results, tertiary entry and Year 12 results plus completion rates and the ability of schools to help underperforming students perform better than otherwise might be expected.

A 2001 paper titled LSAY Research Report Number 22, co-authored by Marks and in relation to the ability of schools to improve student performance as they move from Year 9 to Year 12, concludes that “achievement growth was greater among Catholic and independent school students than among government school students”.

When analysing the impact of school sector on tertiary entry scores the paper also states, “higher proportions of students attending Catholic and independent schools gain higher ENTER scores compared to students at government schools”.

In relation to how successful school sectors are helping students do well at Year 12 and gain tertiary entry Marks, in a 2004 paper titled School sector differences in tertiary entrance: improving the educational outcomes of government school students, is unequivocal when he writes, “So a variety of studies using different sources of data all show substantial sector differences in university entrance”.

In a 2009 paper titled Accounting for school-sector differences in university entrance performance Marks revisits the issue of school sector impact on tertiary entrance and he concludes that schools in the non-government sector “promote a more academic environment that lifts student performance”.

And the stronger performance of Catholic and independent school sectors is not restricted to Year 12 results and tertiary entry.  A third LSAY paper published in 2013, School Completion: what we learn from different measures of family background concludes that, “Respondents from Catholic or independent schools are more likely to complete Year 12 than those in government schools”.

Clearly, Ben Jenson is wrong to argue there is no evidence to support the claim that Catholic and independent school sectors outperform the government sector.  Even worse, the fact that Catholic and independent schools outperform government schools even after adjusting for students’ socioeconomic status (SES) belies the argument that such schools do well only because they enrol already privileged students from wealthy backgrounds.

Based on his research Gary Marks concludes that, “Differences in student performance between and within schools cannot be accounted for by socio-economic background” and “Socio-economic background does not substantially account for the relationship between educational differentiation and student achievement”.

In answer to those arguing that Australian education has an equity problem it should also be noted that based on the 2012 PISA results we are categorised as ‘high-quality’ and ‘high equity’ and as noted in the 2013 LSAY paper previously referred to, “The impacts of family background (parental occupations and education) and type of school have become less marked since the late 1980s”.