NAPLAN fails the test

The controversy over the testing of Australian students’ literacy and numeracy skills has included little or no focus on the most critical issue of all - the quality of the tests themselves.

The National Assessment Program - Literacy and Numeracy (NAPLAN) tests ostensibly provide a “snapshot” of student progress, which is intended to inform teaching practices and is being used to evaluate the performance of Australian schools. However, at least one instrument used to collect this data - the test of language conventions - is demonstrably inadequate for these two purposes. Further, Australian students are sitting the tests in an educational context characterised by the longstanding absence of a nationally agreed approach to teaching grammar and punctuation and a low level of confidence among teachers, including English teachers, in regard to their competence in this aspect of the curriculum.

An overarching concern must also be that professional learning in this area has not been made a mandatory component in the strategic plans of all Australian schools. Those who object to the use of the NAPLAN data under the current circumstances are right, therefore, but for reasons other than the ones that have made headlines so far.

According to the Australian Curriculum, Assessment and Reporting Authority (ACARA), the NAPLAN testing regime “enables teachers to focus their teaching and learning on learning programs in the future” (Education Review, May 2010). However, an analysis of the 2008-2010 tests reveals that they do not provide an appropriate platform on which teachers and their students can build a sophisticated understanding of English grammar and punctuation.

The major weakness of the tests of language conventions is that they lack order and coherence. In the opinion of the Head of English at one high-achieving secondary school, “They are a random collection of questions on grammar, punctuation, meaning, idiom and instinctive understanding”.

Many items are unclear as to their purpose, or test meaning rather than grammar, punctuation or spelling. Some items appear to be testing multiple points simultaneously. The range of questions is limited, in that the tests do not adequately address the significant language errors that challenge students in the production of written work throughout their schooling, and which are most detrimental to fluency, such as the run-on sentence and the sentence fragment. Some questions are written in ways that rely on native speaker intuition, or common sense and logic, rather than a solid grasp of how the English language works.

The age-appropriateness of some material is also questionable, given the inclusion of a question on the subjunctive mood in the 2010 Year 5 paper, for example. Notably, although these tests are about language conventions, the type of language used to frame the questions is inconsistent, sometimes referring to a part of speech by its appropriate name, and at other times asking simply for the correct “word/s”.

One fundamental obstacle to quality is that the NAPLAN tests are being written in a kind of pedagogical limbo. The principal of Melbourne High School has commented that, “NAPLAN bears no direct relation to the curriculum program of any state or territory”. Professor Barry McGaw, the ACARA Chair, has stated that “We do have in Australia now a common assessment of literacy and numeracy, but that is a common assessment designed in the absence of a common framework - with a national curriculum we have the prospect of a common framework to shape the tests”.

It is also arguable that the NAPLAN data have revealed little that is not obvious to those who work in Australian schools. For example, the most recent national report on the tests concluded that:

Although not shown in comparative figures, and despite considerable improvements in the comprehensiveness of the parental occupation and education data, the 2009 findings with respect to relationships between NAPLAN results and parental education and occupation are remarkably consistent with those from 2008. For example, in 2008 a higher proportion of students whose parents have a degree was likely to be at or above the national minimum standard than the proportion of students whose parents have a Year 11 equivalent or below (p. 312) … the strongest relationships are for Writing and Grammar and Punctuation and the weakest are for Numeracy (2009, p342).

In other words, children who live in households where the adults have more formal education do better in language-oriented tests. Those adults are likely to have a better command of Standard English, to value formal education more highly, and to be more able to support their children’s literacy skills. The children are also more likely to attend schools in which they mix with students from like backgrounds. The NAPLAN tests in their current form will not greatly challenge these students.

The ACARA authorities have claimed that “Every question is reviewed by every state and territory, as well as experts in Indigenous education, education for students from language backgrounds other than English, students who have a visual impairment and other experts in teaching and learning for students with disabilities” (Education Review, May 2010).

Such an extensive consultation process certainly meets the requirement to be inclusive. The design and content of the tests must meet the same high standards. By producing a stand-alone test of language conventions, Australian education authorities are placing a very high value on students’ acquisition of skills in spelling, grammar and punctuation. To be pedagogically valid, the tests must be absolutely clear in their structure and purpose. They should complement the expectations of the NAPLAN tests of reading and writing, and provide the best possible guidance for teachers.

If one objective of the NAPLAN testing is to identify and to help students and schools in need of support, how will the tests be useful? The answer is that in their present form, they cannot be.

In every profession there are those who resist accountability. Some in the education sector are uncomfortable with the expectations placed on them by the annual NAPLAN tests, the My School website, and the imminent implementation of a national curriculum. However, the new Australian curriculum places a renewed focus on language as a foundation skill, and all teachers in all subjects, across all stages of learning, will be expected to support the development of their students’ skills in this area. The quality of the NAPLAN tests is a critical aspect of making a difference to the way in which Australian students learn and Australian teachers teach.