Putting NAPLAN literacy testing to the test

The 2010 NAPLAN tests of language conventions reveal inherent design flaws. As was the case in the 2008 and 2009 papers (see our On Line Opinion article of March 2010), many items fail to show the relationship of grammar to the conventions of punctuation. This fundamental weakness makes the tests considerably less effective and credible as measuring instruments and as teaching tools.

The major weakness of the tests is that they lack order and coherence. As one Head of English has noted, “They are a random collection of questions on grammar, punctuation, meaning, idiom and instinctive understanding.”

The Year 3 test contained 25 spelling words. It addressed the language conventions in this order: subordinating conjunction; tense; reflexive pronoun; pronoun; capitalisation; punctuation; sentence fragment; comparative v superlative; countable v uncountable nouns; capitalisation; commas; compound sentence; word order; demonstrative pronoun; commas; capitalisation; apostrophe; meaning; modal with a past participle; adjective; apostrophe.

The Year 5 test contained 25 spelling words. It tests the following grammar and punctuation items in this order: reflexive pronoun, meaning, comma, direct speech, spoken v written usage, comparative v superlative, capitalisation, commas, subject-verb agreement and spelling, use of brackets, defining phrase (tense), punctuation, brackets, apostrophe, modal verb, subjunctive mood, compound sentence (with coordinating conjunction), apostrophe, comparative and superlative, subject-verb agreement (with neither-nor), knowledge of conjunctions, main clause (plus independent clause creating complex sentence).

The range of questions is limited. The tests do not address the many errors that characterise students’ written work throughout their schooling, and which are most detrimental to fluency, such as the run-on sentence and the sentence fragment. Some items appear to be testing multiple points simultaneously. Other questions are written in ways that rely on native speaker intuition, or common sense and logic, rather than a solid grasp of how the English language works. Many items test usage rather than conventions. The type of language used to frame the questions is inconsistent, sometimes referring to a part of speech by its appropriate name, and at other times asking simply for the correct “word/s”. The random nature of the tests is reflected in the inclusion of a question on the subjunctive in Year 5.

The use of grammatical terminology is also problematic. With regard to punctuation, the only specific terms are commas and apostrophes. The only parts of speech that are mentioned are adjectives and conjunctions. In every other item, students are simply asked to choose “which word” completes the sentence. If students are expected to learn and to use the metalanguage in other subjects such as mathematics, music and geography, why is this not the case in English?

The instructions are frequently confusing. In item 37 of the Year 5 paper, students are asked “Which words and punctuation correctly complete this sentence?” The sentence reads: “I collect model cars and ________________ do you collect, Jasper?” Felicity asked.

Possible answers are:

a) shells what
b) shells. What
c) shells, what
d) shells? What

The answer is b). However, this creates two sentences, not one, making the test question inaccurate.

A further inconsistency can be found in item 42 in the Year 5 test. The question requires students to complete the statement “I have lost my bag but my keys are in my pocket _______________.” Possible answers are:

a) luckily I can still drive home
b) it is lucky I can still drive home
c) so luckily I can still drive home
d) because it is lucky I can still drive home