What is actually being tested by NAPLAN?

What exactly do the National Assessment Program - Literacy and Numeracy (NAPLAN) tests of language conventions achieve? In a recent interview, the Australian national curriculum project leader, Professor Barry McGaw, reflected that “We do have in Australia now a common assessment of literacy and numeracy, but that’s a common assessment designed in the absence of a common framework. With a national curriculum, we have the prospect of a common framework to shape the tests.”

An analysis of the 2008 and 2009 NAPLAN papers reveals contradictory messages, which are indeed evidence of the absence of a common framework.

In particular, the tests of language conventions lack coherence and order. Although all of the spelling questions, which constitute approximately 50 per cent of each test, are placed together, albeit not in a separate section, the items that test grammar and punctuation do not appear to follow any logical pattern. How are teachers expected to teach and their students expected to learn from test papers whose nature and purpose are not clear?

Are all schools encouraged to undertake a forensic analysis of the tests and to discuss strategies for teaching the syntactical points? What support material will be provided that clearly identifies the areas to be covered in future tests and the ways in which these may relate to the national curriculum?

Notably, the tests’ own use of language is problematic. In the 2008 practice tests, specific terminology was used in the questions relating to spelling, punctuation and grammar (i.e. In the first sentence, the word “a” is “a noun”, “a definite article”, “an indefinite article”).

However, in the 2008 and 2009 papers, only the questions relating to spelling and punctuation used the relevant terminology (i.e. Which of the following has the correct punctuation? OR Where should the commas be inserted?). If the questions related to grammar, no explicit terminology was used. Instead, the rubrics simply asked “Which word(s) correctly complete the sentence?”

What was the rationale for this shift?

The questions in the 2009 tests are clearer than those in the 2008 version, at least with regard to which points of grammar or punctuation are being tested. However, there are items in both the 2008 and 2009 papers that are extremely unclear in that they appear to be testing multiple points simultaneously or are written in a way that relies on native speaker intuition rather than a sound grasp of how the English language works. In addition, the range of questions is extremely limited.

Our work with employers, schools and tertiary institutions has enabled us to compile a list of the errors that students are most likely to make, and which frequently require remedial study. At the top of this list are the run-on sentence and the sentence fragment. In the 2008 and 2009 NAPLAN papers, the former is difficult to find, and the latter is not tested at all.

The practice questions for 2008 gave an indication of the format of the tests, but provided little guidance of the specific grammatical concepts to be examined. In one practice question, which was the same for Years 3, 5, 7 and 9, the answer depended on native speaker intuition. The question was “Do you have ______ pet?” Possible answers were “a”, “if”, “he” and “she”. What is actually being tested in such a question - the indefinite article, a subordinating conjunction, or a personal pronoun? It would appear that the indefinite article is being tested, but there is only one logical choice. This is a process of elimination that would not require students to distinguish between the correct and incorrect indefinite article.

Similarly, in the Year 5, 2009 paper, item 31 asked students to complete the sentence “Jo likes to listen to music ______ she is cleaning her room”. Possible answers were “even”, “after”, “while” and “during”. “Even” could take the role of an adjective, verb or adverb. The option “after” acts either as a preposition or an adverb, “while” is an adverb, and “during” is a preposition. How does this test a student’s understanding of grammar?

In the Year 3, 2009 paper, item 43 asked the students to complete the sentence “The boy put on his shoes ______ he tied his laces”. Possible answers were “next” (an adverb), “and so” (two coordinating conjunctions), “because” (a subordinating conjunction) and “and then” (a conjunction followed by an adverb). In such questions, the advantage is likely to lie with the student who has had the greatest exposure to such everyday phrases.

In the Year 7, 2008 paper, item 49 asked which sentence had the correct punctuation. The choices were:

• The man, got out of the car before it burst into flames and ran away.
• The man got out of the car before, it burst into flames and ran away.
• The man got out of the car, before it burst into flames, and ran away.
• The man, got out of the car, before it burst into flames, and ran away.

None of these sentences is correct. The first sentence is incorrect because the subject of a sentence cannot be separated from the verb by a comma. The second sentence is incorrect because “before” is a subordinating conjunction that begins its own clause (i.e. before it burst into flames), so the comma after the word “before” is wrong. The last sentence is incorrect because the predicate (i.e. got out of the car) in the independent clause (i.e. the man got out of the car) is corralled with commas, which means that the phrase could be removed from the sentence, leaving the subject without a verb.

It will be argued that the third sentence is correct. This cannot be the case because the reason the man got out of the car is predicated on the adverbial clause of time (i.e. before it burst into flames). Therefore, if the adverbial clause is corralled with commas, it suggests that the clause is not important for the meaning of the sentence. On the contrary, it is! “The man got out of the car” is an independent clause. “Before it burst into flames” is a dependent clause (dependent on the independent clause). “The man (implied) ran away” is an independent clause. The correct answer is “The man got out of the car before it burst into flames, and ran away”. This is a good example of a compound- complex sentence (i.e. an independent clause plus a dependent clause separated with a comma from the second independent clause).

What is the question testing? How would teachers approach it?

In the Year 9, 2009 paper, item 31 tested a non-restrictive clause, as follows:

Q. Where do the two missing commas (,) go? A. The computers_which were donated_ by local businesses_are located_in the library.

This item gives the answer away by advising that the commas are missing and nominating four possible places where they could be inserted. How does this demonstrate the student’s understanding of this point of grammar? How would a teacher teach it?

The sceptic will argue that language is dynamic and that those who strongly encourage correct usage are pedants who place more emphasis on the mechanics than on the message.

Our response to that argument is threefold. First, a mastery of the mechanics of English means the freedom to concentrate on meaning and message. As one teacher ruefully commented to us recently, “Every day we ask the students to produce pieces of written work - narratives, expositions and so on, but we don’t give them the skills to actually do that, and so many of them just give up”.

Second, by virtue of their contract, teachers, like doctors and other professionals, are accountable.

Third, adults who choose to ignore correct usage must be prepared to defend their choices. Those who insist on expressions such as “Between you and I ...”, for example, must explain how this sets an appropriate example to their students.

According to the NAPLAN authorities, “the tests provide valuable diagnostic information about each student, and support the ability to focus teaching on areas of need”. Given that the next round of NAPLAN tests will be conducted in May of this year, we believe it is fair to question whether the tests are intended as learning tools or as measuring instruments only.