Floating gently on a waft of edu-dribble

The recent report, Australian Certificate of Education: Exploring a way forward, gives as its objective is “to set nationally consistent high standards (and) to improve the comparability of results across Australia”. That objective is laudable. Unfortunately the report is, for all its good points, fatally flawed.

In the last resort if there are to be “consistent” standards then there must be assessment systems in place enabling that consistency to exist and be seen to exist. The report “sees the development of shared achievement standards” which sounds promising, but then states “there can continue to be flexibility in how evidence is collected (for example, external assessments and/or school based assessments)”.

Queensland has for many years used school-based assessment. Because the report implies the existing Queensland system would be acceptable, an examination of that system has national significance.

In the mid 1970s an ingenious system of assessment for Years 11 and 12 was set up by the Board of Secondary School Studies. In outline it operated as follows:

Syllabi for each subject were written that provided a sufficiently detailed description of the concepts and material that was to be studied and assessed in each school.

Tests and exams were given regularly. The results of these were normally given as a mark. District Moderation Committees for each subject, later District Panels, comprising teachers from the schools in the area met and examined the exam papers and student work from all schools. The school was then advised as to whether the schools suggested results (then on a seven point scale) were acceptable.

The key issue always was comparability of results and standards between schools. Essentially the objective was to ensure that a student receiving a 6 rating, for example, would have received a 6 at any other school. Where a panel considered that a school’s ratings were inappropriate, not comparable with other school’s standards, detailed suggestions were forwarded to the school.

If, after due consideration and discussion, a school still considered the District Panel was incorrect, it could appeal to the subject State Panel. That panel re-examined the student work, and being cognisant of standards over the whole state, informed the school of its decisions.

The employees of the board were astute and combined a forward-looking approach while maintaining a highly practical grasp of reality. It was altogether a professional and committed organisation.

In my experience the overall outcomes for the students were fair. I cannot remember a single case when I thought a significant injustice had been done to a student - either too harsh or too soft. It was a good, transparent, system: it worked.

It is vital to note that:

1. the subject syllabi were clear, hence ensuring that it was evident to the panel that each school had fulfilled its syllabus obligations;
2. the various panels knew the conditions under which the various assessment instruments were done. Their provenance was certain;
3. the students knew the worth of each piece of assessment and how the various pieces of assessment would be used to reach their final result - they knew the rules of the game.

With the rise in the influence of (mainly) university-based education academics all of those three basic requirements were weakened. When the Board of Senior Secondary Schools Studies was abolished and replaced by the Queensland Studies Authority (QSA) what had been a slow decline became a Gadarene leap into disaster.

The newest syllabi are hopelessly short on detail of material and concepts to be studied. Wide variations in what, for example “physics” means from one school to another are inevitable. There is effectively no guidance on the content to be taught.

The QSA knows this and considers it to be quite acceptable, if not actually desirable. It is an exercise in futility to worry about differences between Bowen and Bunbury when we already have the potential for huge differences between Bowen and Biloela.

With the rise of “assignments” as both the central teaching approach and for the assessment of results, it is certain that nobody, either panellist or student, can know for certain whose work they are looking at. (Even universities have now realised that students cheat.) Of course they will cheat, they will get “help” from any source available - mother, father, uncles, aunts and so on - and get rewarded for their cheating.

That simple idea still eludes the QSA and their ilk. I once read a piece of “learned” work about this problem. The conclusion of much talk was that the longer the student had to do an assignment the less valid it was! Really? Fancy that.

Although not directly relevant to the assessment issue, it may be of interest to observe that as a consequence of the endless and unbroken series of assignments there is very little actual teaching taking place in many subjects. Mathematics, physics, chemistry and to a substantial extent biology seem to be the exceptions - for now.

Because the modern Queensland assessment “system” is non-numerical the “methods” to estimate final student results are vague, wordy and depend on “overall judgment”. The student has no idea whatsoever as to the relative importance of a piece of work. Hence they are playing a game, a very serious game, but they do not know what the rules of that game are. That is a scandal

Bear in mind that the abortion that is now assessment in Queensland in Years 11 and 12 follows three years of lower secondary schooling for which there has been no valid assessment now for nigh on 20 years. None whatsoever.

The basics that made the Queensland assessment system so good years ago were syllabus clarity, certainty that assessed work was that of the student alone and a transparent system to reach the final result. None of those characteristics now exists in Queensland.

The thrust of the Australian Certificate of Education concept is that there should be a nationwide certificate that would be acceptable everywhere. For any state to agree to being involved it will want to be certain that an ACE emanating from all the other states will be based on syllabi and assessment systems that are defined, reliable and valid. Current structures in Queensland fail completely to meet those requirements.

Gresham's law is that bad money drives out good. Feeble reliability of an ACE issued in just one state will drive down the validity of ACEs issued in all other states. It is unlikely that any state will agree to join a system that will degrade the standing of its own certification.

The syllabi and assessment systems - if the current mess can be called a system - has been spawned mainly by the Queensland Studies Authority. It is the organisation that has been charged with reviewing the Year 11 and 12 syllabuses. So Caesar is judging Caesar. There are a number of people within QSA that do have some connection with reality, but the organisation as a whole floats gently on a waft of edu-dribble. QSA has smashed an adequate system. It is hence psychologically and intellectually unable to make the radical changes that are needed: it neither believes nor thinks that there are monstrous problems.

In terms of student assessment Queensland has moved from excellence to lunacy, from penthouse to cellar.

How are the mighty fallen.