Improvements in maths and numerical science demand transparency

In the decade since OLO posted my Wadderloader, Maths and Science in Australia (OLO 25/01/05) there has been some slight improvement in Maths and numerical Science standards generally, but there have been exceptions to that trend.

However there are now signs that matters may improve significantly. That is pleasing and gives hope; but the reactionary Education Establishment are still very strong.

In my dozen or so postings between Wadderloader and Educational sexism in Queensland (26/04/2013), and also in an e-petition to Parliament, were a number of common themes:

• Standards of maths and numerical science in Australia are low. The Trends in International Maths and Science (TIMSS) data for Queensland are very poor indeed. The performance of the upper third of any cohort is appalling, very few reaching 'Advanced' on TIMSS. The condition of Algebra 'the language of higher mathematics' is even worse than the rest, scandalously bad.
• The blame for that must lie within the education establishment which consists of (a) feeble and incompetent education faculties, (b)Trendy groups within the government Education Departments, (c) teacher unions who utterly oppose verifiable assessment of student outcomes and, biggest and worst of all (d) various Boards of Study who are responsible for all syllabi and all assessment systems.
• The education establishment is intellectually and psychologically incapable of making improvements. Only parliaments can produce the changes that are so badly needed.
• My conviction that any syllabus, and its associated assessment systems, must meet the triple criteria Defined, Reliable and Valid.

A major difficulty faced by the commonwealth parliament when dealing with education is the fact that education, in particular assessment, is a state issue. Neither the house nor the senate can influence what happens in, for example, Queensland. A senate inquiry into the education of boys concluded that:

Assessment procedures for maths and sciences must, as a first requirement, provide information about students' knowledge, skills and achievement in the subject, and not be a de facto examination of students' English comprehension.

The Queensland Studies Authority ignored that recommendation - as they legally could - and with overweening arrogance made things worse by an even further emphasis on English even in the STEM subjects. The failure to respond to the senate's remarks have meant that those students who are relatively weaker in English than Maths/Physics/Chemistry are punished in the STEM subjects for their weaker English.

The states' total control of assessments killed Howard's push for an Australian Certificate. However one thing that emanated from the commonwealth was NAPLAN. That has been a game changer because for the first time there is some data. That brought home to people, notably state parliamentarians/ministers just how bad things were in the basics - Maths and English. The results were described by then Premier Anna Bligh as 'a wakeup call'. Various suggestions (from ACER) were proposed but little progress has been made. To get anywhere what was needed was an inquiry by the Queensland parliament into education in the state. Even QSA and the education establishment generally would have to listen to the outcomes of any such inquiry.

The current Queensland government set up a parliamentary inquiry into Year 11/12 Assessment in Maths, Physics and Chemistry. As with all inquiries submissions were called. The response was excellent, there being nearly 300 submissions. They could be divided into (a) a minority who thought the existing system was satisfactory and (b) the overwhelming majority who did not. They considered the system to be unfair, unjust and that cheating was rampant. They wanted radical change.

The minority came almost entirely from education academics and their allies plus some teachers who were seemingly on the promotion trail. As they were/are the people who erected the system their behaviour is just patting themselves on the back!

The majority came from everywhere else: parents, 'coal face' teachers, Maths/Physics academics and the public generally. The backgrounds of these submitters were across the board. A most noticeable feature is that they did not/do not think that the assessment system is bad; they know it. They have endured the mess for years and seen the injustice. The submissions are heartfelt and frequently angry and are knowledgeable because they are the fruit of experience. They are desperate for change. One thing that they do not do is talk eduspeak gobbledegook. They talk normal English.

And so do parliamentarians. They clearly understood the situation and must have been surprised, shocked. The report from the inquiry contained sixteen recommendations. (The inquiry, the submissions and the recommendations are all available online). Taken together they represent a revolution; a rough summary is: there will be an external exam to count 50% of overall achievement. That will ensure that 'content and knowledge' is emphasised and provide commonality and validity. The external exam results are to be used to scale any internal assessment results. (Totally crucial). 'Basics, content and procedural knowledge' should be the primary determinant of results. In all Maths/Physics and Chemistry there is to be numerical marking. Extended experimental Investigations and other 'assignment' type activities are to be 'completed in class time under supervision.' Time to be specified. They can count between 12.5% and 25% of overall result. The Inquiry emphasised that 'clarity' is needed at all stages.

The Inquiry recommendations represent a total defeat for the QSA and the Education Establishment generally; their ideas and methods have been totally repudiated.

All of my triple criteria - defined, reliable and valid can be met and satisfied by a system using parliament's recommendations if they are applied in letter and spirit. Therein is a massive problem; who or what will operate any new system? It will be the Queensland Curriculum and Assessment Authority QCAA, the child of the QSA. I see nothing in QCAA to encourage optimism. I anticipate that they will obfuscate, play with the meaning of words and act in the secretive manner that they have brought to a fine art.

The 2013-14 Annual Report of the Education and Innovation committee stated inter alia

:…the committee made 16 recommendations covering the number, format and proportion of marks for assessment tasks; the introduction of external examinations as part of the assessment process; expanding syllabus documentation to be more clear and explicit…. On 6 January 2014, the government tabled its response…. All the committee's recommendations were supported by government either directly or in principle, with several recommendations referred to the review of senior assessment and tertiary entrance processes in Queensland as they were directly in the scope of that broader review.

The review referred to in that summary was the Queensland Review of Senior Assessment and Tertiary Entrance produced by Matters and Masters of ACER. Whilst there is some recognition of the need for change – they advocate an external exam and the use of numbers but the general feel is of a document by the education establishment based on views of the establishment and for the greater glory and power of the establishment.

The recommendation that received most publicity was that the Overall Position, the OP, should be discontinued. It is important to note that the OP calculation does not and never has affected or been affected by syllabi or subject assessment. Hence the dropping of the OP has no implications for parliament's recommendations. The OP is irrelevant in that context.

The intention is that tertiary admission procedures should be done by those institutions; that the methods should be made as clear as possible and that there should be 'Transparency in tertiary selection.' Recommendation 11 makes that very clear and uses the word should repeatedly –'institutions should make as transparent…'. I can go along with the idea that tertiary institutions should sort out and explicitly state what their selection system will be. However I find the demand by the establishment for transparency from others to be totally hypocritical.

To explain in simple terms: I have been tutoring students in maths/physics for many years now. During that time I have never seen a test or exam paper at all. Does anybody get to see old exam papers? I do not know what a 'modern' exam paper looks like. The students tell me that each question has a little matrix next to it that contains some letter and small numbers. I have no idea what that all means, I have no idea how the 'results' are totalled. Students tell me that some questions are A, some B etc Transparent? Has anyone – a parent for example - ever seen any marked work? Has anyone any idea how the extended pieces of work are marked? Transparent? I presume by the way that the reason the schools keep the old exam papers hidden is to save the bother of writing another one later.

The Parliamentary Inquiry Recommendation 4 states that '…the external exam for mathematics, physics and chemistry be used to scale school based assessment.' As mentioned earlier such scaling is essential. However the ACER document in its recommendation 5 states categorically that 'teacher assessments should not be statistically scaled against the external examination.' (My emphases). This difference is fundamental; there must be scaling.

ACER also state that there will be just one mark (result) for each subject, that mark is to be 'the sum' of the .. (various) activities (with external counting 50% and the internal also 50%). To simply add such data is crude, simplistic and unacceptable. Furthermore ACER demands that tertiary institutions must make clear how they are to sort entry to the courses; but ACER deliberately hides important information. It is very common for a student to score much more highly in assignments than on supervised exams. A faculty may decide that a result on the external exam is of greater significance than the 'internal' result. Surely that is their business not QCAA or the education establishment generally. There are a number of possibilities: (a)three separate results - external exam, internal assessment and some averaged result. (b) one result which is obtained after proper and legitimate scaling, (c) two results external and internal. But definitely not unscaled adding.

ACER Recommendation 8 aims to 'assure validity and reliability of school assessments.' The proposed system 'includes three elements, 'Endorsement', 'Confirmation' and 'Ratification'. Such a system might work using a 'guild of Assessment Supervisors', However a central aim of the parliamentary inquiry will be for schools to emphasise 'basic content and procedural knowledge….(which are to be) the primary determinant of results in Maths Physics and Chemistry'. To that end the eternal extended items are to be shortened and done in school. The remainder of the time should be spent emphasising the skills, understandings and knowledge etc. and then application of those skills. Any time used meeting the ACER ideas of endorsement, confirmation and ratification will cut into the teaching and learning time. ACER needs to simplify this material.

The Parliamentary Inquiry recommends that basic content and procedural knowledge should be the primary determinant of results. That is anathema to QSA/QCAA. To them content, knowledge and skills is the least important aspect of maths. What they seem to want is chatter in English. It is vanishingly unlikely that the QCAA will 'interpret' Parliaments clear intentions; certainly not in spirit and, by redefining words, not really in letter either. Student outcomes/results will be at odds with what Parliament wants. Note that as QCAA, like QSA, determine everything that happens in all schools so they will determine the external exam. It is probable that any external exam they set will give low significance to procedural knowledge and skills.

ACER want tertiary institutions to be 'transparent' re entry procedures. I accept that, but such transparency must permeate everything in QCAA assessments. The external exam must be publicly available within a couple of months of it being set and taken. Marking schemes and method(s) to reach final result must also be available. All assessments set within schools must be available to parents and others concerned. Parents must be allowed, with ease, to see marked work of their child, whether that be a test or some other instrument. Students and parents should be able to keep copies of test papers.

So I add to my usual criteria defined, reliable and valid an extra essential: transparency.

The Minister of Education has decisions to make. I submit that in addition to the powerful democratically achieved outcome of the Parliamentary Inquiry there are other indications that things are moving in the right direction. Clearly the Wiltshire/Donnelly paper emphasised the total importance of Maths and English (the basic tools of course). They kept NAPLAN. The Senate Inquiry into NAPLAN left it unchanged.

I am convinced that with the application both in letter and spirit of the Parliamentary Inquiry's recommendations, taken in conjunction with improved work and learning up to Year 10 exit, there will be major improvements in Maths and the numerical Sciences. NAPLAN could be cheaply built on to provide data on 'value added' –a fair system.

We must assume that QCAA will drag the chain continuously. But with transparency it will be evident what is happening and rectification will be possible; probably by Regulation.

I thank the Parliamentary Inquiry members. They have done a fine job and in these cynical days shown the democracy really does work.