Menu

Education Today Logo


newsletter

Education Today Cover Browse Issue

Automarking 'methodologically flawed and massively incomplete'

News Image

The NSW Teachers Federation claims that ACARA is rushing through with plans to have robots mark next year’s NAPLAN tests despite their justifications being discredited by world-leading research. A report by Dr Les Perelman from Massachusetts Institute of Technology decribes the plan as 'methodologically flawed and massively incomplete'.

Acting President of the Federation Gary Zadkovich said: “Parents, teachers and principals have not been consulted about these radical plans and students have not been prepared for such a dramatic shift. NAPLAN has been a high-stakes and controversial test and this latest development will further bewilder youngsters who are subject to such testing.

”The Australian Curriculum and Reporting Authority announced in a newsletter that they intend to use computer-robots to mark the 2018 NAPLAN tests. In justification, they cited their own anonymous 2015 report that they claim guarantees that robots can mark children’s work just as well as teachers.

"However, the ACARA paper has been carefully examined by one of the world leaders in Assessment and Writing from the Massachusetts Institute of Technology, Dr Les Perelman, who describes the ACARA report as, “So methodologically flawed and so massively incomplete that it cannot justify any use of AES [automated essay scoring] in scoring the NAPLAN essays.”

Dr Perelman's report, commissioned by the The Federation, claims that robots can only detect low grade attributes of writing; cannot detect the most important elements of a text; the ACARA report uses evidence and data selectively and poorly and ignores any authority critical of robot marking; robots and humans should not be used in conjunction in marking; even robots as grammar checkers are seriously flawed; overseas evidence shows robot marking discriminated against some social groups; and introducing robots will encourage regressive teaching practices.

Responding to release of Dr Perelman's report, Dr Stanley Rabinowitz, general manager of assessment and reporting at ACARA, said all NAPLAN writing tasks completed online next year would be marked by a person as well as an automated scoring system.

He said ACARA had done further research since the 2015 report, including work based on Dr Perelman's research, which would be released next month. He defended ACARA's report and said Dr Perelman and the Teachers Federation "are known critics of automated marking systems and the report findings should be viewed with this in mind."


20 Mar 2019 | International
Nice kids win in the end News Image

The lure of the bad kid is strong in the early years of high school but their charm fades and by Year 12 the nice kids win out with more friends of the opposite sex, general popularity and better wellbeing. Read More

20 Mar 2019 | National
Half a million dollars for more than 500 Aussie schools to grow enviro champions News Image

Some 538 Australian schools are set to receive up to $1000 each, around 500k in total, for environmental projects as the recipients of the first round of the Woolworths Junior Landcare Grants program. Read More

19 Mar 2019 | Jakarta
New British School Jakarta Principal introduces IB for Middle Years News Image

With over 50 nationalities represented British School Jakarta has a very international student group and a global perspective, the IB program is a good match.
Read More

19 Mar 2019
2019 SACE Art Show reveals plenty of new talent News Image

The 2019 SACE Art Show four-week exhibition at Light Square Gallery features paintings and drawings, sculptures, jewellery, costumes and multimedia works created by 129 Year 12 students. Read More

19 Mar 2019 | NSW
Vivid Sydney’s Vivid School introducing creative industry futures News Image

Vivid Sydney has launched its inaugural Vivid School giving High School students in Years 9-12 the opportunity to meet artists behind the event’s light installations and inform them about careers in creative industries. Read More