There’s a doomsday scenario where uni students hand in work generated by artificial intelligence (AI), which is then marked by AI and university assessment no longer involves any humans at all.
There’s also an alternative utopia where teachers use AI to escape the burden and time pressures of endless piles of marking. In this bright new “AI-powered” world, bias against student differences is avoided and staff use the extra free time to engage in high-quality teaching and research.
So what’s happening with AI marking in Australian universities?
AI marking isn’t new
Most people don’t realise AI marking has been commercially developed since at least 1998. The educational publisher Pearson reports its specialised trained systems have scored 39 million responses, including essays, short answers and equations.
In previous decades, the tech was an expensive specialist product that may not have been easy for unis to access. But this has changed as AI has become more readily available.
How does AI marking work?
AI models, such as ChatGPT, predict what mark should be awarded by comparing the textual patterns of a student’s work to other long-standing texts. If the AI has been trained on a lot of texts similar to the assessment task, its predictions are more likely to be closer to a human’s response.
With an openly available AI, such as ChatGPT or Claude, a teacher could upload an essay and a rubric and ask AI to grade and provide feedback on the essay.
How are unis approaching it?
Universities are generally not advertising their approaches to AI marking. But policies are guided by strong privacy regulations. Some universities state how this intersects with AI.
For example, Deakin University prohibits teachers from uploading university data and information into openly available AIs. This includes students’ assignments, as doing so might expose their private data to offshore data storage, which is beyond Australia’s privacy regulations.
In addition, universities typically emphasise humans must take responsibility for their judgements when marking assessments assisted by a university-approved AI. For example, the University of Melbourne lets teachers use approved AIs “to support their evaluation of students’ work”, but prohibits them from relying on AI alone to determine marks or grades.
Assessment almost always involves text or video feedback, alongside a mark or grade. Universities typically appear more accepting of using AI to help provide students with feedback comments, although human oversight, again, is required.
Is AI reliable for marking?
Research suggests the answer is no, at least not at the moment.
A University of Melbourne and Cardiff University study, published in August, looked at whether later versions of AI software ChatGPT could effectively imitate human markers’ scores when marking 50 undergraduate bioscience essays.
Researchers found there was some alignment in the overall essay marks, but concluded ChatGPT could not be relied upon to assign and explain marks. It was particularly poor with respect to individual marking criterion, such as the appropriateness of essay content and sufficient supporting evidence. Researchers also found ChatGPT, on average, tended to grade students’ essays higher than humans did. This reflects how AI works – it doesn’t evaluate; it predicts.
Relationships are impacted
Another obvious risk is that AI marking can negatively impact student-teacher relationships.
My research, investigating students’ and teachers’ perspectives on feedback and AI, suggests AI may create distance between students and teachers. For instance, students can now turn to AI for feedback first, when they might otherwise have approached a tutor.
Other research describes how students benefit from the personal recognition that comes with comments provided by a human teacher. Feedback shows the student their learning matters.
Defaulting to AI marking could also mean teachers miss out on important opportunities to improve their classes. For example, if 40 student essays all show the same misunderstanding of a concept, the teacher marking the assignments ideally will realise there’s a problem with how they’re teaching the subject.
Grading is overlooked
In response to AI, a huge amount of effort is now being spent on assessment reform in Australian universities. However, research shows many universities are reworking assessments with little attention given to marking or the final grades that result from this marking.
But there is an opportunity to explore other grading options at the same time. Research offers longstanding critiques of numerical grading, which most universities use. These critiques highlight how they can undermine independent thinking due to focusing on “meeting the mark”. Grades also encourage unnecessary competition among students.
What next?
Removing numerical marks and grades could be one way forward – although this does not provide all the answers.
The University of New South Wales has piloted the removal of numerical marks in some subjects. Instead, teachers evaluate students on whether they are competent against defined outcomes. The university says this shifts focus towards student learning.
While these changes weren’t made in response to AI, the approach still offers clues as to how marks and grades might change in the world of AI. It shows the future is not necessarily a question of technology versus no technology.
Rather, universities can focus on systems of assessment that bring students and teachers closer together. Students want meaningful feedback, not technical tick-box exercises. By worrying about the ways AI could replicate human marking, we may miss what matters in assessment.
- is a Professor (Assessment and Digital Learning), Deakin University
- This article first appeared in The Conversation
