‘Writing is thinking’: Do students who use ChatGPT learn less?

PARIS - When Professor Jocelyn Leitzinger had her university students write about times in their lives they had witnessed discrimination, she noticed that a woman named Sally was the victim in many of the stories.

“It was very clear that ChatGPT had decided this is a common woman’s name,” said Prof Leitzinger, who teaches an undergraduate class on business and society at the University of Illinois in Chicago.

“They weren’t even coming up with their own anecdotal stories about their own lives,” she told AFP news agency.

Prof Leitzinger estimated that around half of her 180 students used ChatGPT inappropriately at some point last semester – including when writing about the ethics of artificial intelligence (AI), which she called both “ironic” and “mind-boggling”.

So she was not surprised by recent research that suggested that students who use ChatGPT to write essays engage in less critical thinking.

The pre-print study, which has not been peer-reviewed, was shared widely online and clearly struck a chord with some frustrated educators.

The MIT researchers behind the paper have received more than 3,000 e-mails from teachers of all stripes since it was published online in June, lead author Nataliya Kosmyna told AFP.

‘Soulless’ AI essays

For the small study, 54 adult students from the greater Boston area were split into three groups. One group used ChatGPT to write 20-minute essays, one used a search engine, and the final group had to make do with only their brains.

The researchers used electroencephalography (EEG) devices to measure the brain activity of the students, and two teachers marked the essays.

The ChatGPT users scored significantly worse than the brain-only group on all levels.

The EEG showed that different areas of their brains connected to each other less often.

And more than 80 per cent of the ChatGPT group could not quote anything from the essay they had just written, compared with around 10 per cent of the other two groups.

By the third session, the ChatGPT group appeared to be mostly focused on copying and pasting.

The teachers said they could easily spot the “soulless” ChatGPT essays because they had good grammar and structure but lacked creativity, personality and insight.

However, Dr Kosmyna pushed back against media reports claiming the paper showed that using ChatGPT made people lazier or more stupid.

She pointed to the fourth session, when the brain-only group used ChatGPT to write their essays and displayed even higher levels of neural connectivity.

Dr Kosmyna emphasised it was too early to draw conclusions from the study’s small sample size but called for more research into how AI tools could be used more carefully to help learning.

Dr Ashley Juavinett, a neuroscientist at the University of California San Diego, who was not involved in the research, criticised some “off-base” headlines that wrongly extrapolated from the pre-print.

“This paper does not contain enough evidence nor the methodological rigour to make any claims about the neural impact of using LLMs (large language models such as ChatGPT) on our brains,” she told AFP.

Thinking outside the bot

Prof Leitzinger said the research reflected how she had seen student essays change since ChatGPT was released in 2022, as both spelling errors and authentic insight became less common.

Sometimes, students do not even change the font when they copy and paste from ChatGPT, she said.

But Prof Leitzinger called for empathy for students, saying they can get confused when the use of AI is being encouraged by universities in some classes but is banned in others.

The usefulness of new AI tools is sometimes compared to the introduction of calculators, which required educators to change their ways.

But Prof Leitzinger expressed worry that students do not need to know anything about a subject before pasting their essay question into ChatGPT, skipping several important steps in the process of learning.

A student at a British university, who is in his early 20s and wanted to remain anonymous, told AFP that he found ChatGPT was a useful tool for compiling lecture notes, searching the internet and generating ideas.

“I think that using ChatGPT to write your work for you is not right because it’s not what you’re supposed to be at university for,” he said.

The problem goes beyond high school and university students.

Academic journals are struggling to cope with a massive influx of AI-generated scientific papers. Book publishing is also not immune, with one start-up planning to pump out 8,000 AI-written books a year.

“Writing is thinking, thinking is writing, and when we eliminate that process, what does that mean for thinking?” Prof Leitzinger asked. AFP