Teachers are using software to see if students are using artificial intelligence. What happens when something goes wrong?

The teacher did not respond and deducted Ostowitz’s grade.

Ostowitz’s mother, Stephanie Rizk, said her daughter was a high-achieving student concerned about doing well in school, and she was shocked when teachers drew conclusions about Ostowitz’s work so early in the school year.

“Understand their skill level, and then your AI detector might be useful,” Rizk said.

Rizk told NPR that she met with teachers in mid-November who said they had never seen her daughter’s information.

The Prince George’s County Public Schools district made it clear in a statement that Ostowitz’s teachers used the artificial intelligence detection tool themselves and that the district did not pay for the software.

“During staff training, we advise educators not to rely on such tools as their potential inaccuracies and inconsistencies have been documented by multiple sources,” the statement said.

PGCPS declined to make Ostowitz’s teacher available for an interview. Rizk told NPR that after their meeting, the teacher no longer believed Ostowitz used artificial intelligence.

But what happened to Ostowitz was not surprising.

More than 40% of teachers in grades 6 to 12 used artificial intelligence detection tools last school year, according to a nationally representative poll from the Center for Democracy and Technology, a nonprofit organization that advocates for civil rights and civil liberties in the digital age.

This is despite numerous studies showing that AI detection tools are far from reliable.

Mike Perkins, a lead researcher in the field of academic integrity and artificial intelligence at the British University in Vietnam, said: “It is now fairly established in the academic integrity field that these tools are not fit for purpose.”

Perkins found that some of the most popular AI detectors (including Turnitin, GPTZero, and Copyleaks) were labeling something as AI when in fact it was not, and vice versa. When AI texts were manipulated to appear more human-like, their accuracy dropped further.

“We’re seeing some really concerning issues with some of the richest AI text detection tools,” he said.

Despite these problems, NPR found that school districts from Utah to Ohio to Alabama are still spending thousands of dollars on these tools.

Why one of the largest districts in the U.S. is using AI detection software

Broward County Public Schools near Miami spent more than $550,000 on a three-year contract with Turnitin. The long-established edtech company has historically provided plagiarism detection software to schools; in 2023, it introduced AI detection capabilities. When educators place student work through the tool, it generates a percentage that reflects the amount of text the software determines may have been generated by artificial intelligence. One caveat: Scores of 20 percent or lower are less reliable, according to the company.

“Turnitin tools help us facilitate conversations and feedback rather than grades,” said Sherri Wilson, director of innovative learning for the Broward School District, which with more than 230,000 students is one of the largest in the country.

Wilson said the district is “fully aware” of research showing that artificial intelligence testing tools, including Turnitin, are not 100% accurate or reliable.

Turnitin acknowledges this, too: On the company’s website, it says, “Our AI writing detection may not always be accurate…so it should not be used as the sole basis for taking adverse action against a student.”

Turnitin wrote in a statement to NPR that it was more important to avoid falsely accusing students of cheating than to capture all AI writing.

Wilson said the Turnitin tool remains valuable because it can quickly scan student work for suspicious uses of AI, saving teachers time.

Wilson said another reason Broward teachers are able to use the tool is because the district participates in academic programs such as the International Baccalaureate (IB) program, in which students’ work must be verified by teachers before being sent for external review.

The IB and Cambridge International Education programs offered in Broward told NPR that schools are not required to use artificial intelligence detection software as part of the identity verification process. Nonetheless, Broward told NPR in a statement, “We have chosen to provide our teachers with [Turnitin] as one of the tools to meet the requirements. “

But Wilson said teachers, not AI detection tools, are the final authority on whether students’ work is their own.

“They use these tools as feedback and then have those teachable moments with their students,” she said.

Why one teacher uses an AI detection tool

Language and literature teacher John Grady said that for him, the AI ​​detection tool provides “a starting point” to start a conversation with students who may have used AI.

“It’s certainly not foolproof,” he said. “But it gives you something to hang your hat on.”

Grady teaches at Shaker Heights High School, which is part of the Shaker Heights City School District outside Cleveland. The district, which serves about 4,400 students, will pay GPTZero, another AI detection software company, about $5,600 this year to provide annual licenses for the district’s 27 teachers. The tool calculates the percentage of likelihood that a student’s assignment was generated by artificial intelligence.

Grady said he puts all his student papers through GPTZero; if the tool shows more than a 50% chance of using artificial intelligence to complete the task, Grady digs deeper. This includes using revision history tools to see how much time students spent on an assignment and how many edits they made during the writing process. He would contact a student if he only did some editing and spent little time writing.

“I’ll say, ‘Hey, this is flagged. Can you tell me why?’ I would say most of the time, like 75 percent, if it’s AI, they’ll say, ‘Yeah, I did that. “I thought, ‘Well, now you have to rewrite it with less credit,'” Grady said.

This is how educators do it, says GPTZero co-founder and CEO Edward Tian should Use his company’s tools.

“We absolutely do not believe this is a punishment tool,” Tian said. “This needs to be a tool in the toolbox rather than the final smoking gun.”

He said it’s important to understand that a GPTZero probability score below 50% means the text is more likely to have been generated by a human rather than an AI. Scores above 50 percent, he said, deserve closer inspection — just like Grady described.

Tian does not deny that GPTZero is not always reliable research. But he noted that educators like Grady still find the information it provides valuable.

He said tools like his provide “a signal about what’s going on in the classroom,” but teachers should always follow up with students if that signal shows something worrisome.

AI detects skeptics

Zishi, a third-grader at Shaker Heights whose first language is Mandarin, said his writing style sometimes looks like artificial intelligence “because the words I use are repetitive. I think it’s because of my limited vocabulary.”

Shi, who is not a student of Grady’s, said he is still working on improving his writing skills and worries that the AI ​​detection software might be biased against non-native English speakers like him.

Some educators share these concerns, although research to date is limited and conflicting.

Shi said an assignment he completed for an English class earlier this fall was flagged by GPTZero as possibly generated by artificial intelligence. He said his teacher suggested he might have triggered the detection software using an online tool called Grammarly. Grammarly uses artificial intelligence to correct grammar and generate text when prompted. (The teacher confirmed Shi’s account to NPR.)

Shi said he only used Grammarly to clean up his writing, and he wrote the assignments himself. “It’s definitely disappointing to see comments labeling it as artificial intelligence,” Shi said.

Shi believes AI detectors should be thought of as “a smoke alarm, it’s a sign or a warning. But, you know, sometimes it can feel like a false alarm.”

He questioned whether school districts should spend thousands of dollars on artificial intelligence detection software. He said the money could be better spent on professional development for teachers.

Carrie Cofer, a high school English teacher in the Cleveland Metropolitan School District just a few miles from Shaker Heights, agreed.

Last year, as an experiment, she uploaded a chapter from her PhD thesis. The paper enters GPTZero. “Eighty-nine or 91 percent of it was written by artificial intelligence, and I thought, ‘Oh no, I don’t think that’s right because this is all mine,'” Cofer said.

Cofer is helping her district develop AI policies and guidelines; she said Cleveland schools are not currently paying for AI detection software and she opposes doing so.

“I don’t think it’s a good use of their money,” Cover said. “Kids are going to figure it out somehow.”

Some solutions students can take include using AI detection software themselves, taking workshop assignments so they don’t get flagged, and using “AI Humanization” programs, which claim to make AI-generated writing look more human.

Ultimately, teachers will need to adapt to AI by changing the way they teach and assess student learning, she said.

Back in Maryland, high school junior Elsa Ostowitz is adjusting too. Now, she runs all her assignments through a variety of AI-powered checking tools before handing them in.

She writes the essay herself, but she rewrites sentences that the software identifies as likely generated by artificial intelligence, an extra step that adds about half an hour to each assignment, she said.

“I think I’ve definitely become more vigilant about presenting my work as my work, rather than as artificial intelligence,” she explains.

She didn’t want to take any risks.

This report was supported by the Tarbell Artificial Intelligence News Center.