AI Detectors Aren’t the Answer. They’re One Data Point.

Sep 7

by Claire L. Brady, EdD

I get asked about AI detectors on nearly every campus I visit. Sometimes it comes during the presentation. Sometimes during Q&A. Sometimes a faculty member catches me afterward between presentations.

“What do you think about AI detectors?”

At this point, I know my answer.

I take a deep breath, lean toward the microphone, and say it slowly: I. Don’t. Like. Them.

Not because I don’t care about academic integrity. I care deeply about it deeply. And not because I think faculty should look the other way when they have legitimate concerns about whether a student’s work reflects their own learning. I don’t like AI detectors because I don’t think they are accurate enough for the weight we are increasingly asking them to carry, especially when the consequences for a student, the faculty, and the institution can be significant.

And I have another concern that gets far less attention: faculty independently purchasing or using third-party detectors and uploading student-created work into them. Ouch!

A new Guidance on AI Detectors resource from AI for Education and Vic Chamness gives educators something much more useful than another debate about whether detectors are “good” or “bad.” It summarizes what the evidence tells us about their limitations and, importantly, offers practical guidance for what educators can do instead. Although developed primarily with K–12 educators and school leaders in mind, there is a lot here that higher ed should be paying attention to.

The guide is straightforward about the evidence. Accuracy varies across tools and decreases when writing combines human and AI-generated text or has been edited or paraphrased. Detectors can miss AI-generated work and identify human writing as AI-generated. Research has also raised equity concerns, particularly for non-native English writers and students who translate their own work.

A distinction between AI detection and plagiarism detection

There is also an important distinction between AI detection and plagiarism detection. A detector score is not evidence in the same way a plagiarism match is. With plagiarism software, I can potentially examine the original source and compare the texts. An AI detector gives me a score or probability. There is no source document sitting on the other side that proves what happened. The guide’s bottom line is important: if detectors are used at all, they should be treated as a “preliminary signal, not proof.” That distinction matters enormously when we are talking about academic misconduct.

What happens to student work once we upload it somewhere?

Before student-created work is entered into any external product, institutions should be asking questions about privacy, data retention, intellectual property, security, terms of service, and whether faculty are authorized to provide student work to that vendor in the first place. Convenience shouldn’t bypass institutional review.

So what can faculty do when they have a legitimate concern? This is where I especially appreciate the second page of the guide. It shifts the focus toward professional judgment: clarify what AI use was permitted, review drafts and notes, talk with the student about their ideas and process, compare the work with other evidence of learning, document concerns, and follow established procedures.

And we can do even more upstream: Design assessments that make learning visible. Incorporate drafts, conversations, presentations, reflection, and process. Build AI literacy. Make expectations for AI use and disclosure clear. Our strongest response to AI is better teaching, clearer expectations, stronger assessment, and more direct conversation with students.

AI detectors may contribute one piece of information to that process. But they should never carry the whole case. Academic integrity deserves more than a percentage on a screen.

Read the full Guide here: https://www.aiforeducation.io/ai-resources/guidance-on-ai-detectors