What Is an AI Writing Detector and Can It Make Mistakes?

How AI Detectors Analyze Writing

Large Language Models (LLMs) such as ChatGPT (GPT-4), Anthropic Claude, and Google Gemini generate text by predicting the mathematically most probable next token (word or sub-word) in a sentence. Because of this predictive nature, AI-generated writing leaves distinct statistical traces:

1. Perplexity (Word Choice Predictability)

Perplexity measures how "surprising" or statistically unexpected a word sequence is. LLMs select conventional, statistically optimal transitions. Human writers use quirky synonyms, metaphors, and unexpected phrasing that generate high perplexity.

2. Burstiness (Sentence Length and Rhythm Variance)

Humans write with significant rhythmic variation: short declarative statements followed by lengthy compound-complex sentences with varied clause structures. AI models tend to produce uniform sentence lengths and structured paragraph tempos.

3. Transition Vocabulary Signatures

AI models disproportionately rely on specific transitional connectors: "delve", "testament", "beacon", "pivotal", "in conclusion", "furthermore", "tapestry".

The Reality of False Positives

AI detection is probabilistic, not deterministic. An AI score of 75% means the text exhibits linguistic features characteristic of AI generation; it is never absolute forensic proof of academic dishonesty.

Common reasons for false positives include:

CiteRight's Ethical Guidelines for Lecturers & Universities

1. Never Fail a Student Solely on an AI Score: Treat AI detector reports as a conversation starter, not as definitive evidence for disciplinary action. 2. Review Document Edit Histories: Ask students to share Google Docs or Microsoft Word version histories demonstrating draft progression. 3. Conduct Oral Defenses (Viva Voce): Inquire about the research methodology; students who authored their papers can explain arguments with ease.