DetectGPT Explained: How It Detects AI Text With Probability Curvature

DetectGPT is a zero-shot method for detecting AI-generated text that asks one simple question: if you nudge a passage with small rewrites, does a language model like it less? Machine-written text tends to sit at a local peak of the model's probability, so almost any change lowers its score. Human text usually does not show that sharp peak.

The idea comes from a 2023 paper by Eric Mitchell and colleagues, presented at ICML 2023. It shaped much of the detection research that followed, including the faster methods many tools rely on today. This explainer walks through the method in plain terms, the results the authors reported, and what it means if your own writing is checked by a detector.

The problem DetectGPT set out to solve

Before DetectGPT, detectors came in two broad families. Trained classifiers learn from labelled examples of human and machine text, which works well on data that looks like the training set and less well on anything new. Zero-shot statistics, such as average log probability or token rank, need no training but are crude: a passage can be highly probable simply because its topic is common or its style is formal.

Mitchell and colleagues wanted something in between. Their detector would use only the log probabilities of the model suspected of writing the text, plus a generic tool for making small edits. No labelled dataset, no classifier training, and no watermark planted at generation time. If you want the wider context first, our overview of how AI detectors work covers the main detector families.

How DetectGPT works: probability curvature in plain language

A language model assigns a probability to every passage. Picture all possible passages as a terrain of hills and valleys, where height is how likely the model thinks a passage is. When a model generates text, it tends to choose high-probability words at each step, so its outputs land near the tops of hills.

Now take a passage and make a few small, meaning-preserving edits. If the passage sat on a hilltop, every step away goes downhill, and the edited versions score lower. If the passage sat on a gentle slope or flat ground, as human writing often does, some edits go up, some go down, and the average change is close to zero. The paper calls this shape negative curvature: the top of a hill curves downward on every side.

• Model-generated passage
• Human-written passage

Illustrative example of the idea, not data from the paper

In practice, the method runs in four steps.

1. Score the originalCompute the log probability of the passage under the model you suspect wrote it.
2. Perturb itMask short spans of words and let a separate model, T5, fill them in with plausible alternatives.
3. Score the copiesCompute the log probability of each perturbed copy under the same suspect model.
4. CompareIf the original scores clearly higher than the average copy, flag the passage as likely machine-generated.

The perturbation model matters. The authors used T5, a mask-filling model, to replace short spans so the copies stay fluent and close in meaning. In their main setup they masked about 15% of words in spans of two words and drew 100 perturbed samples per passage. The final score is the gap between the original and the perturbed average, normalized by how much the perturbed scores vary.

What the DetectGPT paper reported

The headline result came from fake news detection. On news articles generated by the 20-billion-parameter GPT-NeoX model, DetectGPT reached an AUROC of 0.95, compared with 0.81 for the strongest zero-shot baseline. AUROC measures how well a score separates two groups across all possible thresholds: 0.5 is a coin flip and 1.0 is perfect separation.

The authors tested models from 1.5 billion parameters (GPT-2) up to 175-billion-parameter models such as GPT-3, across news (XSum), Wikipedia paragraphs (SQuAD contexts) and creative writing prompts from Reddit. They also ran a simple test of human-style revision: replacing growing portions of machine text with T5 rewrites. Detection stayed above 0.8 AUROC even when nearly a quarter of the text had been replaced.

Why DetectGPT matters for people whose writing gets checked

Most commercial detectors do not publish their internals, and many are trained classifiers rather than DetectGPT-style tests. Still, the paper names the signal that a lot of detection rests on: text that is too predictable for a model. That has a few practical consequences for researchers.

• Formal prose is predictable by design. Methods sections, standard hedges and field phrases are repeated across thousands of papers. A model finds them likely, which pushes careful human writing closer to the machine side. We discuss this in why formal prose can look AI-written.
• The score is about the passage, not the author. DetectGPT measures a property of text under one model. It cannot see your drafts, notes or intent.
• Short text is harder. Fewer tokens means a noisier curvature estimate. A single abstract gives the test less to work with than a full chapter.

The detector also responds to the specific model used to score the text. Text that sits on a peak for one model may not sit on a peak for another, which is one reason different tools disagree about the same paragraph.

Limits of DetectGPT

The paper is candid about its constraints, and later work exposed more.

It needs the right model and a lot of compute

DetectGPT works best when it can query the log probabilities of the model that wrote the text. For closed commercial models that access is limited, and using a different scoring model weakens the signal. Cost is the other problem: each passage needs one hundred extra forward passes. The follow-up Fast-DetectGPT paper timed DetectGPT at 79,113 seconds on its benchmark against 233 seconds for its own method on the same A100 GPU, a gap of roughly 340 times.

Paraphrasing breaks the curvature signal

The method assumes the passage still sits where the model left it. A paraphrase moves it. Krishna and colleagues tested this directly in their NeurIPS 2023 paper on the DIPPER paraphraser: rewriting machine text cut DetectGPT's detection rate from 70.3% to 4.6% at a fixed 1% false positive rate. Sadasivan and colleagues later reported that paraphrasing dropped the AUROC of this kind of zero-shot detector from 96.5% to 25.2%. The same weakness is why prompt tricks keep coming up; see why simple prompts rarely make ChatGPT text undetectable.

• Original machine text70.3%
• After DIPPER paraphrase4.6%

Source: Krishna et al., "Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense", NeurIPS 2023

The perturbation model shapes the result

The authors note that their approach needs a reasonable perturbation function. T5 works for English, but in domains where an off-the-shelf mask filler does not produce realistic rewrites, performance can drop. Specialized scientific language is a plausible case.

What came after DetectGPT

DetectGPT started a line of zero-shot detectors that keep the core intuition and cut the cost. Fast-DetectGPT replaces full-passage rewrites with token-level sampling, and Binoculars compares two related models to correct for topics that are surprising in themselves. We cover both in Fast-DetectGPT and Binoculars explained. On the other side, research on paraphrasing attacks, summarized in can AI-generated text be reliably detected?, shows how much these signals depend on text staying close to what a model would produce.

Where Academic Humanizer fits

Detectors built on predictability ideas tend to react to individual sentences that read as too smooth or too generic, even in drafts whose ideas and data are entirely the author's. Academic Humanizer works at that level: it scans a draft for sentences that detectors flag, rewrites only those while keeping claims, numbers, citations and terminology, and leaves the rest exactly as written. The built-in AI Detection Report shows each flagged sentence next to its rewrite so you can check every change before using it, and it is worth confirming your institution's or journal's AI policy as part of that review.