Can AI-Generated Text Be Reliably Detected? The Sadasivan Paper

Are AI detectors reliable? Not when someone is actively trying to evade them, according to "Can AI-Generated Text Be Reliably Detected?", a 2023 paper by Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang and Soheil Feizi, and less and less as language models get better at sounding human. The paper, later published in Transactions on Machine Learning Research, became one of the most cited arguments in the debate over AI detectors.

It made three claims. Paraphrasing machine text, especially repeatedly, defeats every major type of detector. Detectors can also be spoofed into flagging human text. And there is a mathematical ceiling on how well any detector can do. This post explains each, then looks at the work that pushed back.

Claim 1: recursive paraphrasing breaks detectors

The attack is easy to describe. Take a passage written by a language model and run it through a paraphrasing model. Then paraphrase the paraphrase, and repeat. The authors call this recursive paraphrasing and tested it on passages of about 300 tokens using several paraphrasers, including DIPPER, an 11-billion-parameter model built by another team for exactly this kind of study.

They aimed it at four detector families:

• Watermark detectors, which look for a hidden statistical pattern planted at generation time. Our explainer on the LLM watermarking paper covers how they work.
• Trained classifiers, such as the RoBERTa-based detector released by OpenAI.
• Zero-shot detectors such as DetectGPT, which read the model's own probabilities.
• Retrieval detectors, which compare a passage against a database of everything a model has generated.

• Watermark, original (TPR at 1% FPR)99.3%
• Watermark, after recursive paraphrasing9.7%
• Zero-shot detector, original (AUROC)96.5%
• Zero-shot detector, after paraphrasing (AUROC)25.2%

Source: Sadasivan et al., "Can AI-Generated Text be Reliably Detected?", arXiv 2303.11156

For watermarked text, recursive paraphrasing dropped the true positive rate at a 1% false positive rate from 99.3% to 9.7%. For the zero-shot detector, AUROC fell from 96.5% to 25.2%, which is worse than random guessing. The authors also ran human evaluations and reported that the paraphrases mostly kept their meaning and grammar, though quality slipped in some cases.

Claim 2: detectors can be spoofed

The second attack runs the other way: make human text look machine-made. The authors showed that an attacker who only queries a watermarked model, without seeing how the detector works, can estimate which words the watermark favours. Writing with those words makes human text trigger the watermark. In their experiment, spoofing drove the soft watermark's detection AUROC from 99.8% down to 1.3%, meaning the detector could no longer tell real watermarked text from forged text.

For anyone whose work is checked by detectors, this is the more uncomfortable result. A detector that can be fooled into accusing human writing is a reputational risk for the people it judges, not just for the model developer.

Claim 3: the AUROC bound

The theoretical result ties detection to a single number. Think of all the text humans might write as one probability distribution and all the text a model might write as another. The total variation distance between them, TV, measures how different the two are, from 0 (identical) to 1 (no overlap). The paper proves that for any detector:

AUROC ≤ 1/2 + TV − TV²/2

When TV is large, the bound is loose and a good detector is possible. As models improve and their output distribution moves toward human writing, TV shrinks and the best achievable AUROC slides toward 0.5, a coin flip.

Values computed from the paper's bound; they are not experimental results.

• Distributions far apart
• Distributions close together

Illustrative example of the bound's meaning, not data from the paper

The counterarguments

The paper drew strong responses. Three are worth knowing.

More text makes detection possible again

Souradip Chakraborty and colleagues argued in a position paper at ICML 2024, "Position: On the Possibilities of AI-Generated Text Detection", that the bound applies to a single sample. If a detector sees more text, either longer passages or several samples from the same source, the distributions become easier to tell apart, and detection stays feasible unless human and machine text are identical across their whole range. They derived how much text is needed. In short: the ceiling is real per sample, but it rises with length.

Watermarks survive more than expected

The watermarking team behind the original green-list method, led by John Kirchenbauer, studied this in "On the Reliability of Watermarks for Large Language Models" (ICLR 2024). They found watermarks remained detectable after both human and machine paraphrasing, because paraphrases tend to keep fragments of the original. In one setting with heavy human rewording, the watermark was detectable after about 800 tokens on average at a very low false positive rate.

Retrieval as a defense

Kalpesh Krishna and colleagues, who built DIPPER, proposed in their NeurIPS 2023 paper that a model provider could store its own outputs and search them. With 15 million stored generations, they detected 80% to 97% of paraphrased outputs at a 1% false positive rate. Sadasivan et al. responded that recursive paraphrasing also weakens retrieval. We cover DIPPER in how DIPPER paraphrasing evades AI detectors.

What it means in practice

The debate settles into a reasonable middle position. Detection works better on long texts than short ones, works better when the detector knows which model to expect, and gets weaker as models and rewriting tools improve. None of the counterarguments claims that a single paragraph can be judged with certainty. For how this plays out with one commercial detector, see can Turnitin detect humanized text?

• Short passages carry the most uncertainty. An abstract or a single paragraph is exactly where the per-sample bound bites. Benchmarks like RAID show how performance varies across settings.
• Errors cut both ways. Detectors miss rewritten machine text and can also flag human text, as the work on bias against non-native writers shows.
• A score is a probability, not a verdict. Institutions that treat it as proof are relying on more certainty than the research supports.

Where Academic Humanizer fits

If detection is uncertain at the level of a paragraph, the sensible response for honest authors is to look at the sentences that drive a score rather than guess. Academic Humanizer scans a draft for sentences that detectors flag, rewrites only those while keeping meaning, numbers and citations, and leaves human-read sentences as written. Its AI Detection Report compares the before and after versions on the same measure and says plainly if anything is still flagged. You review and approve every change, and checking your institution's or journal's AI policy remains part of the job.