AI Detection Research Papers: A Reading List of 11 Key Studies

If you want to understand AI text detection rather than just trust a score, about a dozen papers do most of the work. They explain how detectors measure text, how well they hold up, who they fail, and how easily surface rewriting moves their verdicts. This reading list covers eleven of them, from 2019 to 2024, plus one study written for educators.

Each entry gives the full title, authors, venue, one verified finding and a link to the paper. Where we have a longer explainer, it is linked too.

1. 2019: early toolsGLTR and the GPT-2 output detector show two approaches that still dominate: token statistics and trained classifiers.
2. 2023: zero-shot and watermarksDetectGPT and LLM watermarking propose detection without a trained classifier.
3. 2023: first stress testsDIPPER shows paraphrase evasion; Liang et al. show bias against non-native writers.
4. 2024: cheaper detectors, bigger benchmarksFast-DetectGPT, Binoculars and Ghostbuster improve detection, while RAID and M4 test it at scale.

Foundations: how detection started

1. GLTR: Statistical Detection and Visualization of Generated Text

Sebastian Gehrmann, Hendrik Strobelt, Alexander M. Rush · ACL 2019 (System Demonstrations)

GLTR colors each word by how highly a language model ranked it. Machine text tends to stay among the top-ranked choices, while people pick less expected words more often. In a human study, GLTR raised people's accuracy at spotting generated text from 54% to 72% without prior training. Its core idea, that predictability is a signal, sits under most detectors today. Paper

2. Release Strategies and the Social Impacts of Language Models

Irene Solaiman et al. (OpenAI) · Report, 2019

This report on the staged release of GPT-2 describes the RoBERTa-based classifier that became the widely used "GPT-2 Output Detector". The authors report it detected text from the 1.5-billion-parameter GPT-2 with approximately 95% accuracy. The same detector reappears in later evaluations: in RAID it scored above 95% on GPT-2 text in five domains, but rarely above 60% on text from other models. Paper

Zero-shot detection and watermarking

3. DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature

Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, Chelsea Finn · ICML 2023

DetectGPT observes that model-written text tends to sit at a local peak of the model's log probability: small perturbations make it less likely. It needs no training data, only the model's probabilities and a perturbation model such as T5. For fake news generated by GPT-NeoX, it raised AUROC from 0.81 for the best earlier zero-shot method to 0.95. Read our DetectGPT explainer. Paper

4. A Watermark for Large Language Models

John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, Tom Goldstein · ICML 2023

Instead of guessing from style, this approach plants a signal at generation time. Before each word, a pseudo-random "green list" of tokens is chosen and softly favored; a detector later counts green tokens and runs a statistical test with interpretable p-values, without needing the model itself. It only works if the model provider turns it on. Read our watermarking explainer. Paper

Stress tests: evasion and bias

5. Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense

The DIPPER paper (card above) is the starting point of humanizer research. An 11B-parameter paraphraser weakened watermarking, DetectGPT, GPTZero and OpenAI's classifier while keeping meaning. Its proposed fix, searching a provider's stored generations for matching content, caught 80% to 97% of paraphrases with a 1% false positive rate. Read our DIPPER explainer.

6. GPT detectors are biased against non-native English writers

Weixin Liang, Mert Yuksekgonul, Yining Mao, Eric Wu, James Zou · Patterns, 2023

Seven popular detectors were tested on 91 TOEFL essays by non-native writers and 88 essays by US eighth graders. They were near perfect on the US essays but flagged over half the TOEFL essays as AI, with an average false positive rate of 61.22%; 89 of 91 were flagged by at least one detector. Enriching the vocabulary of those same essays cut the rate to 11.77%. Read our explainer on detector bias. Paper

2024: faster detectors and harder benchmarks

7. Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature

Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, Yue Zhang · ICLR 2024

Fast-DetectGPT replaces DetectGPT's expensive perturbation step with a sampling-based measure of conditional probability curvature. The authors report about a 75% relative improvement over DetectGPT in both white-box and black-box settings, and roughly 340 times faster detection. Paper

8. Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text

Abhimanyu Hans et al. · ICML 2024

Binoculars compares how surprising a text is to one model with how surprising one model's predictions are to another. Without any training data, it detects over 90% of samples from ChatGPT and other LLMs at a false positive rate of 0.01%. In RAID it was the standout at low false positive rates. Read our Fast-DetectGPT and Binoculars explainer. Paper

9. Ghostbuster: Detecting Text Ghostwritten by Large Language Models

Vivek Verma, Eve Fleisig, Nicholas Tomlin, Dan Klein · NAACL 2024

Ghostbuster runs documents through several weaker language models, searches over combinations of their features and trains a classifier on the result, so it needs no access to the target model's probabilities. It reached 99.0 F1 across domains, 5.9 F1 above the best earlier model, and came with new datasets of student essays, creative writing and news. Paper

10. M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection

Yuxia Wang et al. · EACL 2024

M4 spans six generators (including GPT-4, ChatGPT, Cohere, Dolly-v2 and BLOOMz), domains from Wikipedia to arXiv abstracts and peer reviews, and seven languages: Arabic, Bulgarian, Chinese, English, Indonesian, Russian and Urdu. Its main finding is that detectors struggle to generalize to unseen domains and models, and in those cases tend to call machine text human. It became the basis of SemEval-2024 Task 8. Paper

11. RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors

Liam Dugan et al. · ACL 2024

RAID tests 12 detectors on over 6 million generations from 11 models, 8 domains, 4 decoding strategies and 11 adversarial attacks, with every detector calibrated to the same false positive rate. Detectors that look excellent in one setting can fail in another, and a repetition penalty alone cut accuracy by up to 32 points. Read our RAID explainer. Paper

One more for educators

Testing of detection tools for AI-generated text by Debora Weber-Wulff et al. (International Journal for Educational Integrity, 2023) tested 14 tools on 54 controlled documents. None reached 80% accuracy, and machine paraphrase cut average accuracy to 26%. Read our summary of the 14-tool study or the paper. For the theory behind why detection may stay hard, see our post on whether AI text can be reliably detected. For what instructors look at besides a score, see can professors detect AI?

How to read these papers

• Check the false positive rate. An accuracy figure without one is hard to interpret. DIPPER, Binoculars and RAID all report detection at a fixed low false positive rate for this reason.
• Check which models and domains were tested. Results on GPT-2 or news text may not carry over to a current model writing a methods section.
• Note the date. Commercial tools update often, so a 2023 score for a named product is a snapshot.
• Look for who gets misflagged. Liang et al. and Weber-Wulff et al. both show errors falling hardest on non-native and translated writing.

Where Academic Humanizer fits

Read together, these papers show that detectors respond to surface statistics that careful human writing can share. Academic Humanizer is built for researchers whose own ideas get mislabelled for that reason. The Humanizer rewrites only the sentences detectors flag, keeps your argument, numbers and citations, and its AI Detection Report compares the draft and the output on the same measure so you can review every change. As always, check your institution's or journal's AI policy before you submit.