King, G., & Worthington, S. (2026). AI article reviews dominate human peer reviews at their best: Evidence and policy choices. Gary King.
Summary
This paper presents evidence that AI-generated reviews of academic articles now outperform reviews written by well-compensated human experts — and does so across every quality measure the authors constructed. Crucially, the human comparison set was not drawn from ordinary peer review but from reviews that journal editors and authors had already judged to be of unusually high quality, making the AI’s superiority a result against a demanding benchmark. Having established this empirical finding, the authors turn to its implications for journal policy, weighing the concerning downsides of AI in peer review against potential gains in publication speed, review quality, and the experience of everyone involved in the process.
Key Contributions
- Empirical evidence that AI article reviews surpass strong human expert reviews on every evaluation measure constructed.
- A demanding human benchmark built from reviews pre-judged by editors and authors to be of unusually high quality.
- A policy framework for journals to reason through the risks and opportunities of adopting AI in peer review.
- Identification of concrete potential benefits: faster publication, higher quality, and reduced adversarial interactions.
Methods
The authors compared AI-generated reviews against human expert reviews using multiple constructed measures of review quality. The human benchmark was deliberately stringent — a set of reviews that editors and authors had independently flagged as high quality. The empirical comparison is paired with a policy analysis discussing downsides and benefits across the stakeholders of peer review (editors, reviewers, authors, and readers), though the fine details of that analysis are not available from the abstract.
Findings
- AI reviews outperformed high-quality human expert reviews on every evaluation measure used.
- The result holds even against the unusually well-regarded human benchmark.
- Alongside the capability gains, the authors flag concerning downsides as well as potential paths to improve the publication process.
Connections
This paper sits within the growing body of work on using LLMs to augment scholarly evaluation and research workflows. It connects most directly to work benchmarking LLMs against human judgment and annotation quality, such as Tan2024-vl and Alizadeh2026-es, and to studies examining LLMs as substitutes or complements for human expertise in evaluative tasks like Balluff2026-if and Matias2025-px.