How malicious AI swarms can threaten democracy

Summary

This Science Policy Forum article warns that the convergence of large language models with multi-agent AI architectures enables a qualitatively new threat: malicious AI swarms—persistent, adaptive, cross-platform collectives of agents that can infiltrate online communities, mimic human social dynamics, and fabricate consensus at scale. The authors argue that such swarms threaten democracy through multiple pathways beyond direct persuasion, including synthetic consensus, epistemic vertigo, training-data poisoning (“LLM grooming”), coordinated harassment, and erosion of institutional legitimacy. They propose a layered response combining continuous detection, provenance infrastructure, narrowly authorized defensive AI, an international “AI Influence Observatory,” and commercial-incentive levers to disrupt manipulation markets—while resisting both naive self-regulation and state-controlled speech regimes.

Key Contributions

• Introduces malicious AI swarms as a distinct category beyond prior coordinated inauthentic behavior and generative-AI disinformation.
• Offers a taxonomy of five swarm capabilities: fluid real-time coordination, social-network mapping and infiltration, human-level mimicry, self-optimization via live A/B testing, and persistent presence.
• Develops a typology of democratic harm pathways: manufactured consensus, segmented realities, LLM grooming, harassment, FUD/disengagement, elite attention concentration, antidemocratic mobilization, and legitimacy erosion.
• Proposes a multilayered governance agenda spanning detection, provenance, defensive AI under democratic oversight, simulation stress-tests, and a distributed AI Influence Observatory.
• Argues for shifting from voluntary platform compliance to commercial-incentive levers (delisting, demonetization of swarm content, audited bot-traffic metrics).

Methods

Conceptual and policy synthesis rather than empirical study. The authors integrate literature on influence operations, multi-agent LLMs, social contagion, and democratic theory; use historical framing across print/broadcast/digital eras; and draw on cases such as the 2016 IRA operation, the pro-Kremlin Pravda network, and 2024 elections in Taiwan, India, Indonesia, and the US. Documented trends, projections, and uncertainties are explicitly distinguished.

Findings

• Swarms differ from prior botnets through persistent identities, memory, coordinated-but-varied tone, real-time adaptation, and minimal human oversight.
• Earlier human-driven operations (e.g., IRA 2016) showed limited measurable persuasion, but AI removes prior cost, cadence, and iteration constraints.
• “LLM grooming”—flooding the web with fabricated content—appears designed to contaminate future model training data, embedding adversarial narratives in model weights.
• Symmetric “pro-social swarms” cannot reliably counter malicious ones, because the attention economy rewards outrage and ethical actors are constrained from manipulative tactics.
• Detection faces an arms race; the realistic goal is raising attacker cost rather than prevention.
• Provenance mechanisms raise manipulation costs but create trade-offs around privacy, dissident safety, and unverified users.

Connections

This piece sits upstream of empirical work probing whether LLM-driven agents can actually persuade or coordinate at scale, including Hackenburg2025-dj on LLM persuasion, Triedman2025-uy and Lin2025-xp on adversarial/agentic manipulation, and DeVerna2025-dl on AI-generated influence content. Its swarm framing extends the coordinated-inauthentic-behavior literature represented by Luceri2025-tr, Minici2024-tf, Kulichkina2026-zk, and Tornberg2025-ir, while its concern with training-data contamination and epistemic environment degradation connects to Yang2025-iv and Kuznetsova2025-nu. The governance proposals dialogue with platform- and election-focused analyses such as Schiffrin_undated-gi and Gerard2025-br.

Podcast

A research-radio episode discusses this paper: Listen