EXPRESS: A short history of misinformation-at-scale and efforts to mitigate it

Summary

Joan Donovan offers a historical and conceptual account of how “misinformation-at-scale” emerged as a public problem between 2016 and 2021, and how platform companies — not states — became its de facto governors. She argues that the amplification of falsehoods by densely networked actors (politicians, journalists, celebrities) is not an aberration but a structural feature of engagement-driven, advertising-funded platforms. Tracking the arc from non-intervention through COVID-era and 2020-election interventions to post-2021 retrenchment under Musk-era X and a downsized Meta, Donovan contends that platforms preferred behavioral moderation (e.g., “coordinated inauthentic behavior”) over truth adjudication precisely because veracity conflicts with their revenue model. She proposes the concept of TALK — timely, accurate, local knowledge — as the public good platforms systematically fail to deliver, and calls for a whole-of-society mitigation strategy.

Key Contributions

• Defines and operationalizes misinformation-at-scale as a sociotechnical event distinct from individual rumor.
• Introduces TALK (timely, accurate, local knowledge) as the missing public good in platform-mediated information environments.
• Provides a taxonomy of fourteen content moderation methods mapped to the actors (individuals, groups, platforms, governments) empowered to deploy them.
• Periodizes platform content moderation from 2016–2021: non-intervention → active moderation → retrenchment.
• Synthesizes platform governance, public health communication, and democratic theory to argue for civil society coalitions, advertiser pressure, journalistic strategic silence, and pre-bunking.

Methods

Comparative historical analysis of platform moderation policies (2016–2024) combined with ethnographic case studies of Charlottesville, COVID-19, the 2020 election, January 6th, and the Hunter Biden laptop story. Donovan also draws on participant-observation with civil society coalitions (Disinformation Defense League, Election Integrity Partnership), and synthesizes whistleblower disclosures, congressional hearings, and investigative reporting alongside a conceptual taxonomy of moderation methods.

Findings

• Nine of fourteen catalogued moderation methods target amplification rather than removal — reflecting platforms’ preference for limiting scale over adjudicating truth.
• Facebook’s CIB framework let it act on behavioral signals without ruling on veracity, repurposing spam/fraud tooling.
• Labeling Trump’s tweets reduced on-platform engagement but increased cross-platform engagement (“platform filtering”).
• Medical misinformation was moderated more aggressively than political misinformation; COVID information centers exemplify information displacement toward authoritative sources.
• Post-2021 retrenchment: Meta laid off 11,000 staff, Musk dismantled X’s civic integrity and ethics teams, Google cut 6%.
• Misinformation actors adapt rapidly via coded language (“va33ine”), backup accounts, and platform migration.
• Trump’s January 2021 deplatforming marked an inversion of power: platform corporations could challenge state authority.

Connections

Donovan’s structural account of engagement-driven amplification connects to platform-infrastructure histories like Helmond2026-ll and Rieder2025-ju, and to the post-2021 retrenchment of trust-and-safety documented in Bak-Coleman2025-pm and Murtfeldt2025-wu. Her case-study work on the 2020 election and January 6th sits alongside Starbird2025-jj and Prochaska2025-ef on the Election Integrity Partnership milieu, while the call for pre-bunking and whole-of-society mitigation links to van-der-Linden2026-jt and Lewandowsky2026-ob. The argument that misinformation-at-scale is amplified by elite, densely networked actors resonates with Budak2024-ef and Gonzalez-Bailon2024-rq on the empirical distribution of exposure.