fg-zettelkasten

Deepfake financial fraud: The global regulation of ai-driven scams

3 min read

Deepfake financial fraud: The global regulation of ai-driven scams

Summary

This policy brief surveys the global regulatory landscape for deepfake-enabled financial fraud, arguing that generative AI has qualitatively transformed scams by making convincing impersonation cheap and scalable. The authors map a “scam ecosystem” spanning social platforms, messaging apps, telecoms, banks, crypto exchanges, and money laundering networks, and contend that effective regulation must shift responsibility from individual victims to the gatekeepers and intermediaries that enable AI-driven fraud. Through comparative review across roughly fifteen jurisdictions, they argue for liability-based regimes — exemplified by the UK’s mandatory APP reimbursement and Singapore’s Shared Responsibility Framework — over transparency-only or victim-focused approaches.

Key Contributions

  • One of the first comparative, cross-jurisdictional surveys focused specifically on deepfake-enabled financial fraud regulation.
  • A “scam ecosystem” framework that maps enabling actors (platforms, telecoms, banks, launderers) to discrete regulatory intervention points.
  • A normative distinction between prevention-oriented regulation and liability regimes, with an argument that liability is the stronger lever for changing platform behavior.
  • Synthesis of dispersed evidence on losses, victim reporting rates, and platform ad revenues into a policy-actionable picture.
  • Concrete recommendations: gatekeeper accountability, real-identity ad verification, federated data-sharing infrastructures, and cross-border coordination.

Methods

Comparative policy review across the UK, EU, US, Singapore, China, Taiwan, Australia, Indonesia, the UAE, Turkey, Azerbaijan, Denmark, Ireland, Brazil, and others. The authors synthesize investigative journalism (OCCRP, ICIJ, ProPublica, Reuters), industry reports (Deloitte, Resemble AI, F-Secure), and government sources (UNODC, FBI, US TIP Report) alongside academic literature on platform governance and SIM-registration regimes. Illustrative case studies — the Arup CEO deepfake, the Brad Pitt romance scam, the Indonesian Prabowo deepfake, the Hong Kong triad ID fraud, and the Chen Zhi/Prince Holding Group indictment — anchor the analysis.

Findings

  • Deloitte projects US generative-AI fraud losses rising from 40B (2025); Resemble AI estimates ~$350M in Q2 2025 alone; 92% of 575 surveyed businesses reported some deepfake-related loss.
  • Only ~37% of scam victims across 12 countries report the crime, and just 27% of those go to police — massive underreporting.
  • Meta serves over 15 billion “high risk” ads per day generating ~$7B annually, removing scam ads only when ~95% certain of fraud, while ad personalization steers vulnerable users toward more scam content.
  • End-to-end encrypted messaging apps serve as critical downstream infrastructure, evading moderation once initial contact migrates off public platforms.
  • Mandatory SIM registration has not reliably reduced scam prevalence and may introduce new vulnerabilities (e.g., third-party SIM purchases in Kenya/Tanzania).
  • Promising models include Singapore’s Shared Responsibility Framework and COSMIC platform, Taiwan’s Eagle Eye Anti-Fraud Alliance (federated learning across banks), and the UK’s mandatory APP fraud reimbursement.
  • Southeast Asian “scam compounds” rely on trafficked, coerced labor and integrate AI translation and voice cloning at industrial scale; convictions remain rare.
  • US Section 230 and corporate domicile effectively export US liability rules globally, as shown by Andrew Forrest having to sue Meta in California.
  • Transparency measures (EU AI Act deepfake labels, China’s deep synthesis rules) are necessary but insufficient: labels can be stripped and scams migrate to private channels.

Connections

This brief speaks directly to platform-governance debates about systemic risk and intermediary liability, complementing work on platform data access and the operational realities of moderation such as Rieder2026-pp and Rieder2025-ju. Its concern with AI-enabled deception at scale connects to empirical studies of generative-AI misuse and persuasion, including Hackenburg2025-dj and Triedman2025-uy, while its focus on synthetic media ecosystems resonates with platform-level analyses like Bouchaud2026-lr. The argument for gatekeeper accountability also intersects with broader work on platform power and regulation such as Larsson2026-ro and Helmond2026-ll.

Podcast

A research-radio episode discusses this paper: Listen

Graph View

Backlinks