In
the six months before Taiwan’s January 2024 presidential election,
researchers at the Doublethink Lab documented over 200 instances of
AI-generated content designed to influence the electoral outcome, including
synthetic audio clips attributed to candidates, AI-generated news anchor
videos presenting fabricated political stories, and coordinated networks of
AI-generated social media accounts amplifying divisive narratives. The
operation was attributed to actors with links to Chinese state infrastructure,
and its sophistication represented a qualitative step beyond the
disinformation campaigns documented in previous Taiwanese elections. Taiwan’s
sophisticated counter-disinformation infrastructure, including government
fact-checking services, media literacy education embedded in school
curricula, and strong investigative journalism, meant that the campaign was
substantially exposed and its impact was limited. The lesson from Taiwan is
not that AI disinformation is unstoppable, but that the defences required to
withstand it are extensive and require sustained societal investment that
most democracies have not made.
The use of AI to generate synthetic political content designed to
mislead voters has moved in the past three years from a speculative concern
of researchers to a documented operational reality in elections across every
inhabited continent. The 2024 cycle, which included major elections in the
US, UK, India, Mexico, Indonesia, and dozens of other countries, produced the
most extensive documentation yet of AI-generated electoral disinformation at
scale. The combination of accessible AI synthesis tools, declining platform
content moderation resources, and a geopolitical environment in which several
state actors have clear motives to disrupt democratic processes in adversary
countries has created conditions in which AI electoral disinformation is not
merely possible but actively deployed in ways that are affecting democratic
outcomes.
The Taxonomy of AI Electoral Disinformation
AI electoral disinformation takes several distinct forms with
different risk profiles and different countermeasures. Synthetic audio, which
can convincingly reproduce the voice of any public figure from a few seconds
of genuine recording, carries the highest immediate credibility risk because
most people have no experience evaluating audio for authenticity and no tools
to do so readily available. The robocall using President Biden’s cloned voice
to suppress New Hampshire primary turnout was crude by current standards, but
it demonstrated that voice cloning for electoral manipulation is accessible,
cheap, and capable of affecting voter behaviour even when subsequently
debunked. Synthetic video is more technically demanding but increasingly
accessible, with tools including Sora and open-source alternatives enabling
convincing face swap and lip sync operations that were previously available
only to well-resourced state actors.
AI-generated text content, produced at scale by automated systems,
is the most prevalent category of AI electoral disinformation by volume.
Networks of AI-generated social media accounts, sometimes called bot farms
with an AI upgrade, can flood platforms with consistent messaging that
creates the appearance of grassroots political sentiment where none exists.
AI-generated news sites that produce superficially credible journalism
amplifying specific political narratives have been documented in multiple
countries, providing a distribution infrastructure for disinformation that
bypasses the credibility gatekeeping of established media. Research from
EU DisinfoLab has
mapped several such networks operating in European electoral contexts,
finding operational similarities that suggest coordination between campaigns
targeting different countries.
Platform Response and Its Inadequacy
The major social media platforms have introduced policies
requiring disclosure of AI-generated political content in paid advertising,
but these policies have three significant limitations that reduce their
effectiveness against the documented disinformation campaigns. First, they
apply only to paid advertising, not organic content distribution, which is
how most AI electoral disinformation actually spreads. Second, they depend on
self-disclosure by the content creator, which bad actors do not provide.
Third, they require detection capabilities that are outpaced by generation
capabilities, a structural deficit in the current technological arms race.
Meta, Google, TikTok, and X have all been criticised by researchers and civil
society organisations for enforcement that is substantially below the
standard their policies nominally require, particularly for content in
non-English languages where their moderation capabilities are weakest.
The platforms’ structural incentive to moderate content is
complicated by the fact that political disinformation generates engagement,
and engagement is the metric their advertising businesses depend on. A
divisive synthetic political video that generates millions of views before
being removed has already accomplished most of its disinformation objective,
and the platform has generated advertising revenue from those views. Fixing
this requires changing the incentive structure, not merely the policies, and
that requires regulatory intervention that platforms cannot
self-impose.
What This Means for You
Every voter in a democracy now has a practical interest in
developing the media literacy skills and verification habits that the
synthetic information environment requires. This means treating audio and
video of political figures with the same scepticism previously reserved for
text claims, verifying surprising or emotionally provocative political
content against multiple independent sources before sharing it, and being
aware that the platforms delivering political content to you have neither the
capability nor the incentive to reliably distinguish synthetic from authentic
material. These individual practices matter and are teachable. They are not,
however, sufficient as a societal response to what is fundamentally a
structural problem requiring regulatory, platform governance, and
international cooperation solutions. For related analysis, see our coverage
of synthetic
voices and political choice and AI
and the UK general election.
Legislative response to AI electoral disinformation is
accelerating internationally but substantially lags deployment. Germany
passed AI disinformation disclosure legislation ahead of its 2025 federal
election. The EU Code of Practice on Disinformation has been strengthened to
include AI-specific provisions under the Digital Services Act. In the UK, the
Elections Act 2022 made some progress on digital imprint requirements but did
not address AI-generated content specifically, and subsequent legislative
proposals have not advanced. Building the cross-party consensus required for
electoral law reform is difficult when different parties calculate different
strategic interests in the rules. The Electoral Reform
Society has published detailed recommendations for AI electoral
disinformation governance that provide a substantive policy starting point,
and the civil society pressure needed to translate these recommendations into
legislation depends on sustained public engagement with the electoral
integrity implications of AI disinformation that most voters have not yet
developed.
The media literacy infrastructure
needed to support voter resilience against AI electoral disinformation is
underdeveloped in most democracies. Finland’s model of media literacy
education integrated throughout the school curriculum, which has been cited
by researchers as contributing to Finnish citizens’ relatively strong
resilience against disinformation campaigns, provides a benchmark that most
other countries including the UK have not approached. Investing in media
literacy education that specifically addresses AI-generated synthetic
content, teaching young people from an early age to verify sources, evaluate
evidence, and maintain scepticism about emotionally compelling political content,
is a long-term investment in democratic resilience that has a stronger
evidence base than most short-term regulatory interventions.
The technical arms race between
deepfake generation and detection has no foreseeable end under current
conditions. Detection tools improve; generation tools improve faster and
become more accessible. The asymmetry between the cost of creating convincing
synthetic political content and the cost of debunking it at scale is
structural and is not being closed by current investment in detection
technology alone. Addressing this asymmetry requires mandatory watermarking
of AI-generated content at the generation stage, a technical requirement that
can be imposed on AI synthesis tools through regulation and that would
provide a detection signal that cannot be removed after the fact. The Coalition for Content Provenance and
Authenticity is developing the technical standards for content
watermarking that regulation could mandate, and accelerating the adoption of
these standards is one of the most tractable near-term interventions
available for reducing AI electoral disinformation impact.
About the Author
Stuart Kerr is a technology correspondent at LiveAIWire, covering
artificial intelligence, digital innovation, and the social impact of
emerging technologies. Follow LiveAIWire for daily analysis at liveaiwire.com.