Sky
News published an investigation in early 2025 documenting over 1,400 reported
cases of AI-facilitated abuse in the UK in the preceding twelve months, a
figure researchers described as likely representing a small fraction of
actual incidents given significant under-reporting. The cases spanned a range
of harms: non-consensual intimate imagery generated using face-swap AI tools,
voice-cloned audio used to extort victims, AI-generated child sexual abuse
material, and deepfake videos created to harass and silence women in public
life. What united them was the inadequacy of the legal and regulatory
framework available to address them, and the speed with which the technology
enabling them was outpacing the institutional response.
The UK government’s response to AI-facilitated abuse has been
fragmented and slow relative to the scale of documented harm. The Online
Safety Act 2023 included provisions on non-consensual intimate imagery,
making the sharing of such content a criminal offence and requiring platforms
to remove it promptly. These provisions represented genuine progress, but
they addressed sharing rather than generation, and they predated the
democratisation of AI tools capable of generating convincing synthetic
intimate imagery from a small number of publicly available photographs. The
gap between the legal framework and the technological reality it is supposed
to govern is widening faster than the legislative process can close
it.
Non-Consensual Intimate Imagery at Scale
The creation of AI-generated non-consensual intimate imagery,
sometimes called deepfake pornography, is the AI abuse category that has
attracted the most attention and generated the most measurable harm. AI tools
capable of generating convincing intimate imagery from a face photograph are
freely available, require no technical expertise to use, and are being
deployed at scale against women, girls, and LGBTQ+ individuals. The victims
include not only public figures but ordinary people, primarily women, whose
images have been scraped from social media or supplied by people they
know.
The Revenge Porn Helpline in the UK, now the Revenge Porn Helpline
and AI Safety, reported a 400 percent increase in cases involving
AI-generated imagery between 2022 and 2024. The psychological harm to victims
is severe and well-documented: anxiety, depression, career damage, social
withdrawal, and in the worst cases suicidal ideation. The barriers to justice
for victims are substantial: identifying perpetrators of AI-generated abuse
is technically challenging, legal aid for civil action is largely
unavailable, and police forces have variable capacity and motivation to
investigate these crimes. The Home
Office established an AI and Online Harms task force in 2024, but
concrete legislative progress from this body has been
limited.
Voice Cloning and Financial Fraud
AI voice cloning has enabled a significant expansion of
impersonation-based financial fraud. Tools that can reproduce a person’s
voice from a few seconds of audio are commercially available, and criminals
are using them to impersonate family members in distress calls designed to
extract emergency payments, to clone executive voices for authorisation fraud
in business contexts, and to create synthetic recordings used to manipulate
targets in longer-running confidence fraud schemes. UK Finance reported that
authorisation fraud enabled by AI voice impersonation grew by over 30 percent
in 2024, with significant sums lost by individuals and businesses despite
bank fraud prevention improvements. The challenge for law enforcement is that
the technical sophistication required to prosecute these cases exceeds the
capability of most forces, and the perpetrators frequently operate from
outside UK jurisdiction.
Child Protection and the Generation Challenge
AI-generated child sexual abuse material represents the most
serious category of AI-facilitated harm from a child protection perspective.
The Internet Watch Foundation documented a significant increase in
AI-generated child sexual abuse material on the open internet in 2024, and
noted that the quality and realism of this material had improved to the point
where it was increasingly difficult to distinguish from recorded abuse. The
legal position in the UK, where generating synthetic child sexual abuse
imagery is a criminal offence under the Protection of Children Act as
interpreted by the courts, is clearer than in some other jurisdictions, but
enforcement is severely limited by the volume of material and the technical
challenges of detection. The Internet Watch
Foundation has called for mandatory AI safety testing requirements
for image generation models, specifically including testing for their
capacity to generate prohibited imagery.
What This Means for You
The AI abuse cases documented by Sky News and other investigators
represent a systemic failure of governance that has direct consequences for
real people, disproportionately women and girls, who are targeted with
AI-facilitated harassment and abuse in ways that existing law cannot
adequately address. The practical steps available to potential victims are
limited: reducing public availability of high-quality images, using platform
tools to limit scraping, and documenting abuse thoroughly when it occurs to
support any subsequent legal action. At the systemic level, the reforms
needed include a specific offence of creating non-consensual intimate
imagery, mandatory safety testing requirements for AI image and voice
synthesis tools, and adequately resourced police and prosecution capacity for
AI-facilitated abuse cases. The international governance dimension of AI
abuse is also significant. Tools enabling AI-facilitated abuse are developed
and distributed globally, hosted on servers across multiple jurisdictions,
and used by perpetrators who may be in different countries from their
victims. Effective enforcement therefore requires international cooperation
that is currently developing slowly through bodies including Interpol and the
European Cybercrime Centre. The UK’s international engagement on AI abuse
governance, through the Council of Europe’s Budapest Convention on Cybercrime
and bilateral law enforcement cooperation agreements, provides a framework
but not yet the operational capacity to address AI-facilitated abuse cases
with the speed and consistency that victims require. Sustained investment in
international law enforcement cooperation specifically focused on
AI-facilitated harms is a priority that the UK’s technology diplomacy should
pursue more actively than it currently does. For related analysis of AI harms
and regulation, see our coverage of AI-generated
deepfakes and AI
in democratic processes.
The technology industry’s response to AI abuse cases has been
inadequate relative to the scale of documented harm. Platform companies have
deployed AI detection tools for prohibited content, but the arms race between
detection and generation consistently favours generation as tools become more
accessible and more capable. The voluntary commitments made by major AI image
and voice synthesis providers to implement safety measures, including
hash-matching against known non-consensual intimate imagery databases and
watermarking of synthetic content, have not been implemented consistently or
comprehensively. The gap between voluntary commitments and operational
reality has been documented by researchers at the Stanford
Internet Observatory and others, who have consistently found that
safety measures announced by AI companies are less effective in practice than
in press releases. Mandatory requirements, backed by meaningful enforcement
and adequate penalties, are the only intervention with a realistic chance of
closing this gap at the pace that the rate of AI-facilitated harm
demands.
The speed at which AI tools enable
harm is outpacing the speed at which regulatory and law enforcement responses
can be developed and implemented. Closing this gap requires pre-emptive
regulation of AI capabilities with clear harm potential, rather than the
reactive approach that has characterised UK technology policy to date, where
legislation follows documented harm rather than anticipating it.
About the Author
Stuart Kerr is a technology correspondent at LiveAIWire, covering
artificial intelligence, digital innovation, and the social impact of
emerging technologies. Follow LiveAIWire for daily analysis at liveaiwire.com.