In
2024, researchers at Google DeepMind demonstrated an AI system that could
automatically identify and generate proofs for previously unknown software
vulnerabilities in widely deployed code libraries. The same capability that
makes this system potentially invaluable for defenders makes it potentially
catastrophic in the hands of attackers. This is the fundamental paradox of AI
in cybersecurity: the technology that enables better defence simultaneously
enables more sophisticated and scalable attack, and the advantage in this
arms race does not obviously favour either side.
Cybersecurity has always been an adversarial domain, defined by
the continuous interaction between offensive and defensive capabilities. AI
is intensifying this dynamic dramatically. Attack automation, vulnerability
discovery, social engineering at scale, and malware development that adapts
to evade specific defences are all being enhanced by machine learning. At the
same time, AI is improving threat detection, anomaly identification, and
incident response in ways that are genuinely helping defenders. The question
is which side benefits more from the technology, and the honest answer is
that it depends on who is better resourced and more strategically coherent in
their application of it.
AI-Powered Attacks: The Threat Landscape
The most immediate AI-enabled attack enhancement is in social
engineering, specifically phishing and spear-phishing. Large language models
can generate highly personalised, contextually accurate phishing emails at
scale, removing the spelling errors and awkward phrasing that previously
helped people identify malicious messages. Voice synthesis AI enables vishing
attacks using cloned voices of trusted individuals. A 2024 incident in Hong
Kong in which employees of a financial firm transferred $25 million following
a video call with what they believed was their CFO, but was entirely
AI-generated, illustrated the scale of the risk. The cost of mounting a
sophisticated social engineering attack has fallen from requiring a skilled
team to requiring a credit card and access to commercially available AI
tools.
AI is also accelerating vulnerability discovery. Machine learning
models trained on code can identify classes of vulnerability that rule-based
static analysis tools miss, and they can scan codebases at a speed that makes
comprehensive analysis feasible for large software projects. Both legitimate
security researchers and malicious actors are using these capabilities, and
the democratisation of vulnerability discovery means that previously
inaccessible targets are becoming viable for less sophisticated attackers.
Malware that uses AI to adapt its behaviour, evading detection by
security tools trained on previous variants, is an emerging threat that the
security community is actively tracking. The National
Cyber Security Centre published a threat assessment in 2024
describing AI-enhanced malware as one of the highest-priority emerging
threats facing UK organisations, noting that the combination of AI
adaptability and the scale of existing criminal infrastructure creates a
significant uplift risk.
AI in Defence: Capabilities and Limitations
On the defensive side, AI is providing genuine improvements in
threat detection and response. Machine learning models analysing network
traffic can identify anomalous behaviour patterns that indicate intrusion or
compromise far more rapidly than human analysts reviewing logs. Security
operations centres using AI-assisted triage are processing alert volumes that
would be operationally impossible without automation; a typical large
enterprise generates millions of security events per day, of which only a
small fraction are genuine threats.
AI-powered endpoint detection and response systems from vendors
including CrowdStrike, SentinelOne, and Microsoft Defender use behavioural
analysis to identify malicious activity on devices without relying on
signature-based detection, which struggles against novel threats. These tools
have meaningfully improved detection rates for sophisticated attacks,
including the supply chain attacks that have characterised some of the most
significant breaches of the past decade.
The limitations of AI in defence are also significant. AI security
systems can be fooled through adversarial attacks that exploit the specific
characteristics of machine learning models. They can generate significant
false positive volumes that create alert fatigue for human analysts. And they
require substantial amounts of high-quality labelled data to train
effectively, which smaller organisations frequently lack. The benefits of AI
in cybersecurity accrue disproportionately to larger, better-resourced
organisations, widening the security gap between the enterprise and the small
business.
The Talent and Skills Dimension
The cybersecurity skills shortage, estimated by industry bodies
including ISC2 at over 4 million unfilled roles globally, is partly being
addressed by AI automation. Tasks that previously required experienced human
analysts, including initial threat triage, log correlation, and vulnerability
prioritisation, are increasingly automated, allowing existing talent to focus
on higher-complexity work. This is a genuine benefit, though it also means
that the entry-level roles through which the next generation of security
professionals develop their skills are disappearing.
What This Means for You
The cybersecurity implications of AI affect individuals as well as
organisations. AI-generated phishing emails are more convincing than their
predecessors. AI-powered voice cloning enables fraud that exploits trusted
relationships. The social engineering attacks enabled by AI are designed to
defeat the awareness training that organisations have invested in, because
they look like the legitimate communications that training teaches employees
to recognise as safe.
Practical defensive measures remain important: scepticism about
unexpected requests for action or information, verification of high-stakes
requests through independent channels, and basic cyber hygiene including
strong unique passwords and multi-factor authentication. These measures are
not sufficient against sophisticated nation-state attackers, but they address
the vast majority of threats that most individuals and organisations actually
face. For related coverage of AI in fraud and financial crime, see our
analysis of AI
in fintech fraud and AI
in digital criminal networks. Supply chain security represents one
of the most significant AI cybersecurity challenges that organisations face,
and one that is least amenable to individual organisational responses. The
SolarWinds attack, in which malicious code was inserted into widely deployed
network management software, affected thousands of organisations including US
government agencies. AI is being used to monitor software supply chains for
anomalous code insertions and to analyse the behaviour of third-party
software components for malicious activity. However, the complexity of modern
software supply chains, which can involve hundreds of third-party components
with their own dependencies, means that comprehensive AI monitoring of supply
chain integrity remains an unsolved problem. The National Cyber Security
Centre has published specific guidance on AI in supply chain
security that acknowledges both the potential of AI monitoring tools and
their current limitations in addressing the full scope of the supply chain
attack surface.
is a prerequisite for navigating it. The organisations and
individuals who understand that AI has changed both the threat landscape and
the defensive toolkit available to address it are better positioned to make
the investments and behaviour changes that actually improve their security.
Those who treat cybersecurity as a solved problem or an IT department concern
rather than a strategic one will increasingly find that the AI-enabled
attacks targeting them are more sophisticated than the defences they have
invested in. The gap between AI-enabled attack capability and AI-enabled
defence capability is not fixed; it depends on investment decisions and
organisational cultures that leaders at every level of an organisation have a
role in shaping. NCSC
guidance for organisations of all sizes provides a practical
starting point for assessing current posture against AI-enhanced threats is a
prerequisite for navigating it.
About the Author
Stuart Kerr is a technology correspondent at LiveAIWire, covering
artificial intelligence, digital innovation, and the social impact of
emerging technologies. Follow LiveAIWire for daily analysis at liveaiwire.com.