Meta
AI’s Discover feed, launched as a feature within the Meta AI assistant
interface in 2024, allows users to see AI conversations that other users have
chosen to make public. Within weeks of launch, privacy researchers and
journalists including those at 404 Media discovered that the feed was
surfacing conversations containing highly sensitive personal information:
users discussing their HIV status, pregnancy concerns, mental health crises,
relationship problems, immigration status, and financial distress, in many
cases apparently without full understanding that they had opted their
conversations into a publicly visible feed. The incident was not a data
breach in the traditional sense. The information was technically public
because users had opted in to sharing. But the design of the opt-in
mechanism, which many users clearly did not fully understand, created a
privacy harm that was real regardless of its technical
legality.
The Discover feed incident illustrates a pattern in how major
technology platforms approach privacy in AI products that deserves systematic
scrutiny rather than case-by-case reactive coverage. When AI assistants are
deeply integrated into the personal, emotional, and sensitive aspects of
users’ lives, the design of sharing and visibility mechanisms takes on a
significance that does not apply to less intimate product categories. A user
who does not fully read the terms before posting a restaurant review has made
a low-stakes decision. A user who does not fully understand that their
conversation about their mental health crisis is publicly visible has made a
decision whose consequences are categorically different, and the platform
that designed the mechanism bears a corresponding responsibility for making
that distinction clear.
The Design Problem
Privacy researchers who examined the Discover feed’s opt-in
mechanism found that the path to making conversations public was shorter and
more prominent than the path to ensuring they remained private. The default
framing presented sharing as a way to “contribute to the community”
in language that emphasised the social positive of sharing without adequately
foregrounding the privacy implications of making sensitive personal
disclosures publicly visible. Several users who were featured in the privacy
researchers’ coverage expressed surprise that their conversations had been
made public, reporting that they had not understood the sharing mechanism
they had agreed to. This pattern, where technically consensual sharing occurs
in ways that users do not meaningfully understand, is a well-documented dark
pattern in digital product design that consumer protection regulators have
increasingly recognised as a legitimate enforcement target.
The regulatory response to the Discover feed incident was
relatively swift by the standards of technology regulation. The Irish Data
Protection Commission, which is Meta’s lead data protection regulator in the
EU, opened an investigation into whether the Discover feed’s design met GDPR
requirements for informed consent. The UK’s Information Commissioner’s Office
published guidance reinforcing that consent to sharing sensitive personal
data must be specific, informed, and freely given, with specific emphasis on
the inadequacy of consent obtained through dark pattern design. Meta modified
the Discover feed’s opt-in mechanism following the controversy, though
privacy advocates argued the modifications did not fully address the
fundamental design problem.
The AI Assistant Privacy Problem
The Discover feed is a specific manifestation of a broader
challenge: AI assistants designed to be used for intimate personal
conversations create privacy risks that traditional communication platforms
do not, because the content of AI conversations tends to be more personal and
more sensitive than, for example, social media posts. When people use AI
assistants to process grief, navigate relationship difficulties, discuss
medical symptoms, or seek support for mental health challenges, they are
sharing information in a context that they typically understand as private.
The design of AI assistant products that treats this content as a resource
for social features or as training data without explicit, specific, and fully
informed consent violates the contextual integrity of the original
disclosure.
The Information
Commissioner’s Office has published specific guidance on AI and
data protection that addresses the contextual integrity principle,
emphasising that data shared for one purpose should not be repurposed for
another without consent that specifically addresses the new purpose. Applied
to AI assistant conversations, this principle suggests that using
conversational data for community features, training new models, or targeting
advertising requires consent that goes beyond the general data processing
consent buried in terms of service that most users accept without reading.
The gap between this principle and current practice at major AI platform
companies is substantial.
User Rights and Practical Protection
Under GDPR in the EU and the UK GDPR, users have a right to access
the data held about them, to rectify inaccurate data, to delete data under
certain circumstances, and to object to processing for specific purposes.
These rights apply to AI conversation data and are exercisable regardless of
what general terms of service say. Exercising these rights requires knowing
they exist and knowing how to assert them, which requires an engagement with
privacy settings and data subject rights that most users do not currently
have. Meta publishes a data subject rights tool that allows EU and UK users
to access, download, and delete their data, but awareness of this tool among
users affected by the Discover feed incident was evidently
low.
What This Means for You
If you use Meta AI or any AI assistant integrated into a social
media platform, reviewing the sharing and visibility settings for your
conversations is a practical step that takes a few minutes and could prevent
the kind of inadvertent disclosure that affected Discover feed users. More
broadly, treating AI assistant conversations with the same privacy awareness
you would apply to any digital communication, recognising that what you share
with an AI system is held by a commercial company with its own interests in
how that data is used, is an appropriate adaptation to the current AI privacy
landscape. The broader pattern that the Discover feed incident represents is
worth naming explicitly: AI products are systematically being designed to
encourage disclosure of personal information in ways that serve the commercial
interests of the platforms hosting them. Conversational AI that feels
intimate and supportive creates conditions in which users share information
that they would not share with a corporate entity if asked directly, and the
companies operating these systems have financial incentives to retain, use,
and in some cases surface this information in ways that users do not
anticipate. Designing AI products with privacy as a genuine design objective,
rather than as a compliance requirement to be minimally satisfied, requires
organisational cultures and governance structures that most current AI
platform companies have not yet developed. The Privacy
International campaign on AI and intimate data provides sustained
scrutiny of these practices and practical guidance on how individuals and
regulators can respond. For related analysis, see our coverage of Meta
AI and AI
and personal data privacy.
Reviewing privacy settings on every
AI platform you use, understanding what data is retained and for how long,
and exercising data subject rights to delete conversation history that you do
not want retained are practical steps that take limited time and provide
meaningful protection. The cultural shift toward treating AI assistant
conversations with the same privacy awareness applied to email and messaging
is the most important individual adaptation to an AI privacy landscape that
is more complex and less well-governed than most users currently appreciate.
The ICO’s guidance on
AI and personal data provides UK-specific rights information.
About the Author
Stuart Kerr is a technology correspondent at LiveAIWire, covering
artificial intelligence, digital innovation, and the social impact of
emerging technologies. Follow LiveAIWire for daily analysis at liveaiwire.com.