By Stuart Kerr, Technology Correspondent, LiveAIWire
Emotional AI can now detect subtle shifts in vocal tone, facial microexpressions, and written language that correlate with human emotional states, and this capability is being deployed at scale in customer service, healthcare, recruitment, and education. The technology is advancing faster than public awareness, faster than regulatory frameworks, and in some cases faster than the science underpinning it can justify.
Emotional AI, sometimes called affective computing, has been developing for decades as an academic discipline. What has changed is the commercial viability of the underlying technology. Advances in computer vision, natural language processing, and audio analysis have made it possible to build systems that infer emotional states from widely available data streams, including video calls, customer support recordings, and job interviews, at a cost that makes mass deployment practical for large organisations.
How Emotional AI Actually Works
Current emotional AI systems typically combine multiple modalities. Facial action coding systems analyse movement in dozens of facial muscle groups to classify emotional expressions. Voice analysis tools examine pitch, rhythm, and prosody for indicators of stress, confidence, or deception. Sentiment analysis models parse text for emotional content using large language model architectures trained on labelled datasets.
The accuracy of these systems is a matter of genuine scientific dispute. Research published in Psychological Science has challenged the foundational assumption that discrete emotional states map reliably onto facial expressions, a critique that goes to the heart of validity claims made by emotional AI vendors. Despite these limitations, the commercial market has expanded rapidly. The Federal Trade Commission in the United States has raised concerns about the marketing claims of some emotional AI vendors, noting the gap between advertising and scientific evidence.
Workplace and Recruitment Applications
The use of emotional AI in hiring has attracted significant controversy. Video interview platforms that analyse candidate facial expressions and vocal patterns to assess personality traits have been adopted by thousands of employers. Unilever was a high-profile early adopter of AI-driven video interviews; subsequent scrutiny raised questions about the validity and fairness of the underlying assessments, and the company eventually scaled back its use of the technology.
Researchers at the University of Cambridge and elsewhere have demonstrated that AI recruitment tools can exhibit systematic biases related to protected characteristics, not through intentional discrimination but through correlations embedded in training data, a pattern LiveAIWire has traced closely in our coverage of AI gender bias baked into hiring algorithms more broadly. The Illinois Artificial Intelligence Video Interview Act, passed in 2019, requires employers using AI to analyse video interviews to notify candidates and explain how the technology works.
Healthcare and Mental Health Applications
In healthcare, this technology shows genuine promise alongside genuine risk. Systems that detect early signs of depression, anxiety, or cognitive decline from speech patterns and behavioural data could enable earlier intervention and better outcomes. The challenge is that the same technology deployed without clinical oversight becomes a liability. Mental health apps that use AI to infer emotional states from user behaviour and respond with content recommendations are operating in a regulatory grey area in most jurisdictions.
Emotional AI in the Classroom
The education sector represents a significant deployment context that receives insufficient attention. Learning management platforms are beginning to incorporate engagement monitoring tools that analyse student facial expressions and attention patterns during online learning sessions. The use of these tools in educational contexts raises particular concerns because of the power differential between educational institutions and students, and because young people may not have the agency to meaningfully consent to or resist this monitoring.
In the EU, the General Data Protection Regulation provides protections for biometric and special category data, and the AI Act adds requirements for high-risk systems. The World Health Organization has published guidance on AI in healthcare settings that addresses emotional data specifically, calling for explicit consent requirements and independent validation of clinical claims before deployment. Adoption of this guidance remains voluntary and inconsistent across health systems.
What This Means for You
The most important practical implication of emotional AI is that you are almost certainly being assessed by systems you are unaware of. If you have participated in a video job interview, called a customer service line, or used a mental health or wellness app, your emotional signals may have been processed by AI without meaningful disclosure.
The commodification of emotional data is not a technical inevitability, it is a policy choice, and it can be reversed by policy choices that require meaningful consent, impose accuracy standards, and create liability for harm. This same asymmetry between what an algorithm knows about a person and what that person can see or contest runs through LiveAIWire’s coverage of AI insurance premiums calculated from data the person being priced never directly agreed to, and in our reporting on the AI shadow workforce, where human labour that shapes AI systems remains similarly invisible to the people affected by the resulting outputs.
About the Author
Stuart Kerr is Technology Correspondent at LiveAIWire, covering artificial intelligence, cybersecurity, and the social impact of emerging technology. He publishes daily at LiveAIWire.com.