AI hallucination detection may already be hiding inside some of the models most likely to answer you with complete confidence. In a new preprint, researchers found that the internal routing activity of two open mixture-of-experts language models could expose false answers, sometimes down to the individual token, even though the models never displayed that warning to the person using them.
The system, called InnerExpert, reached up to 0.91 answer-level AUROC and 0.76 token-level AUROC across five question-answering datasets, according to the paper submitted to arXiv on 18 August 2026. It required one generation pass rather than repeatedly asking the model for alternative answers. That could make a future warning layer faster than many current checking methods.
But the viral interpretation needs one immediate correction. This is not evidence that an AI consciously knows it is lying, feels doubt or is choosing to deceive you. The researchers trained a separate, lightweight classifier to read patterns in the model’s numerical activity. The model itself does not understand or announce the verdict. A better analogy is a dashboard sensor that detects an engine problem before the driver sees a warning light.
AI Hallucination Detection: What This Means for You
For an ordinary chatbot user, nothing changes today. InnerExpert is a research prototype, not a button inside ChatGPT, Gemini or Claude. It was tested on two open-weight mixture-of-experts models, not the closed consumer systems most people use. You should still verify important claims, open cited sources and avoid treating a fluent answer as proof.
The longer-term possibility is much more interesting. If a model provider can recognise a risky sentence while it is being generated, an interface could shade uncertain words, pause before making a consequential claim, retrieve fresh evidence or send the answer to a human reviewer. An autonomous agent could be blocked from acting when its internal warning score crosses a calibrated threshold.
That last use matters because a false sentence is annoying, while a false sentence connected to tools can become an event. LiveAIWire’s examination of what happens when an AI agent makes a costly mistake showed why confirmation gates, permissions and audit logs become essential once software can spend money or change records. An internal risk signal could become another control, but not a replacement for those safeguards.
Why These Models Have More Signals to Read
A conventional dense language model sends every token through the same large set of neural components. A mixture-of-experts model, or MoE, works differently. Each layer contains many smaller feed-forward networks called experts, while a router selects only a few of them for each token. This sparse design can increase total model capacity without activating every parameter for every word.
That routing decision leaves a trail. The router assigns scores to experts, different experts produce representations, and the pattern of expert use changes from token to token. InnerExpert records features such as router entropy, expert disagreement, similarity between expert outputs and expert usage patterns. It combines those MoE-specific measurements with familiar transformer signals, including hidden states, attention statistics and output entropy.
Imagine asking a panel a factual question. Agreement does not prove the panel is right, and disagreement does not prove it is wrong. Yet the pattern of who answered, how strongly they were selected and how their responses diverged can help a trained observer identify risky moments. InnerExpert is that observer, operating on numbers rather than spoken opinions.
The Warning Can Point to a Particular Word
Most hallucination detectors return one score for an entire answer or sentence. That is useful, but crude. A response can contain ten correct facts and one invented date. Rejecting the entire answer hides where the problem occurred, while accepting it leaves the fabricated detail looking as trustworthy as everything around it.
InnerExpert builds a compact feature vector for every generated token and asks its classifier whether that token belongs to a hallucinated span. In the strongest reported token-level results, it reached 0.762 AUROC on OLMoE-1B-7B and 0.753 on Gemma 4 26B-A4B. The gains over the best individual signal were 0.12 and 0.20 respectively.
Those scores are promising rather than conclusive. A token is not always a complete fact, and the boundary of a false claim can be ambiguous. Even so, localisation creates options that an answer-wide score cannot offer. A system could flag only the unsupported date, regenerate one clause or attach a source request to the exact claim that triggered the warning.
How the Researchers Trained the Detector
The team, Joao Fonseca, Rodrigo Rodrigues and Paolo Romano of INESC-ID and Instituto Superior Técnico in Portugal, used questions from the RealTimeQA dynamic benchmark dated from January 2024 to December 2025 for training. For each question, the host model produced one answer with reference evidence and another without it, simulating the difference between retrieval-assisted and closed-book answering.
GLM-5.1 then acted as a judge. It labelled the answer and identified the exact spans it considered hallucinated, which the researchers mapped back to tokens. This avoided the cost of manually labelling the entire training set, but it also transferred some of the judge model’s mistakes and preferences into the detector.
The test set deliberately moved forward in time. It included RealTimeQA questions from January to June 2026, plus 200 examples each from SQuAD, TruthfulQA, NQ-Open and FreshQA. All tests were in English, generations were produced greedily and answers were limited to 65 tokens. That is broader than one dataset, but much narrower than the variety and length of real conversations.
What the 0.91 Result Really Measures
The headline result is 0.91 answer-level AUROC, not 91 per cent accuracy. AUROC measures how well a detector ranks a randomly selected hallucinated answer above a randomly selected non-hallucinated one across every possible decision threshold. A score of 0.5 is no better than random ranking, while 1.0 is perfect separation.
The strongest average answer-level configuration reached 0.912 on Gemma and 0.882 on OLMoE. That shows strong discrimination in these experiments, but it does not tell a product team how many warnings will be false after choosing a threshold. The practical result depends on the mix of questions, the cost of missing an error and how many false alarms users will tolerate.
This distinction is similar to the care required when reporting other striking AI numbers. In LiveAIWire’s analysis of Claude’s 67.2 per cent mathematics result, the percentage was a lower bound on a related theorem, not the proportion of the Riemann Hypothesis solved. A compelling number becomes misinformation when its metric is silently changed.
There Is No Single Hallucination Neuron
The paper does not reveal one universal internal alarm. No individual feature performed consistently well across both architectures. On OLMoE, the best single MoE feature achieved 0.693 answer-level AUROC, while the full XGBoost detector reached 0.882. On Gemma, expert usage entropy was much stronger at 0.884, but even there the combined detector improved the result to 0.912.
Some signals even changed character between models. Router entropy was negatively correlated with hallucination on Gemma, expert similarity was close to random on OLMoE and feature importance shifted with architecture. The useful warning emerged from combining weak and architecture-dependent clues, not from discovering a stable truth meter that can be copied into every AI.
Earlier peer-reviewed research had already shown that hidden activations can contain factuality information. A 2023 EMNLP Findings paper, provocatively titled The Internal State of an LLM Knows When It’s Lying, trained classifiers on hidden states and reported 71 to 83 per cent accuracy on balanced true-and-false sentence tests. InnerExpert extends that broad idea into MoE routing signals and token-level localisation.
One Pass Is Faster, but It Is Not Free
Many uncertainty methods sample several answers and compare them. The influential semantic entropy research published in Nature, for example, groups different phrasings by meaning and looks for instability across repeated generations. That approach can detect confabulation without task-specific labels, but its experiments used multiple generations.
InnerExpert gathers its features during one generated answer. In the paper’s tests, however, recording and processing the internal states still increased time per 100 tokens from 1.146 to 3.498 seconds on OLMoE and from 3.825 to 9.384 seconds on Gemma. That is roughly 2.5 to three times the uninstrumented run, even before a production team optimises the prototype.
Peak GPU memory rose by about 3 per cent, much less than approaches that load a second model. The trade-off is therefore plausible for high-value work, but less obvious for every casual chat. A bank reviewing a transaction or a hospital checking a generated summary may accept extra latency that a consumer autocomplete product would not.
The Labels Are the Most Important Weakness
A detector learns what its labels define as an error. Here, most labels came from another language model rather than a panel of human experts. The researchers manually checked a stratified sample of 200 answers using one blinded annotator. GLM-5.1 disagreed with the human on 27, giving 86.5 per cent accuracy in that small validation.
That is substantially better than the paper’s simple lexical and semantic heuristic, which scored 59.5 per cent, but it is not perfect ground truth. There was only one human annotator, the sample was small and disagreements about partial truth or span boundaries were not independently adjudicated. A detector trained on noisy labels can learn to reproduce the judge’s blind spots.
The authors acknowledge other limits. Only two MoE models were tested. The best classifier was not stable across them. The experiments covered English question answering, not coding, legal drafting, long reports or multilingual dialogue. XGBoost’s F1 score was particularly sensitive to threshold choice on Gemma, which reinforces the gap between high AUROC and a dependable product setting.
Detection Does Not Make the Answer True
Uncertainty methods are best at finding errors associated with unstable internal behaviour. A model can be confidently and consistently wrong because its training data contain a misconception, its retrieved document is false or its reasoning follows the same invalid step every time. Low uncertainty is not a certificate of truth.
The Nature semantic entropy team makes the same distinction: its method targets confabulations that vary across samples, not systematic falsehoods. InnerExpert’s temporal test data reduce some contamination risks, but the paper notes that a model which faithfully copies bad context can still appear internally certain. Source quality and external verification remain necessary.
That is why LiveAIWire’s guide to hallucination rates and practical AI trust recommends checking citations and matching oversight to the consequence of an error. Internal monitoring could tell you where to look. It cannot establish that a medical claim, court case or financial figure exists in the outside world.
Why Better Prompting Is Not the Same Fix
Users can reduce avoidable errors by supplying sources, asking the model to identify uncertainty and inviting it to challenge false assumptions. Those habits are useful, but they work through the input and context. InnerExpert watches the calculation happening underneath the generated words.
The two approaches could complement each other. LiveAIWire’s review of what reliable prompt engineering actually changes found that structure, examples and explicit critical framing matter more than magic phrases. A future system could pair good context with an internal detector, then retrieve evidence or ask a clarifying question when the detector sees risk.
Yet the warning should never be designed as a decorative confidence badge. Users tend to over-trust precise percentages, and a reassuring green score may suppress healthy scepticism. Product teams would need calibration data from the actual task, clear language about what the detector measures and an escalation path when the cost of being wrong is high.
The Model Has a Signal, Not Self-Knowledge
It is tempting to say the AI knew the answer was false and kept quiet. That phrasing makes the result memorable, but it smuggles in a claim the experiment did not test. InnerExpert reads correlations in activations through an externally trained classifier. It does not demonstrate awareness, intention, honesty or an inner narrator withholding a confession.
The finding is still significant without anthropomorphism. Modern language models can contain information about the reliability of their own output that their normal user interface discards. If that information survives independent testing across more models, languages and tasks, the next leap in trust may come partly from exposing useful signals that computation already produces.
The researchers identify the next tests themselves: transfer the detector between models without retraining, calibrate thresholds, expand to more tasks and languages, and use the signal for online monitoring or controlled generation. Independent replication and peer review also matter because the current report is a newly submitted preprint.
Your AI may one day underline the exact phrase it suspects before you ever have to discover the error. For now, the hidden warning exists only in a limited laboratory setting, and it still needs a separate system to interpret it. The model is not secretly confessing. Researchers have simply found a promising place to listen.
About the Author
Stuart Kerr is Technology Correspondent at LiveAIWire, covering artificial intelligence, cybersecurity, and the social impact of emerging technology. He publishes daily at LiveAIWire.com.