Keeping human answers separate from AI improved joint decisions in an analysis of ten existing datasets. Researchers compared ordinary AI advice with a process in which a person and a machine answered independently, and a second person settled disagreements. Their revised April 2026 preprint found higher accuracy across all ten datasets. It tested combinations of recorded judgements, rather than deploying the process in a new workplace trial.
The finding concerns a small but consequential design choice: whether a person forms a judgement before seeing the machine’s answer. Asking somebody to check a recommendation sounds like meaningful oversight. But that person may already be responding to the recommendation instead of providing a genuinely separate assessment of the evidence.
Human answers have value before the AI speaks
The proposed process, called a hybrid confirmation tree, accepts agreement between the first human and AI. If they disagree, another unaided human provides the deciding judgement. The researchers analysed more than 41,000 decisions by 1,229 people across 3,220 cases, reporting a pooled accuracy gain of 4.45 percentage points over the advice-first approach. Individual dataset gains varied.
The essential feature is not simply adding another person. It is preserving the independence of the inputs being combined. If everybody reads the same suggested answer first, agreement can tell us less about whether separate assessments reached the same conclusion. The workflow should make clear which judgements were formed independently and which were influenced by an earlier response.
Consider a hypothetical review of a customer complaint. A staff member and an AI system could each assess whether the supplied evidence supports the complaint, before comparing conclusions. A disagreement could then trigger another review. That example illustrates the structure; it is not a claim that the study tested complaint handling or proved the method suitable for that business.
Checking an answer is a different task
Once an AI answer is visible, the human’s job changes. They must decide whether to accept, reject or alter something already presented. That requires both subject knowledge and an ability to evaluate the particular recommendation. A polished explanation may make the decision easier to follow without making its underlying conclusion correct.
Earlier research offers a useful comparison. A 2021 experiment involving 199 participants tested designs intended to make people think more actively before relying on AI. These interventions reduced overreliance relative to simpler explanation-based designs. However, the most effective designs received less favourable subjective ratings, and benefits differed with participants’ motivation to engage in effortful thinking.
That result exposes a product-design tension. An interface can feel effortless precisely because it removes steps that make a person pause and evaluate. Users’ preference for a smoother experience is relevant, but it cannot establish that the smoother experience produces better decisions. Satisfaction and accuracy need to be measured separately.
What this means when you use AI at work
For an individual, a modest practical experiment would be to record an initial answer and its reasons before asking AI for its view. The purpose is not to defend that first answer at all costs. It is to preserve a record of what you thought before being influenced by another response, making a later comparison more informative.
For example, someone reviewing a document could first write down the question they think remains unresolved and the passage supporting that concern. They could then compare the assistant’s assessment with that record. If the two differ, the discrepancy becomes a specific issue to investigate rather than a vague impression that the AI sounds more confident.
This exercise is not the full process tested in the paper. It lacks the independent second person used to resolve disagreement and should not be advertised as delivering the same accuracy gain. Its value as a proposed working habit is that it makes the sequence of judgement visible. Whether it helps a particular task should be assessed rather than assumed.
The distinction also clarifies why “a human approved it” is incomplete information. Was the person given the source material? Did they have time to examine it? Could they disagree without penalty? Did they reach an initial view independently? A final click can record a decision without revealing how much meaningful evaluation preceded it.
Better than a person is not necessarily best
A separate MIT-led meta-analysis of 106 experiments found that human–AI combinations, on average, performed worse than the better of the human or AI alone. Results differed by task. This does not mean collaboration is pointless; it means improvement over an unaided person is a weaker test than improvement over the best available performer.
LiveAIWire’s article on the limits of human–AI teamwork explains that comparison. The newer independent-judgement proposal addresses a more specific question: whether changing the way answers are combined can improve on the familiar advice-first arrangement.
For a manager, this suggests a more informative evaluation than counting licences or completed prompts. Compare representative decisions under the current process, an AI-assisted process and any proposed alternative. Decide in advance what counts as success. Average accuracy, serious mistakes, turnaround time and cost can point in different directions.
A process that catches more errors might require more staff time. A process that is faster might produce a small number of particularly costly mistakes. Neither trade-off can be settled by quoting a model’s benchmark score. The object being evaluated is the complete decision process, including the people and the conditions under which they work.
Independence does not guarantee correctness
Separate assessments can still share weaknesses. Two reviewers may rely on the same incomplete document or misunderstand the same ambiguous instruction. A machine and a person may both lack the decisive piece of evidence. Agreement should therefore be treated as a feature of a procedure, not as proof that the world matches the answer.
This matters when designing an escalation rule. Some decisions may warrant further examination even if initial assessments agree, particularly when the evidence is missing or the consequence of error is serious. The sensible rule depends on the task. The research does not establish a universal policy for accepting every human–AI agreement automatically.
Nor is every workplace question a classification problem. Deciding whether a statement is supported by evidence differs from choosing an acceptable commercial risk or balancing competing priorities. An accuracy measure requires a defensible way to distinguish right from wrong. Many consequential decisions also involve values, responsibilities and choices that cannot be resolved by majority agreement.
The paper’s datasets included several different judgement tasks, but breadth within a research collection is not the same as universal applicability. A prospective trial would still be needed to establish whether a particular implementation works under real staffing constraints, with changing cases and people who know the system will affect actual outcomes.
The extra reviewer is a real resource
One easily overlooked part of the proposal is the second human. Their contribution cannot be treated as free. A practical assessment should record how frequently disagreement occurs, how long another review takes and whether qualified people are available. An accuracy improvement can be useful while still being expensive to obtain.
There is also a question of how that reviewer receives the case. If the interface prominently displays the original AI answer, the intended independence may be weakened. A team testing this approach would need to specify what each person sees, when they see it and how their decision is recorded. Otherwise, different employees may be following different procedures under one label.
Those implementation questions do not invalidate the research. They explain the distance between an encouraging result and a dependable service. A method can perform well when combining recorded answers yet encounter new difficulties when people have deadlines, incomplete information and competing responsibilities.
Design oversight around a meaningful contribution
The strongest implication is that human oversight should be designed, not merely declared. A reviewer needs a clear task, access to relevant evidence and a useful point at which to contribute. If the person is present only after the machine has defined the problem and supplied the conclusion, an opportunity for independent judgement may already have been lost.
This is an editorial reading of the evidence, not a claim that every organisation should adopt the confirmation tree. Some tasks may benefit from independent initial assessments; others may need discussion, specialist escalation or a different division of labour. The appropriate process should follow the task and the cost of mistakes.
What the study contributes is a testable alternative to a familiar assumption. Rather than asking whether a person remains somewhere in the loop, ask what independent information that person adds. The answer may depend less on their final approval than on whether they had the chance to think before the AI spoke.
About the Author
Stuart Kerr is Technology Correspondent at LiveAIWire, covering artificial intelligence, cybersecurity and the social impact of emerging technology. LiveAIWire is an independent, human-led technology publication using AI-assisted research, editorial production and original AI-assisted editorial illustrations under his direction.