ChatGPT and critical thinking do different things, even when both make a student’s assignment better. In a Bocconi University experiment involving 1,053 first-year students, access to the chatbot improved the quality of written work. Training in causal reasoning, meanwhile, helped students develop ideas that were more distinctive. The finding is particularly important for schools and employers that increasingly judge competence by the finished document rather than the thought behind it.
The research does not say that ChatGPT prevents original thinking. It found that access to AI did not drive the same diversity of ideas as the reasoning training, while students who received both could benefit from both approaches. The distinction is easy to miss when a neat, fluent answer receives a better mark. Good presentation can be valuable without being the same as independent judgement.
ChatGPT and critical thinking faced the same assignment
Researchers at Bocconi University in Milan worked with OpenAI and other collaborators on a randomised controlled trial. The students were studying subjects including economics, finance and management. They had to develop recommendations to increase awareness and use of the university’s merchandise among alumni. That made the assignment a concrete business problem, rather than a test of abstract philosophical reasoning.
Classes were allocated to four conditions. Some students were given access to ChatGPT, some received training in causal reasoning, some had both, and a control group had neither. Causal reasoning means identifying why an action might lead to a result, what assumptions the argument depends on and what could happen if those assumptions fail. It is a way of making reasoning explicit rather than just offering a plausible answer.
Human graders scored submissions using a five-point scale. The university reported that ChatGPT access lifted scores by around 0.86 points relative to an estimated control-group score of 2.09. The chatbot group produced more coherent proposals and work that looked closer to expert recommendations. Those are results from a defined assignment and marking system, not evidence that students became professional marketing experts.
The trial also examined originality separately from the main quality score. Students trained to analyse causes and consequences produced ideas that were more different from those offered by their peers. That mattered because a recommendation can meet an assessor’s expectations while failing to offer anything genuinely new. The experiment put a sharper distinction between sounding convincing and exploring an unusual but defensible solution.
The uncomfortable lesson about polished answers
AI can make a novice’s work appear much more experienced. It can organise an argument, supply possible options and improve the flow of a response. For a reader assessing the finished work, those improvements are real. A clearer document is easier to use, and an additional relevant idea may make the proposal stronger. The mistake would be assuming that quality on the page measures every skill used to produce it.
Suppose two people submit equally tidy plans. One has examined the cause of the problem, questioned the obvious options and checked what evidence would change the recommendation. The other has accepted the first plausible plan from a chatbot and made cosmetic edits. The finished documents might look similar, but the second person may struggle when circumstances change or a customer challenges the proposal.
The experiment did not observe that imaginary pair, and it should not be stretched into a claim that chatbot users are unthinking. Students in the AI condition still had to choose prompts and decide what to submit. The research instead shows why a single score for a finished assignment can miss different underlying capabilities. Where independent thought matters, assessment needs to look for it directly.
This concern is not confined to classrooms. Managers can receive remarkably professional reports from inexperienced staff, while finding it harder to know whether the author understands the calculations, risks or dependencies. A polished recommendation is helpful, but someone must still answer questions about where it might fail. That is especially important when decisions affect customers, money, safety or the reputation of an organisation.
Why the reasoning exercise changed the ideas
The training asked students to trace cause and effect and consider how an action would produce a result. That can expose hidden assumptions: perhaps a campaign would reach the wrong audience, or an apparent sales problem is really a problem of availability. Once those assumptions become visible, the obvious solution is no longer the only one. The student has a reason to explore a different route.
In an account written by the researchers for CEPR’s VoxEU, the authors explain the design of the trial and why reasoning from first principles generated more unusual proposals. Their account helps separate the intervention from an ordinary instruction to be creative. Students were taught a specific reasoning method, not merely encouraged to write longer answers or use more imaginative language.
The distinction matters in workplaces where expertise involves diagnosis as much as delivery. A template can produce a respectable recommendation for an ordinary case. A professional becomes particularly valuable when the case is unusual and the template no longer fits. Training people to identify assumptions and test alternatives gives them a skill they can use both with and without an AI assistant.
It also offers a more useful way of thinking about education policy. Banning chatbots from every assignment might prevent some forms of copying, but it would also remove a tool that can improve writing and organisation. Allowing unrestricted chatbot use without changing assessment may reward fluency while overlooking reasoning. The research points towards tasks that deliberately examine both the final answer and how its author reached it.
What students can do differently today
A practical way to preserve independent judgement is to outline the problem before opening the chatbot. Write down what outcome is needed, what is already known and which assumptions need testing. Then ask the tool to challenge that plan, show alternatives and identify missing evidence. The person remains responsible for deciding which criticisms are valid and which suggestions are irrelevant or invented.
Another useful habit is to separate idea generation from editing. First produce several approaches unaided or with a group. Ask why each might work and where it could fail. Only then use AI to help organise the options, improve the wording or examine overlooked counterarguments. This does not guarantee originality, but it prevents the first fluent response from automatically defining the boundaries of the task.
Students can also practise explaining a final recommendation without looking at the generated text. If they cannot say why an option was selected, what evidence supports it and which conditions would change their mind, the assignment may be teaching less than the presentation suggests. LiveAIWire has previously explored the risk that AI assistance can reduce persistence on independent tasks, a related concern that should not be mistaken for the result of this particular experiment.
For teachers, assessment can include oral explanation, revision after an unexpected constraint or a short account of rejected alternatives. Such methods can reveal reasoning that a finished essay alone cannot show. They also make it easier to reward a student who develops a defensible original approach even if the final prose is less polished than a machine-assisted version.
What an employer should read between the lines
The same principle applies to recruitment and promotion. A beautifully presented work sample may show that someone can use useful modern tools; that is a legitimate skill. It is not necessarily a test of their ability to diagnose a difficult problem, notice a false premise or respond to conflicting evidence. Interviews and practical exercises should ask questions the candidate cannot answer by simply repeating the original document.
Managers can use AI-produced drafts effectively while demanding clear ownership of claims. Whoever submits a recommendation should be able to identify its sources, distinguish estimates from known facts and explain trade-offs. LiveAIWire’s examination of overconfidence when people trust AI answers is a reminder that fluency can encourage unwarranted certainty. A professional response includes knowing what has not been established.
It is equally important not to make every person reinvent routine work. If software makes an established process faster, spending hours on the old method may offer no educational value. The challenge is choosing which skills deserve deliberate practice. Judgement, fact-checking and explaining causes remain useful precisely because a quicker first draft does not settle the decision.
The limits of this educational experiment
The trial involved students in one university and a specific business assignment. The results may look different in mathematics, creative writing, law, engineering or early school education. It measured the quality and diversity of submissions under defined conditions, not the students’ performance years later or their ability to transfer the reasoning skill to every kind of work.
The study also involved a collaboration with OpenAI, whose product was part of the experiment. That does not invalidate the design, but it makes transparent descriptions of the methods and independently replicable results important. The university and researchers describe the findings in considerable detail; broader claims about the entire education system still require further evidence.
An experiment can show that an intervention worked in a particular setting without proving that one universal rule should follow. ChatGPT improved some measured aspects of work; reasoning training improved other aspects. The idea that one should always replace the other is not supported. The most defensible response is to develop ways of testing both skills in circumstances where they matter.
Better work is not the same as better judgement
A previous LiveAIWire investigation considered why confident chatbot errors can survive repeated questioning. That familiar problem helps explain why independent thinking remains necessary after a document looks complete. An elegant argument still needs correct premises. A distinctive idea still needs evidence. And a good mark for one assignment does not prove the author can adapt when the world refuses to behave like the example.
The Bocconi experiment gives schools and employers a useful choice. They can reward the speed and clarity that modern tools provide, while also making space for reasoning that finds unfamiliar answers. The two do not need to compete. A student who can think independently and use AI to communicate that thinking may be better equipped than one trained only to produce the smoothest possible page.
The harder question is what education systems will bother to measure. If all the incentives favour the tidy final answer, students will naturally seek the easiest route to one. If assessments also ask for the thinking behind the recommendation, good use of AI can become part of a stronger education instead of a replacement for it.
About the Author
Stuart Kerr is Technology Correspondent at LiveAIWire, covering artificial intelligence, cybersecurity and the social impact of emerging technology. LiveAIWire is an independent, human-led technology publication using AI-assisted research, editorial production and original AI-assisted editorial illustrations under his direction.