Robot instructions often sound simple: ask someone to bring a cup of coffee to your desk and they will probably avoid knocking over the laptop. Ask a robot, and the obvious parts of the instruction may not be obvious at all. MIT researchers have developed a method that helps robots interpret vague instructions and learn which objects in a scene deserve attention. In their controlled demonstrations, a robotic arm moved cups while avoiding a computer, even when people had not described every detail of the safe route.
The work addresses a stubborn obstacle to useful domestic and workplace robots: humans routinely leave important conditions unstated. We might say to place a glass nearby, move a chair out of the way or tidy a table, assuming the other person understands that fragile objects should not be damaged. A machine has to determine what the instruction means, what matters in its surroundings and how to turn that understanding into an action.
The robot vague instructions research reported by MIT in June 2026 offers a partial solution. It combines demonstrations from people with language-model assistance to clarify a task and ignore irrelevant details. The published tests are encouraging, but they do not mean an ordinary robot can now follow any household request without supervision.
Why robot instructions leave out the hardest part
People rely heavily on shared experience. If you ask a colleague to put a folder on your desk, you do not normally specify how to steer their arm past the lamp or where to place their fingers. The colleague understands how objects behave and recognises that some things must be protected. A robot may have very little of that common-sense knowledge, particularly when a request uses words such as nearby, gently or out of the way.
One response is to write highly detailed instructions. Another is to show the machine exactly how to complete the task, sometimes many times. Both methods take effort. The person teaching the robot may also fail to show every awkward combination of objects it will encounter later. If the robot learns only a narrow path, it may become confused when the laptop has moved or the cup starts from another position.
MIT’s researchers looked for a way to reduce the burden on the instructor. They used demonstrations in which a person physically guided the robot through an action. Sensors recorded the movement. Language models then helped infer what the person appeared to be trying to achieve, including constraints that were not perfectly described in the original request.
This is a useful distinction between copying and understanding. Copying the position of a person’s hand is insufficient if an obstacle has moved. Understanding that the goal is to deliver the cup without striking the computer offers a principle that may survive a change in the scene. Researchers still need to show that the machine follows that principle reliably, but it is a more useful target than memorising one trajectory.
Two AI steps decide what the robot should notice
The researchers’ method, called Masked Inverse Reinforcement Learning, has an awkward technical name but a comprehensible purpose. First, a language model looks at the instruction and the demonstrated movement to expand an incomplete description. It might turn a vague command to stay close into something more precise about remaining near the table’s surface. The model’s job is to help identify the intended behaviour rather than supply a final physical movement by itself.
A second language model then helps determine which details of the surroundings are important to the movement. For example, the position of a laptop could matter because the cup must not collide with it. A person’s posture during one demonstration might be irrelevant if it does not affect the task. The system masks, or filters out, details judged unimportant so the planning process can focus on the parts that change the correct action.
A separate algorithm uses those priorities to guide the robot’s motion. This layered arrangement is important. The language models are not simply issuing a stream of unchecked motor commands. They are assisting the interpretation of a task, and the physical movement is handled through a more structured planning process. The researchers hope that separation will help a machine generalise from fewer examples.
Of course, a system that filters information introduces a new danger: it might ignore something that actually matters. A chair, a cable or a person’s arm could become an obstacle after the training demonstration. The model must not treat its initial judgement of relevance as an unchangeable law. Safe robotics will require reliable sensing and ways to respond when the scene is different from what was expected.
How much better were the trained robots?
In the research paper, first posted in 2025, the authors compare their method with alternative ways of learning tasks. MIT reports that the approach identified users’ unspoken preferences up to 15 per cent more often than comparison methods in the defined experiments. Simulated and physical tests included moving a cup around a laptop and following commands that involved staying close to, or away from, particular objects.
The researchers also found that their method could learn from substantially fewer examples in some tested settings. MIT describes a reduction approaching fivefold in demonstration requirements. That matters because demonstrations are costly to collect, especially when a robot has to be guided carefully by a person. But the result is not a promise that every robot task will need one fifth as much teaching. Different bodies, homes and jobs could produce very different outcomes.
In a physical experiment, the team trained a robotic arm with 50 guided demonstrations. It then performed movements involving cups and other objects while taking account of constraints such as avoiding a nearby computer. The research also tested prompts not seen in the initial training. These are meaningful steps beyond a purely simulated example, but they remain controlled tasks with a particular robot and set of objects.
It is tempting to turn a photograph of a robot moving coffee into a claim that a domestic assistant is ready for every kitchen. The evidence does not reach that far. A home contains moving children, pets, liquids, clutter, changing light and unexpected interruptions. A dependable household robot must handle those situations, detect when its plan becomes unsafe and stop when appropriate. The experiment tests parts of that problem, not the whole thing.
Why the coffee-and-laptop test is revealing
A cup of coffee sounds like a trivial object, but it exposes several layers of physical judgement. The robot has to grasp and carry the cup, recognise the destination and decide where the cup should end up. It also needs to understand that a nearby laptop is something to avoid, even if the human did not spell that out. If the machine approaches too closely, a small error in position could have an outsized consequence.
People make these choices almost without thinking. We know that a warm drink might spill and that an electronic device should not be covered in liquid. A robot trained on one movement does not automatically possess those broad expectations. The MIT approach attempts to infer a narrower set of task-specific priorities from language and demonstration. That is useful, but it is not the same as a full model of household safety.
LiveAIWire has reported on training robots in virtual homes. Simulation can expose a machine to different room layouts and tasks before it enters a real building. The coffee experiment complements that work by showing how people might provide imperfect instructions and examples. A combination of varied environments and better task interpretation could ultimately make robots more adaptable, although both methods still need convincing field tests.
Another connection is the problem of control. LiveAIWire has examined human oversight of AI agents. Physical robots make the need for oversight especially visible. A misplaced sentence can be corrected in a document, while a misplaced robotic arm may strike a person or break an object. An intelligent assistant should be able to signal uncertainty and request help before turning a doubtful interpretation into an unsafe movement.
A good robot needs to know what not to do
Robot demonstrations often focus on a positive outcome: the machine picks up a block, places a cup or opens a drawer. Real instructions also contain negative requirements. Deliver the item, but do not spill it. Put away the books, but do not discard important papers. Clear the surface, but leave the fragile object untouched. Those constraints may matter more to the user than achieving the nominal task as quickly as possible.
The researchers’ idea of concentrating on relevant environmental details addresses that difficulty. If the machine can infer that the computer matters to a cup-delivery task, it may choose a path that avoids it without needing a separate hand-written prohibition. Yet no learning method can guarantee that every hidden requirement will be noticed. A robot could decide that an object is irrelevant simply because its training examples never showed a collision with it.
Developers therefore need methods for detecting errors, asking clarifying questions and halting an action. A robot ought to be able to recognise that a path has become blocked or that the user gave contradictory instructions. It should not respond to uncertainty by pressing on with its best-sounding guess. Such safeguards would be particularly important in shared workplaces, where many people move through the environment and expectations change throughout the day.
There is a training trade-off too. If people have to provide exhaustive examples of every safe movement, robot adoption becomes expensive and slow. If a system learns from very few examples without enough checking, it could behave unpredictably. The productive goal is to reduce unnecessary teaching while maintaining, and eventually improving, how reliably robots respect important constraints.
The long road from a careful arm to a useful helper
The study demonstrates that language models can help organise how a robot learns an incomplete instruction. It also shows why robotics cannot be solved by language alone. Sensing, movement planning, physical control and safety testing all remain essential. A model may correctly identify that a laptop should be avoided, but the robotic arm must still execute a collision-free motion in the real space in front of it.
Future work could connect clearer task descriptions to richer sensing, including cameras that let a robot spot changes in its surroundings. The MIT team has discussed making the approach more dynamic in this way. That would help with scenes that differ from the demonstrations, but introduce fresh problems such as visual errors, occlusion and the need to handle objects the system has never encountered.
LiveAIWire’s coverage of robots learning from ordinary videos adds another piece to the picture. Examples, language and visual observations can all contribute to training. The hard work is deciding which parts of those inputs still apply when the robot has to act. A useful helper must separate the goal from the accidents of one example, and must notice when it lacks enough information.
Bringing coffee without knocking over a laptop may sound like a modest ambition for artificial intelligence. For a physical machine, it is a demanding test of what a human really meant. The research makes a credible step towards robots that understand the instructions we actually give, rather than the impossibly detailed instructions a machine might wish we had provided.
About the Author
Stuart Kerr is Technology Correspondent at LiveAIWire, covering artificial intelligence, cybersecurity and the social impact of emerging technology. LiveAIWire is an independent, human-led technology publication using AI-assisted research, editorial production and original AI-assisted editorial illustrations under his direction.