Group conversation captions can look helpful until a meeting, family meal or classroom discussion turns lively. Words appear on a screen, but the text does not always reveal who said them. If several people speak from different positions, a single stream of captions can become as difficult to interpret as the conversation it is supposed to clarify. Researchers at Google have tested a way to make the words easier to follow by adding information about where each voice is coming from.
The SpeechCompass research described by Google in July 2025 combines speech transcription with microphones that estimate sound direction. Instead of simply placing every sentence in the same block, the prototype can separate speakers visually and add directional cues. The underlying work was presented at a research conference. It is not a claim that every phone already offers the complete feature or that the prototype is ready for every noisy room.
Why group conversation captions can leave people behind
A transcript records words, but a conversation contains far more information. People take turns, interrupt one another, look towards a speaker and use their surroundings to understand the exchange. When a captioning app puts several voices into one continuous stream, it may remove the cues that help a person connect a statement to the right individual. That can be particularly frustrating for someone who relies on the text to participate rather than merely to review it later.
Picture a dinner table where somebody asks a question from the left and another person answers from behind the phone. A simple transcript might display both utterances without distinction. The reader then has to work out which person is speaking by looking around, watching expressions or trying to match the text with movements. The extra effort can make it harder to concentrate on what is actually being said.
The issue is not confined to deaf or hard-of-hearing users. Group transcripts are also useful in busy meetings, unfamiliar languages or settings where a participant cannot clearly hear every speaker. An interface that shows speaker changes could make many transcripts easier to navigate, provided the cues themselves are accurate and easy to understand.
How the phone can estimate who spoke
SpeechCompass experiments with the small differences in timing between sound arriving at microphones. A voice coming from one side may reach a particular microphone fractionally earlier than another. By comparing the signals, software can estimate a direction and link it to text being transcribed. The design then presents the information through coloured speaker blocks, arrows or similar visual markers.
The researchers explored a phone-case prototype with additional microphones and a software approach that uses microphones already present on some phones. Those versions are not identical, and the amount of directional coverage differs. The additional hardware provides a fuller view around the device, while ordinary built-in microphones impose greater limits. The point is to give the user helpful cues without requiring every person in the room to wear a special microphone or register a voice in advance.
This is an intelligent use of audio information that a human listener often processes automatically. We tend to notice when speech comes from our left or right. But a phone must calculate direction from reflections, competing sounds and imperfect microphone positions. The system therefore works with estimates, not an infallible map of the people in the room.
What testing found and what remains difficult
Google reported technical evaluations of sound-direction accuracy and speaker separation. In controlled tests, the prototype could locate the direction of speech with errors that depended on the microphone configuration and conditions. The researchers also surveyed people who frequently used captioning, and gathered feedback from a small group who tried a prototype demonstration. Participants valued distinguishing speakers and showing directional information, but the user demonstration was limited.
A laboratory result cannot answer every real-world question. Busy cafés have overlapping talk, music, kitchen noise and hard surfaces that reflect sound. In a meeting room, two participants may be seated close together, making direction alone less useful. A person may also move while speaking. Research systems need to deal with those situations before a consumer-facing product can be relied upon routinely.
The interface also has to remain readable. A screen covered in arrows, colours and speaker markers could become another source of distraction. People need a way to identify speakers that is clear at a glance, particularly when they are simultaneously watching the room and contributing to the conversation. More information is helpful only when it reduces effort rather than adding to it.
Accessibility benefits are about participation
The value of captioning is not simply producing written words. For many users, it is an aid to understanding people around them in real time. A person following a conversation needs to know not only what was said but when to respond and who is asking a question. A system that provides better conversational structure could support that interaction without demanding that the user constantly reconstruct it from a stream of text.
Google's older Live Transcribe research describes the aim of providing continuous transcription on mobile devices. SpeechCompass addresses a different challenge that emerges after the basic words have been captured: making a many-person conversation intelligible. This is a meaningful progression from recognising speech to presenting it in a way that fits how people actually communicate.
There are privacy choices too. A room contains people who may not expect their voices to be recorded or transcribed. Any future application should be clear about its audio processing, retention and sharing behaviour. A visually separated transcript is useful, but it does not remove the need for appropriate consent and sensible data handling.
Direction is helpful, but it cannot correct every word
Accurately separating speakers does not guarantee that the speech recognition itself is correct. Accents, unusual names and background noise can still produce transcription errors. LiveAIWire has examined how automatic captions can change the meaning of a sentence. That problem remains even if the phone identifies the right person. Better direction and speaker labelling are a complement to accurate words, not a substitute for them.
There is an important distinction between finding the location of a voice and isolating that voice from all others. We have also reported on headphones designed to focus on a chosen speaker, which tackle a different aspect of crowded listening. Separating audio for the listener and organising text for the reader can be related goals without being identical technologies.
Likewise, AI translation can sound fluent while missing important meaning. Adding speaker cues to translated captions may make a multilingual conversation easier to follow, but it does not make the translation trustworthy by itself. Users need to know which parts of the system are estimates and when important statements should be confirmed directly.
The setting matters as much as the software. In a quiet meeting room, a phone may distinguish speakers more easily than on a crowded train or at a family gathering where several people interrupt one another. A person speaking while facing away from the microphone can also sound very different from someone nearby. Even when the system estimates direction correctly, it may still miss a name, a joke or the crucial word in a question. Speaker cues help organise imperfect captions; they do not guarantee accurate transcription.
A useful deployment would let someone correct or ignore those cues without stopping the conversation. It should also make any recording or processing arrangements clear to the people present. Accessibility can become awkward if a tool designed to encourage participation makes other participants uncertain about whether their words are being retained. Good design has to address that social setting, not only the acoustics.
What a useful product would need to get right
Before this kind of research becomes an everyday feature, designers will need to test it with people who depend on live captions across many real situations. They should evaluate different room layouts, speaking styles, accents and lighting conditions. The ability to change text size, colour contrast and the presentation of direction cues will matter. So will battery life and whether the system can work without constant access to a network.
A future app should also make uncertainty understandable. If two voices come from the same direction or people talk over one another, the display should not falsely suggest perfect separation. A graceful indication that the system is unsure can be more valuable than a confident but incorrect speaker label.
The research points towards a modest but important shift. Automatic transcription is moving beyond the question of whether a phone can capture the words. The next question is whether the result makes a real conversation easier to join. For someone trying to follow several people at once, a small arrow showing who spoke may be every bit as useful as another sentence appearing on the screen.
About the Author
Stuart Kerr is Technology Correspondent at LiveAIWire, covering artificial intelligence, cybersecurity and the social impact of emerging technology. LiveAIWire is an independent, human-led technology publication using AI-assisted research, editorial production and original AI-assisted editorial illustrations under his direction.