Speaker Diarization Accuracy in Multi-Participant Meetings

In meetings, that question becomes one of the hardest unsolved problems in applied speech technology, and the gap between what vendors claim and what the audio actually allows is wide enough to change how any serious buyer should evaluate these tools.

What speaker diarization does in meetings

A diarization system runs through a pipeline: voice activity detection flags where speech occurs, a speaker embedding model turns each stretch of speech into a numeric fingerprint, a clustering algorithm groups those fingerprints by presumed speaker, and an assignment stage labels each segment. An error introduced at any one of those stages carries forward into every stage after it, with nothing downstream able to correct it. That matters more in meetings than almost anywhere else, because most published accuracy claims in speech technology are built on audio that looks nothing like a meeting: read audiobooks recorded in a quiet studio, call-center audio with one speaker per channel, controlled lab speech with scripted turns. None of that audio has the free-flowing speaking style, the room reverberation, or the simultaneous talkers that define a real meeting. The M2MeT challenge paper names the task directly as figuring out "who speaks what at when," and lists the conditions that make meetings different: overlapping speech, an unknown number of speakers, signals picked up from far-field microphones, background noise, and reverberation, conditions absent from an audiobook benchmark. That challenge's own baseline diarization error rate came in at 15.60%, but it came from a controlled academic setting built to study this problem. The scale of the real-world gap is visible on the Hugging Face Open ASR Leaderboard, where, as of September 19, 2026, meeting audio produced roughly seven times more word errors than read audiobook audio across the systems tested. Vendor accuracy numbers need a second look before anyone applies them to a meeting room, because that ratio is the clearest reason why.

How overlapping speech degrades diarization

Overlapping speech does more damage to diarization accuracy than any other single factor, and meetings generate overlap by design: people interrupt, finish each other's sentences, and talk over each other when they agree or object. A short frame of audio belongs to exactly one speaker, or so most clustering-based diarization systems assume. The moment two people talk at once, that assumption breaks, and the system produces a speaker confusion error almost by definition, with no mechanism inside the pipeline to notice and fix the mistake once it has been made. The scale of this in real meeting data is striking. The CUHK-Tencent submission to the M2MeT challenge found that speech overlap ratios averaged 42.27% in the AliMeeting training set, and the evaluation set showed a similarly high share. Well over a third of all speech in those real Mandarin meetings was overlapping at any given moment. Detecting that overlap is even occurring turns out to be its own hard problem, one that has to be solved before any system can begin to untangle who said what within the overlapping stretch, and the M2MeT challenge summary calls overlapping speech detection "a challenging task in most situations". Progress has been made on transcribing speech from multiple simultaneous speakers, and error rates on that specific task have fallen substantially since 2019, but they remain far from matching single-speaker performance. Fast turn-taking compounds the problem further: rapid back-and-forth conversation shortens each speaker's segments, leaving the embedding extractor less audio to work with before a clustering decision gets locked in. DM-ASR, a recent research system, frames the challenge as needing to resolve "rapid turn-taking, speaker overlap, and long-range conversational context" all at once, a joint problem that cascaded pipeline systems handle poorly because an error made at the diarization stage flows straight into the transcription stage with nothing to catch it.

Why accuracy degrades further as participant count grows

Adding participants to a meeting does not raise difficulty in a straight line. It multiplies the overlap and embedding-confusion problems already described, producing a measurable step down in accuracy at each tier of group size. Real-world benchmark data on speaker attribution from 2026 shows that staircase clearly: two-speaker calls get the highest attribution accuracy from leading tools, three-to-five speaker meetings see that accuracy fall off noticeably, and meetings with six or more speakers drop further still. The mechanism behind that staircase is straightforward once the pipeline is understood. As the number of speakers in a room grows, the number of possible pairs of speakers who could be confused for one another grows combinatorially, not linearly, and at the same time, the floor time available gets divided among more people, so each speaker's embedding model has less "enrollment audio," meaning fewer seconds of that person's voice to build a reliable fingerprint from. The M2MeT corpus builders knew this dynamic well, so they built their entire challenge around sessions of two to four speakers, a range they chose because it already produces high overlap ratios and high acoustic complexity. Sessions with more than four speakers push further into territory the research community has not fully mapped. A second compounding factor sits upstream of all of this: most systems do not know in advance how many people are in the meeting and have to estimate that number from the audio itself before clustering can even begin. A wrong estimate at that stage corrupts every attribution decision made afterward. The M2MeT challenge summary lists "unknown number of speakers" alongside overlapping speech and far-field signals as one of the primary challenges facing any system. The system has to detect how many speakers are present before it can even attempt to attribute speech to them. The strongest academic result, produced by the winner of the ICASSP 2022 M2MeT challenge, required multi-channel microphone arrays and research-grade processing conditions, and the challenge summary established a baseline DER of 15.60%, showing how far unoptimized systems sit from the ceiling even in controlled conditions. So even with research-grade hardware, this gap between the baseline and the best achievable result shows how much headroom remains unclaimed.

How audio conditions interact with speaker count

Microphone placement and room acoustics do not just add a layer of background noise on top of the difficulty of counting speakers already described. They interact with it, making speaker embeddings less distinguishable from one another in ways that compound errors that would exist regardless. In the AliMeeting corpus, which was recorded in large conference rooms, microphone-to-speaker distance ranged from 0.3 to 5.0 meters. At the longer end of that range, reverberation smears the acoustic signature of each speaker's voice, reducing the distinctiveness that the clustering algorithm depends on to tell one person's voice from another's. Reverberation causes a particular kind of self-interference: a speaker's voice reaches the microphone directly and also arrives moments later as a reflection off the walls and ceiling, so a single person's own voice ends up overlapping with itself in time, degrading embedding quality even when nobody else is talking. Enterprise-grade systems have a tool available that consumer setups do not. A single laptop microphone or a standard conferencing speakerphone has no access to that spatial information at all. So the attribution accuracy figures already cited for six-or-more-speaker meetings assume some reasonable audio setup to begin with; if you put a laptop microphone in the middle of a large conference room, real-world performance will push further toward the floor than those figures suggest. A second, parallel failure path runs alongside the acoustic one: domain-specific vocabulary and accents introduce transcription errors even when the diarization layer gets the speaker right. A meeting can end up with notes that correctly identify who was talking while still putting the wrong words in that person's mouth.

What current commercial engines achieve on meeting audio

The commercial state of the art in 2026 is genuinely strong on clean audio, but it gets materially weaker once conditions shift toward the multi-speaker, overlapping, far-field setting that defines a real meeting room, and that distance between marketing claims and meeting-condition performance is large enough to matter if you build an automated workflow on top of these tools. On clean, read English, leading commercial speech recognition systems get very low word error rates. On noisy, real-world audio those error rates climb meaningfully, and the gap widens further once the audio comes from an actual meeting rather than a studio. For diarization specifically, a clearer picture emerges from direct benchmark comparisons: AssemblyAI's Universal-3.5 Pro, released June 23, 2026, leads commercial APIs at 30.17 cpWER, a concatenated minimum-permutation word error rate measure, with ElevenLabs Scribe v2 at 35.26 and Deepgram Nova-3 at 37.92, all measured on benchmarks that may still understate the hardest meeting conditions. Leading models post high word accuracy on clean audio, then see that accuracy fall off noticeably once crosstalk, accents, and domain-specific jargon enter the picture, with further drops as speaker count climbs past five. The research ceiling established by the ICASSP 2022 M2MeT challenge winner, achieved using multi-channel array processing that no standard commercial meeting tool currently deploys, shows how much of the progress made in research settings has not yet made its way into any product people can actually buy. The honest way to frame this for a buyer is that current tools are accurate enough to be genuinely useful in two-to-three person meetings recorded on good audio, and that accuracy falls off in documented, predictable ways as conditions get harder from there. What matters is not whether a given tool is accurate in the abstract, but whether it is accurate enough for a specific meeting format and a specific downstream use.

Why diarization errors compound in automated workflows

Meeting intelligence tools increasingly route attributed speech straight into CRMs, project management systems, and compliance archives, and once that pipeline exists, a diarization error at the audio layer is no longer a transcript that reads a little oddly but a data-integrity error inside a system of record. The cascade runs in a predictable sequence: speech gets misattributed to the wrong person, a commitment or an objection gets logged against the wrong speaker, a CRM field gets populated with that incorrect attribution, and from there either a sales rep acts on a bad signal or a compliance record ends up reflecting a statement from the wrong person entirely. Sales teams already lose significant time each week to fragmented data entry and incomplete CRM follow-up after customer calls, and poor diarization sits upstream of that problem: if the system gets wrong who said what, the CRM fields built on that transcript are corrupted right at the source, and any agentic automation promising to save hours per week only delivers on that promise if the attribution it is acting on can actually be trusted. Microsoft gave this problem institutional weight in its release notes, which describe Teams meetings becoming reference sources inside Copilot Notebooks, pulling in transcripts, notes, chats, and shared content as part of an organization's working memory. So once meeting transcripts are treated as a durable knowledge source at that scale, the accuracy of the diarization layer underneath them becomes an enterprise-wide dependency affecting every system built on those transcripts. The stakes rise further in financial advisory and other compliance-sensitive fields, where an AI notetaker often acts as the gatekeeper for incoming client communication that flows into a compliance archive; if diarization errs there, you get a fiduciary and regulatory record error, not a minor annoyance. The problem compounds over time, too, as new hires onboarding from a meeting archive inherit decisions and commitments that were misattributed from the start, building institutional knowledge on a foundation nobody checked. The obvious response, that someone should simply review the notes before they get pushed into these systems, does not hold up at scale: the entire value proposition of agentic automation is removing that manual review step, and any team that reviews every AI-generated note by hand is not getting the productivity gain that justified buying the tool in the first place.

Closing the gap: research directions

The most promising research directions treat speaker attribution and speech recognition as a single problem to be solved jointly, rather than as two sequential stages in a pipeline where mistakes in the first stage poison the second. DM-ASR, published in April 2026 by researchers at Wuhan University and Tencent Ethereal Audio Lab, reformulates multi-speaker transcription as a multi-turn dialogue generation process, feeding diarization results into a large language model as an explicit structural prior rather than treating them as a fixed, final answer. A related line of work, Speaker-Reasoner, explores scaling the number of interaction turns and reasoning patterns a model uses when producing timestamped, speaker-attributed transcripts, applying language-model reasoning to resolve speaker ambiguity rather than relying purely on acoustic signal processing. These are genuine advances in how the problem gets framed. But the clearest research result available, the 2.98% diarization error rate reached by the CUHK-Tencent system that won the ICASSP 2022 M2MeT challenge, depended on direction-of-arrival estimation pulled from a multi-channel microphone array, a form of spatial signal processing that remains entirely unavailable to any consumer-grade setup built around a single microphone. The research community continues to close the gap between what is theoretically achievable and what today's systems deliver, and the tools shipping to meeting rooms right now still depend on hardware and data conditions that most commercial deployments cannot replicate.