How AI Meeting Transcription Works Under the Hood

AI meeting transcription is not one piece of technology. It's a chain of five or six separate systems, each handing its output to the next, and the accuracy swing users notice between a clean 1:1 call and a messy hybrid meeting comes from specific breakdowns at specific points in that chain.

The naive expectation is simple: hit record, get text, done. The reality is that spoken language is noisy, full of ambiguity, spoken by multiple people at once, and loaded with jargon no general-purpose model has seen before. None of that gets handled cleanly by a single algorithm. Meetings make it worse: crosstalk, a laptop mic three feet from a whiteboard, a room full of accents, a hybrid setup where half the attendees are on speakerphone and the other half are on headsets. Vendor accuracy claims are between 95% and 99% on clean audio, and drop to somewhere between 80% and 90% on noisy audio. That 10 to 19 point gap isn't noise in the data. It's the sum of specific failures at specific layers, and understanding those layers explains why two tools with nearly identical marketing pages perform so differently once they hit a real meeting.

How audio reaches the model: capture architecture before any AI runs

Before any acoustic model touches a waveform, the audio has to get captured, and by 2026 there are two fundamentally different ways that happens.

Bot-based capture works by sending an external participant into the video call. It joins the link, announces itself as a recording bot, pulls the audio up to a cloud server, and processes it there before saving the resulting file. Device-level capture skips the visible bot entirely. It pulls audio straight from the device's own microphone and speaker output and processes audio without anything joining the call that other participants can see.

This distinction determines the signal a model receives, because that signal is entirely shaped by which path the audio took to get there. A cloud-processed bot recording from a shared conference room mic is a different animal than a laptop's device-level capture of a single headset. Hybrid meetings expose this hardest: the people in the conference room are all sharing one mic, picking up echo, distance, and room reverb, while remote attendees each get a clean individual feed. Diarization and word accuracy for the two groups end up on completely different footing, even inside the same meeting.

Some transcription vendors now ship both a bot and a bot-free desktop option, a tell in itself: no single capture model handles every meeting context well, and the vendors building for enterprise use have started admitting as much in their product lineup.

Somewhere in this same step, before any speech model runs, preprocessing happens: background noise gets stripped out, volume gets normalized across speakers. This is the cleaning pass, and it sets the ceiling for everything downstream.

Acoustic modeling: turning a raw waveform into phonemes

An acoustic model's job is narrow: take a continuous audio waveform and map it to phonemes, the smallest distinct sound units in a language. It does this by slicing audio into short frames, often just milliseconds long, and running each frame through a neural network (in modern systems, a transformer) that outputs a probability distribution over which phoneme that frame most likely represents.

Models like Wav2Vec 2.0, HuBERT, and WavLM defined an influential generation of self-supervised acoustic models, and most of that generation has since been succeeded by newer systems like Whisper. But the underlying mechanics haven't changed: it's still a probabilistic guess, frame by frame, and the quality of that guess depends entirely on what the model saw during training.

That's where accents, non-native speech patterns, unusual cadence, and overlapping speech all become trouble. The model isn't reasoning about language; it's pattern-matching against a training distribution, and speakers who fall outside that distribution get worse results. One documented case makes the point sharply: Wav2Vec 2.0, tested against Portuguese emergency call audio, produced a Word Error Rate of 42.12%, a number that looks nothing like the much lower error rates the same architecture achieves on clean English speech. A model can be excellent and still fail badly the moment it steps outside the data it was built on.

That's not just an accuracy problem, either. If a transcription tool systematically produces worse results for speakers with accents or non-native English patterns, it's a fairness problem too, one that appears in the transcript itself, long before anyone gets to the summary at the bottom of the page.

Domain adaptation can patch some of this at the acoustic layer itself. Custom vocabularies and speech context boosts, flagging brand names or technical terms ahead of time, give the model a nudge toward the right phoneme sequence when the audio is ambiguous. It's a real fix, but it takes setup, and it only works for what the user thinks to flag in advance.

Automatic Speech Recognition: from phonemes to a word-for-word transcript

ASR sits one layer up. Its job is to take the acoustic model's phoneme guesses and combine them with a language model that picks the most probable sequence of actual words, not just sounds. The language model component is what decides "their" versus "there": that's not an acoustic distinction at all, it's a judgment about which word fits the grammar and context of the sentence.

Word Error Rate is the standard yardstick here: count the substitutions, deletions, and insertions needed to turn the ASR output into the correct reference text, divide by the total word count of that reference. On high-quality audio, tools run around 4% to 7% WER, or roughly 4 to 7 mistakes per 100 words. In harder conditions, that number climbs past 15%.

There's also a real trade-off between streaming and offline ASR. Streaming, or real-time, transcription has to commit to a guess as the audio arrives, with no ability to look ahead, so accuracy takes a small hit in exchange for low latency. Offline, post-meeting ASR gets the whole recording at once and can revise an earlier guess based on what gets said two sentences later. That's why live captions during a call are consistently rougher than the transcript the same tool produces an hour afterward.

What comes out of raw ASR, before any cleanup, is not pleasant to read: a wall of unpunctuated text, filler words intact, false starts left in exactly where the speaker stumbled. This is also where transcription products start to diverge architecturally, since different vendors are built on entirely different ASR foundations. Microsoft Teams runs on MAI-Transcribe (now MAI-Transcribe-2), a first-party model Microsoft built in-house. Zoom AI Companion uses Zoom's own ASR engine. Other products build on top of OpenAI's Whisper model, layering proprietary processing on top of it. The point stands regardless of which foundation a given tool picks: ASR gives you words. It does not give you meaning, identity, or structure. Those come from layers built on top of it.

Speaker diarization: assigning words to the people who said them

Diarization answers a question ASR can't: who actually said this? It works by segmenting the audio into distinct utterances and generating a speaker embedding for each one, a numerical fingerprint of that speaker's vocal characteristics. The model then compares embeddings across segments, clusters the ones that sound alike, and assigns each cluster a speaker label.

There's a real gap between what diarization can technically do and what it reliably does in practice. By 2025, diarization systems could differentiate up to 30 distinct speakers in a single recording. But most tools hold up reliably only up to around 8 distinct speakers before attribution starts to slip under real-world conditions. That's the number to plan around, not the technical ceiling.

The failure modes are predictable once you know what the model is doing. Fast back-and-forth exchanges don't give it enough audio per turn to build a confident embedding. Overlapping speech, two people talking at once, collapses into a single segment the model has to guess at, and it often guesses wrong. Hybrid rooms compound the problem in a different way: several in-person participants sharing one mic sound acoustically similar to the model, which makes separating them into distinct speaker clusters much harder than separating remote participants on individual feeds.

Some enterprise systems link diarization output to a company's SSO directory, so instead of "Speaker 1" and "Speaker 2," the transcript carries actual names. That distinction matters more than it sounds. A transcript without speaker attribution is searchable, but it isn't accountable: no action item, commitment, or decision can be pinned to a specific person without this layer working correctly.

NLP post-processing: cleaning raw transcript into readable text

What lands after ASR is a run-on mess: filler words like "um" and "you know," false starts, sentences that change direction halfway through, phrases repeated because the speaker was thinking out loud. NLP post-processing is the layer that turns that into something a human wants to read.

This is where punctuation gets inferred from prosody and syntax, where capitalization and sentence boundaries get inserted, where filler words get stripped or flagged, and where paragraph and speaker-turn formatting gets applied. Vendors rarely document this layer in any detail. It's mostly proprietary and invisible, but its quality becomes visible the moment you read a raw transcript, since a badly tuned post-processing layer leaves a transcript that's technically correct and still hard to skim.

Domain adaptation matters again here. Custom dictionaries make sure brand names, product names, and technical shorthand get rendered as written words rather than phonetic approximations that sort of sound right. Code-switching, alternating between two or more languages mid-conversation, gets handled partly at this stage too, which matters directly for multilingual teams and international calls.

This layer stops at readability. It doesn't extract meaning, flag action items, or produce a summary. That's a separate job, handled by the layer sitting above it.

LLM summarization: where transcript becomes usable intelligence

Everything below this point in the pipeline records what was said. The LLM layer is the first one that tries to figure out what it meant, inferring intent and weighing significance rather than just transcribing words in order.

Two techniques dominate here. Extractive summarization pulls verbatim sentences straight from the transcript, which keeps hallucination risk low but can read stiff and mechanical. Abstractive summarization generates new sentences that capture the meaning in a more natural voice, at the cost of introducing paraphrase drift or, in the worst cases, details that were never actually said.

By 2026, the standard output looks fairly consistent across serious tools: a short executive summary, usually three to seven sentences, tuned to the type of meeting it came from. Sales calls get BANT or MEDDIC fields, objections raised, competitors named, values formatted for direct entry into a CRM. Action items get pulled out with speaker attribution and deadline detection, triggered by phrases like "I'll handle," "we need to," or "by Friday."

Vendors implement this layer very differently. Zoom AI Companion runs a federated mix of its own small proprietary models alongside outside models for the semantic and summarization work. Microsoft's Teams Copilot does its summarization and task extraction through Azure-backed models as part of the post-meeting workflow, though sentiment analysis is gated behind the separate Copilot for Sales add-on rather than included in standard Teams Copilot.

Hallucination is the real risk at this layer. An abstractive LLM can state something with total confidence that was never actually said in the meeting, and the primary defense against that is grounding the model tightly to the transcript instead of letting it generate freely. This is also where the real product differentiation lives. Two tools running the exact same ASR engine, with identical word-error rates, can produce wildly different summaries depending on prompt design, how well the tool detects meeting type, and how tightly it keeps the model anchored to what was actually said.

How the layers interact to produce the accuracy and quality variations users actually experience

None of these layers fail in isolation. Noisy input degrades the acoustic model's phoneme guesses, which degrades ASR's word choices, which hands the LLM a corrupted source to summarize. Errors accumulate rather than staying contained to the layer where they started.

The 95% to 99% accuracy figure assumes clean input from the start, and that assumption produces the figure. Drop in overlapping speech or background noise, and the number falls to 80% to 90%, with every layer above inheriting that damage. Diarization errors work the same way, cascading upward: if the system can't reliably tell who said what, the LLM has no way to correctly attribute an action item or a commitment to the right person, no matter how good its summarization is otherwise.

This is why the same product can feel completely different across two meetings run a day apart. A 1:1 call between two people on headsets, speaking one language, is comfortably at the accuracy ceiling. An eight-person hybrid all-hands with cross-talk and technical jargon stresses every layer at once, and the compounding effect is visible in the output.

Custom vocabulary and domain adaptation can close some of that gap at the acoustic and ASR layers, but they require someone to actually do the setup work first. There's also a structural gap: streaming transcription trades accuracy for speed, so the same tool's live captions will consistently read worse than its finished, post-meeting transcript. Testing a tool only on a quiet solo recording will always overstate how it performs in the conditions where getting it right actually matters.

What capture architecture means for consent and recording visibility

The capture architecture decision from earlier in the pipeline resurfaces here as something closer to an ethical question than a technical one. Bot-based capture is visible by design: a named participant joins the call, announces that recording is happening, and processes the audio in the cloud. Device-level capture shows nothing. The recording happens locally, and other participants may have no idea it's occurring at all.

Consent isn't a checkbox feature buried in settings. It's a legal and ethical obligation, and a tool's architecture either supports that obligation openly or quietly works around it. There's real legal weight behind this too: meeting transcripts that get synced into a CRM or stored in cloud systems count as business records, and regulated firms carry retention and retrievability obligations tied to that status (registered investment advisors, for instance, fall under SEC books-and-records rules).

The equity concern from the acoustic layer resurfaces here as well. If a tool produces systematically lower accuracy or misattributes speech for people with accents, non-native English patterns, or distinctive vocal cadence, decisions made off those flawed transcripts carry real discrimination risk, not a hypothetical one.

The baseline practice, regardless of which architecture a team uses, is telling other participants a call is being transcribed. That obligation doesn't go away just because the bot isn't visible. Enterprise buyers also need to check compliance posture directly: SOC 2 Type II, GDPR, HIPAA, where applicable, since these govern where audio and transcripts live and who can access them, and the answer changes depending on which capture architecture is in play. Neither architecture is correct in every context. A visible bot fits an external client call or a formal review where everyone expects and wants a documented record. Silent device-level capture fits an internal working session where a team has already agreed, as standing policy, that meetings get captured that way.

What to look for when evaluating a tool across the full pipeline

Vendor demos run on clean, quiet, single-speaker audio, which is exactly the condition that produces that 95% to 99% accuracy figure. Test on a recording from an actual team meeting instead, with the accents, jargon, and crosstalk that occur every week, and expect the number to move.

Work through the pipeline layer by layer rather than judging on one headline number. On capture, check whether the tool offers both a bot mode and a device-level mode, and think about which one actually fits the mix of meetings a given team runs. On ASR, ask what the Word Error Rate looks like specifically on domain terms the model has never seen, and whether custom vocabulary upload is even supported. On diarization, push past the "up to 30 speakers" marketing number and find out where accuracy actually starts to slip, since that practical ceiling, often closer to 8 speakers, is the number that matters in a real hybrid room. On summarization, check whether the output is grounded tightly to the transcript or left to generate more freely, since that's the difference between a summary that's slightly plain and one that quietly invents things nobody said.

None of these layers can be judged in isolation, and none of them can be fixed by a better model at just one point in the chain. A transcription tool is only as good as its weakest layer, and the weak layer is rarely the one the marketing page talks about.