The words are rarely the problem. The names are. A modern speech model gets most of a clear meeting right. What it gets wrong more often is who said each line, and that error survives every summary built on top of it.
Diarization and identification are two different jobs. Diarization splits the audio into Speaker 1, Speaker 2 and so on. Identification puts a real name on each voice, and it needs extra information: a participant list, a per-person audio stream or a stored voice profile.
Check the lines that carry consequences, not the whole transcript. Decisions, commitments and anything the AI turned into a task with an owner deserve a second look. Filler turns and small talk do not.
Stored voice profiles are personal data with extra rules. In the EU, a voiceprint used to recognise a specific person can count as biometric data, so ask where the profile lives before you switch that feature on.
Speaker labels in transcription look like a formatting detail until a summary tells your team that the client agreed to a price your sales lead proposed and the client never accepted. That is the failure worth guarding against. A transcript can get nearly every word right and still send an action item to the wrong person, because the speech model and the speaker model are separate systems that fail in separate ways.
This guide explains how tools produce speaker labels, why the labels drift, and a five-step review you can run in a few minutes before a transcript or AI summary leaves your hands.
How transcription tools decide who said what
There are three routes to a labelled transcript, and most people never find out which one their tool uses. The route decides what kind of mistakes you will see.
| Method | What it does | Label you get | Typical failure |
|---|---|---|---|
Speaker diarization | Groups stretches of audio by voice, with no idea who anyone is | Speaker 1, Speaker 2 | Splits one person in two, or merges two similar voices |
MethodSpeaker diarization What it doesGroups stretches of audio by voice, with no idea who anyone is Label you getSpeaker 1, Speaker 2 Typical failureSplits one person in two, or merges two similar voices | |||
Platform metadata | Reads which participant's microphone was active on the video call | The participant's display name | Everyone in a shared meeting room gets one name |
MethodPlatform metadata What it doesReads which participant's microphone was active on the video call Label you getThe participant's display name Typical failureEveryone in a shared meeting room gets one name | |||
Speaker identification | Matches each voice against a stored voice profile | A real name, across meetings | Wrong match when a voice was never enrolled or has changed |
MethodSpeaker identification What it doesMatches each voice against a stored voice profile Label you getA real name, across meetings Typical failureWrong match when a voice was never enrolled or has changed | |||
Diarization is the base layer and the one almost every notetaker runs. The standard open-source recipe pairs Whisper for the words with pyannote for the speaker turns, as the top answers in a 30-comment r/LocalLLaMA thread on diarization models point out. Diarization only answers "which voice is this", never "whose voice is this", so the output is anonymous until something maps Speaker 2 to a person.
Platform metadata is why labels on a video call often look better than labels from a recording. A bot that joins Zoom or Teams can see which participant was talking, so it borrows the display name. In a 60-comment r/macapps survey on meeting transcription, one commenter named a tool that lets the meeting platform handle attribution, rather than guessing from audio, as the closest thing to flawless speaker identification they had found.
Identification goes one step further and remembers voices. Hobby projects show how it works: a local transcription app shared on r/LocalLLaMA saves the detected speakers' voice embeddings after diarization so it can recognise them next time, and a Python library posted to r/Python does the same for any audio file. That memory is useful, and it is also the part with legal weight, which we come back to below.
Why speaker labels go wrong
Every cause on this list comes down to the same thing: the speaker model has less to work with than you think.
- Overlapping speech. When two people talk at once, the model has to give the segment to one of them. The interruption usually goes to whoever was already speaking.
- Short turns. A two-word reply like "got it" or "no, Tuesday" holds too little voice for a confident match. These are exactly the turns where agreement and disagreement live.
- Similar voices. Two colleagues of the same age and accent, speaking at the same distance from the microphone, get merged into one speaker more often than you'd expect.
- More people, less accuracy. Each extra speaker is another cluster the model can confuse. A four-person call is a different problem from a two-person interview.
- Distance and room noise. One laptop microphone at the end of a table hears the nearest person clearly and everyone else as echo. Our guide to AI notetakers for in-person meetings covers the numbers on how much a room degrades a transcript.
- Shared devices. When a conference room joins a call as one participant, platform metadata labels every voice in that room with the room's name. Diarization can split them, but it cannot name them.
The last cause catches teams that trust call-based tools. Their remote participants come out perfectly labelled, and the four people in the office show up as a single speaker called Meeting Room 2.
How to review speaker labels before you share meeting notes
You do not need to proofread every line. The goal is to make sure the lines that create obligations are attached to the right people. Five steps, in this order, cover it in a few minutes for a typical hour-long meeting.
Step 1: Find out which method produced the labels
Look at the raw labels before you look at the summary. Generic tags like Speaker 1 mean diarization only. Display names mean the tool read the call platform. Names on a recording you imported mean the tool matched stored voice profiles or a participant list.
This tells you where to look. Diarization errors cluster around similar voices and short turns. Metadata errors cluster around shared rooms and phone dial-ins. Identification errors show up when a guest was never enrolled and gets matched to the closest colleague on file.
Step 2: Build a speaker map from the opening minutes
Most meetings start with names: introductions, a greeting, someone saying "over to you, Priya". Use the first three to five minutes to decide who Speaker 1, 2 and 3 are, then rename the labels once. A consistent name everywhere beats a perfect first line.
If nobody said their name at the start, note the one distinctive thing each person talked about and map from that. The person presenting the budget is almost certainly the finance lead.
Step 3: Check the turn boundaries, not the paragraphs
Attribution errors live at the seams. When a speaker changes, read the last sentence before the switch and the first sentence after it. Short replies, interruptions and laughter are the usual suspects.
One fast test: search the transcript for the words yes, agreed, fine and no. Each hit is a decision point, and each one should belong to a person who had the authority to say it.
Step 4: Check every name the AI attached to a task or decision
This is the step that matters most, because it is where a labelling mistake turns into work. Many AI notetakers now create action items with owners from the transcript. If Speaker 2 was mislabelled as your account manager, the task "send the revised quote by Friday" lands in the wrong inbox, and nobody notices until Friday.
Go through the generated tasks and decisions one by one and jump to the source line for each. If the tool does not link tasks back to transcript lines, search for a phrase from the task. This takes a minute and prevents the most expensive kind of error.
Step 5: Fix the labels, then regenerate or re-read the summary
Correct the labels in the transcript first, then look at the summary again. Some tools rebuild the summary from the corrected transcript; others keep the original. If yours keeps it, edit the summary by hand or you will share a clean transcript next to a summary that still blames the wrong person.
Before you send anything, export the version you checked. The meeting minutes template is a useful skeleton if you are turning the reviewed transcript into formal minutes.
When to skip the full review
Not every transcript needs this. A solo voice memo, a one-on-one where the two voices are obviously different, or a recording that stays in your own notes can go out unchecked. The review earns its five minutes when the transcript leaves your team, feeds a CRM record or contains a commitment someone could later dispute.
Voice profiles and GDPR: diarization is not identification
Diarization compares voices inside one recording and then forgets them. Speaker identification keeps a voice profile so it can recognise the same person next week, and that difference has legal weight in the EU.
The GDPR defines biometric data in Article 4(14) as personal data from technical processing of physical or behavioural characteristics that allows the unique identification of a person. Article 9 puts biometric data processed to uniquely identify someone in the special-category group, which in practice usually means explicit consent. A stored voiceprint that recognises your client across meetings can fall into that category. Plain diarization inside a single file is much harder to argue into it.
Three questions to ask before you turn on voice recognition in any tool:
- Where is the voice profile stored: on your device or on the vendor's servers?
- Can a participant ask to have their profile deleted, and who does it?
- Did the people being enrolled agree to it, or only the person running the tool?
For the wider question of where AI meeting notes are processed, see our piece on AI meeting notes and the GDPR. This is not legal advice; if your team plans to enrol client voices, involve your data protection officer.
Where meetergo Log fits, and where it does not
meetergo Log is the AI notes app in meetergo's suite, which also covers scheduling, video meetings and CRM. It records system audio on your desktop and transcribes locally with Whisper, so it works with Google Meet, Zoom, Microsoft Teams and meetergo connect without a bot joining, and it also takes imported recordings in eight or more formats.
On speaker labels specifically, here is what meetergo documents today and what it does not:
- Speaker diarization is available with an account on the Light plan. The transcript attributes each passage to a separate speaker. Transcription itself stays free without an account.
- Speaker identification is not shipped yet. Voice recognition for recurring participants is listed on the roadmap, with profiles described as staying on the device. Until it ships, expect Speaker 1 and Speaker 2 style labels that you map yourself in Step 2.
- Label renaming is not described on the product pages. Check it in the app before you build a review process around it.
- Automatic tasks come with owners and due dates. That makes Step 4 above the step to take seriously with Log, as with any notetaker that assigns work.
If you use meetergo's AI assistant Mira, its credits add question-answering and follow-up drafts on top of your transcripts, so a corrected speaker map also improves what Mira answers when you ask who committed to what.
| Plan | Price |
|---|---|
Free | €0, local transcription, summaries, search, export |
PlanFree Price€0, local transcription, summaries, search, export | |
Light | €9.90/user/month, adds speaker diarization and custom templates |
PlanLight Price€9.90/user/month, adds speaker diarization and custom templates | |
Suite | €29.90/user/month, all apps in the suite |
PlanSuite Price€29.90/user/month, all apps in the suite | |
Premium | €49.90/user/month |
PlanPremium Price€49.90/user/month | |
Full plan details are on meetergo's pricing page, and processing locations are listed in the security overview.
Get transcripts with speaker labels, processed on your own machine.
Get transcripts with speaker labels, processed on your own machine.
Mistakes that put the wrong name on a decision
Trusting the summary without opening the transcript. A summary compresses away the evidence. If the transcript has a labelling error, the summary states it as fact, in cleaner prose.
Renaming labels line by line. Fixing each line as you read produces inconsistencies. Build the speaker map once in Step 2, rename globally, then fix the exceptions.
Assuming call-based labels work in the office. A hybrid meeting with a room system gives you named remote participants and one merged voice for everyone in the room. Review the room speakers first.
Enrolling voices without asking. Turning on voice recognition for your whole client list because the setting was there is the GDPR problem described above, created in two clicks.
Sharing the raw export. Send the version you reviewed, and say it was reviewed. The person reading it can then trust the names as much as the words.
If you are still choosing a tool, our comparison of AI notetakers and the roundup of meeting transcription software both note which tools label speakers at all.
FAQ
What is the difference between speaker diarization and speaker identification?
Diarization splits a recording into anonymous speakers, Speaker 1 and Speaker 2, based on how their voices differ. Identification attaches a real name to a voice, either from the video platform's participant list or from a stored voice profile. Most tools do diarization; fewer do identification.
Why does my transcript show the same person as two speakers?
The model decided their voice changed enough to be someone else. That usually happens when a person moves away from the microphone, switches from laptop to headset, or speaks much louder in one part of the meeting. Merge the two labels during your speaker map.
Can an AI transcript label speakers in an in-person meeting?
It can separate voices through diarization, but it cannot know names unless someone maps them or the tool has stored voice profiles. Accuracy also drops with microphone distance, so a phone in the middle of the table beats a laptop at one end.
How accurate are speaker labels in transcription?
It depends on the number of speakers, overlap, audio quality and how different the voices sound, so a single accuracy figure from a vendor says little about your meetings. Test a tool on your hardest recording: several people, one microphone, frequent interruptions.
Is it legal to store voice profiles for speaker recognition?
In the EU, a voiceprint used to recognise a specific person can be biometric data under the GDPR, which generally requires explicit consent and careful storage. Profiles kept on your own device reduce exposure but do not remove the consent question.
Can I add speaker labels to a transcript myself?
Yes. Run it through a plain transcription tool such as meetergo's online audio transcription tool, then add names by hand using the speaker map method from Step 2. For longer recordings, a notetaker with diarization saves most of that manual work. Our explainer on AI transcription covers the trade-offs.
