Gemini 3.5 Transcribe Ships 2026: Speaker-Aware Notes

Google shipped Gemini 3.5 Transcribe this week, and it kills one of the most annoying duct-tape stacks in applied AI: running Whisper for text, then a separate diarization model to figure out who said what, then a third pass to line the timestamps back up. Gemini 3.5 Transcribe does all three in one call — speaker labels, word-level timestamps, and mixed-language audio — and returns structured output you can drop straight into a database. If you run meeting notes, interview workflows, or podcast repurposing, the cost and accuracy math you settled on six months ago just changed. Here is what shipped, what it breaks, and how to wire it up today.

What’s new about Gemini 3.5 Transcribe

Gemini 3.5 Transcribe is a dedicated speech-to-text model in the Gemini family, not a general multimodal model that happens to accept audio. That distinction matters more than the marketing implies. Previous Gemini models could transcribe audio, but they treated it as one more modality inside a general reasoning budget. Long files got expensive, timestamps drifted, and speaker attribution was a best-effort guess embedded in prose. The dedicated model is priced and tuned for transcription specifically, with a single-pass pipeline that emits diarized, timestamped segments as its native output format.

The headline capability is native AI speaker diarization. Instead of clustering voice embeddings and hoping the segment boundaries match the word boundaries, the model attributes speech as part of transcription itself. It uses linguistic context — turn-taking cues, someone saying “thanks, Priya,” a question followed by an answer — to resolve the cases pure audio clustering gets wrong. Crosstalk and short interjections (“right,” “mhm,” “wait, no”) are where bolted-on diarization historically falls apart, and where a language-aware model has a structural advantage.

The second big change is multi-language handling inside a single file. Code-switching mid-sentence is common in real interviews, support calls, and international standups. It used to force you to pick one language hint and accept garbage on the rest, or chunk the audio and run language detection per chunk. Gemini 3.5 Transcribe handles the switch inline and can optionally return a translated track alongside the verbatim one. Combined with structured JSON output and a long audio context window, that removes most of the glue code in a typical Gemini API audio transcription pipeline.

Why it matters

• Your diarization dependency disappears. If you run pyannote or a hosted diarization service alongside Whisper, that is an entire component — with its own GPU, its own version pins, and its own failure mode — you can delete.
• Fewer alignment bugs. Merging two systems’ timestamps is where most transcription pipelines silently corrupt data. One pass means one timeline, so speaker turns and word offsets stay consistent by construction.
• Cost math flips for hosted stacks. Self-hosted Whisper is nearly free at the margin but expensive in ops time. Hosted Whisper plus hosted diarization is two line items. A single API call at transcription-tier pricing often beats the combined stack once you count the GPU you were renting.
• Structured output means less parsing. Ask for a JSON schema of speaker, start, end, and text and you get rows you can insert directly — no regex over SRT files, no fragile VTT parsing.
• Non-English and mixed-language workflows get usable. Teams that gave up on automated notes for bilingual meetings should re-test; this is the specific failure case the model targets.
• Downstream summarization gets better inputs. Speaker-attributed transcripts let an action-item extractor assign owners. “Someone said they would send the deck” is useless; “Marcus said he would send the deck” is a task.

How to use Gemini 3.5 Transcribe today

1. Get a key and install the SDK. Grab an API key from Google AI Studio, then install the current Google GenAI SDK. The older google-generativeai package is on its way out — use google-genai.

   pip install -U google-genai
   export GEMINI_API_KEY="your-key-here"

2. Upload the audio file. Inline base64 works for short clips, but anything over about 20 MB should go through the Files API. Uploaded files persist for 48 hours, which is plenty for a batch job.

   from google import genai
   
   client = genai.Client()
   
   audio = client.files.upload(file="standup-2026-08-26.m4a")
   print(audio.name, audio.mime_type)

3. Request a diarized transcript with a response schema. This replaces your entire post-processing layer. Define the shape you want and let the model fill it in.

   from pydantic import BaseModel
   
   class Segment(BaseModel):
       speaker: str
       start: str   # "MM:SS" or "HH:MM:SS"
       end: str
       text: str
       language: str
   
   class Transcript(BaseModel):
       segments: list[Segment]
   
   prompt = """Transcribe this audio verbatim.
   Label each distinct speaker as Speaker 1, Speaker 2, etc.
   If a speaker states or is addressed by name, use that name instead.
   Include filler words. Do not summarize or clean up grammar.
   Mark the language of each segment with a BCP-47 tag."""
   
   resp = client.models.generate_content(
       model="gemini-3.5-transcribe",
       contents=[audio, prompt],
       config={
           "response_mime_type": "application/json",
           "response_schema": Transcript,
       },
   )
   
   data = resp.parsed
   for seg in data.segments:
       print(f"[{seg.start}] {seg.speaker}: {seg.text}")

4. Pin the speaker count when you know it. Diarization errors cluster around over-splitting — one person’s voice split across three labels because they moved closer to the mic. If you know how many people were in the room, say so.

   prompt = """Transcribe this audio. There are exactly 3 speakers.
   Do not create more than 3 speaker labels.
   Speaker 1 is the interviewer; Speakers 2 and 3 are guests."""

5. Hit it from curl if you are not in Python. Useful for a quick sanity check or a shell-based batch job.

   curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.5-transcribe:generateContent" \
     -H "x-goog-api-key: $GEMINI_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "contents": [{
         "parts": [
           {"file_data": {"mime_type": "audio/mp4", "file_uri": "FILE_URI_HERE"}},
           {"text": "Transcribe with speaker diarization and timestamps. Return JSON."}
         ]
       }],
       "generationConfig": {"response_mime_type": "application/json"}
     }'

6. Chain a second call for the deliverable. Keep transcription and summarization as separate calls. The transcript is your durable artifact; the summary is disposable, and you will want to regenerate it with different prompts.

   notes = client.models.generate_content(
       model="gemini-3.5-flash",
       contents=f"""Here is a speaker-labeled meeting transcript.
   Produce: (1) a 5-bullet summary, (2) decisions made,
   (3) action items as "OWNER — TASK — DUE".
   Only assign an owner if the transcript names them.
   
   {data.model_dump_json()}""",
   )
   print(notes.text)

7. Spot-check before you trust it. Pull three random 30-second windows per hour of audio and verify speaker labels against the source. Word error rate is the easy metric; diarization error rate is the one that embarrasses you in front of a client.

How it compares: Gemini 3.5 Transcribe vs Whisper and the rest

Capability	Gemini 3.5 Transcribe	OpenAI Whisper (self-hosted)	Whisper API + pyannote	AssemblyAI / Deepgram
Speaker diarization	Native, single pass	None — separate model required	Separate model, manual alignment	Native, mature
Word-level timestamps	Yes	Yes, with extra flags	Yes	Yes
Mid-file language switching	Handled inline	Poor — one language hint per run	Poor	Varies by tier
Structured JSON output	Schema-enforced	Parse it yourself	Parse it yourself	Native JSON
Context-aware speaker naming	Yes — infers names from dialogue	No	No	Limited
Ops burden	One API call	GPU, model weights, queueing	Two services to keep in sync	One API call
Marginal cost	Per-minute API pricing	Near-zero compute, high ops	Two line items	Per-minute API pricing
Runs offline	No	Yes	No	No

The honest read: if you already have a working self-hosted Whisper box and you only transcribe single-speaker English audio, there is no reason to move. Whisper remains the right answer for offline, air-gapped, or privacy-constrained work, and nothing here changes that. The case for switching is specifically multi-speaker and multi-language audio, where you currently maintain a second diarization service that produces mediocre results. Against AssemblyAI and Deepgram — both of which have shipped good diarization for years — the differentiator is the language model underneath: context-aware speaker naming and mid-sentence code-switching are things a pure acoustic pipeline cannot do.

What’s next

The obvious next shoe to drop is streaming. Everything described here is batch: upload a file, wait, get a transcript. Live diarized transcription over a WebSocket is the version that unlocks real-time meeting assistants, and it would let Google compete directly with the live-captioning products built on Deepgram. Watch for a streaming endpoint on this model family — that announcement would make this a platform shift rather than a component swap.

The second thing to watch is speaker identity persistence across files. Right now every transcript starts fresh: “Speaker 1” in Monday’s standup has no relationship to “Speaker 1” in Tuesday’s. Voice enrollment — register your team once, get consistent named attribution forever — turns an AI podcast transcription workflow from a per-episode chore into an actual pipeline. Several competitors already offer some version of this, so it is a reasonable bet for the next release rather than a moonshot.

Finally, watch how this lands in Vertex AI and Google Workspace. A dedicated transcription model inside Meet and Docs would put automated speaker-aware notes in front of millions of people who will never touch an API. That is both the biggest distribution story here and the reason anyone building a thin wrapper around meeting transcription should think hard about their durable differentiator. Build on the transcript, not the transcription.

Frequently Asked Questions

Is Gemini 3.5 Transcribe more accurate than Whisper?

On raw word error rate for clean single-speaker English audio, the two are close enough that the difference rarely matters downstream. The gap opens on multi-speaker audio, where Gemini 3.5 Transcribe vs Whisper is not a fair comparison — Whisper does not do diarization at all, so you are comparing an integrated system against a two-model pipeline whose errors compound. Benchmark on your own audio; accents, recording quality, and domain vocabulary swing results more than any published leaderboard.

How many speakers can it distinguish?

It handles typical meeting and interview sizes reliably and degrades gracefully as the count climbs. Accuracy drops with heavy crosstalk regardless of speaker count. If you know the number of participants, state it in the prompt — constraining the label space is the single highest-leverage thing you can do for diarization quality.

Can I use it for real-time meeting transcription?

Not yet in the way most people mean. The current model is file-based, so a “live” experience means chunking audio and transcribing in near-real-time batches, which breaks speaker consistency across chunk boundaries. For post-meeting notes it excels; for live captions, stick with a purpose-built streaming provider until a streaming endpoint ships.

What audio formats and lengths does it accept?

Standard formats — WAV, MP3, M4A, FLAC, OGG — work fine. Use the Files API for anything over roughly 20 MB rather than inlining base64. For very long recordings, chunk on natural silence boundaries rather than fixed intervals, and overlap chunks by a few seconds so you can stitch speaker labels across the seam.

How do I keep speaker labels consistent when I have to chunk audio?

Pass the last minute of the previous chunk’s transcript as context in the next call, along with the speaker labels already assigned, and instruct the model to reuse them. This is imperfect, and it is the main reason to keep files whole when you can. Sanity-check by counting distinct labels across the full file — a five-person meeting that produced eleven speaker labels tells you the stitching failed.

Is my audio used to train Google’s models?

On the paid API tier, no — Google states that paid-tier data is not used for training. The free tier differs, and free-tier data may be used to improve the product. If you handle client interviews, medical audio, or anything under NDA, use the paid tier, confirm the current terms yourself, and put a data processing addendum in place before you upload anything.