Sesame’s Maya Voice API Opens 2026: Real Latency Tested

Sesame AI opened the CSM engine behind its viral Maya and Miles demos to outside developers. If you tried the browser demo last year and came away unsettled by how human the pauses felt, that same stack is now something you can call from your own code. The Sesame Maya voice API lands in the middle of a news cycle dominated by data-retention fights and teen-safety hearings at the big labs, which is exactly why it is being undercovered. For anyone building a voice agent on a small team, this launch changes what you can ship this quarter.

What’s new about the Sesame Maya voice API

Sesame — the Brian Iribe-backed company that came out of the Oculus orbit with a hardware ambition and a voice model to match — has spent most of its public life as a demo. You talked to Maya in a browser tab, it interrupted you naturally, you told three friends about it. The company open-weighted a 1B-parameter version of CSM (Conversational Speech Model) earlier, which proved the architecture but could not reproduce the demo: the small checkpoint is a research artifact, not the production voice. What is new is third-party access to the full hosted stack — the same model, the same speech pipeline, the same conversational context handling that made the demo feel alive.

The technical claim that matters is full-duplex operation at sub-300ms perceived latency. Traditional voice agents run a relay race: speech-to-text, then an LLM, then text-to-speech. Each leg adds buffering and each handoff throws away information. Prosody, hesitation, and emotional tone all die at the STT boundary. The Sesame AI CSM model is speech-to-speech: audio tokens in, audio tokens out, with the language modeling happening in the same token space. That is why the model can start responding before you finish, back off when you talk over it, and produce a laugh that lands in the right place rather than a laugh emoji rendered as sound.

Developer access means a WebSocket streaming endpoint for real-time sessions, a REST endpoint for non-realtime generation, and context injection so the model knows who it is and what it is allowed to say. Voice selection covers the Maya and Miles personas plus a set of neutral options, and conversational context carries across turns rather than getting re-sent as a transcript blob. Pricing runs per-minute of audio rather than per-token, which changes the cost math for anyone modeling voice agents on LLM token pricing.

Why it matters

• Latency stops being the reason your voice agent feels fake. The gap between “impressive tech demo” and “I forgot I was talking to software” is roughly 300ms of turn-taking latency. Below that threshold, users stop performing for the machine and just talk. A low latency voice agent is no longer a research budget item.
• Small teams get parity on the hardest part of the stack. Building a full-duplex speech to speech pipeline in-house means solving VAD, barge-in, echo cancellation, and streaming synthesis simultaneously. That is a year of specialist work you now rent by the minute.
• Voice AI for small business becomes economically boring. Per-minute pricing means a receptionist agent handling 400 calls a month has a cost you can put on a napkin, not a spreadsheet with a token-estimate tab.
• The STT-LLM-TTS architecture is now legacy. If you built on that chain in 2025, you have a migration decision. Not urgent, but real — the quality ceiling of a relay pipeline sits below a speech-native model’s floor.
• Competitive pressure lands on ElevenLabs and OpenAI’s Realtime API. Sesame vs ElevenLabs is now a genuine evaluation rather than a rhetorical question, and that pressure shows up as price cuts and feature releases within a quarter.
• Emotional realism is a product risk, not just a feature. A model that sounds like it cares will be trusted like it cares. Disclosure requirements and abuse surfaces are your problem now, at your scale, with your legal exposure.

How to use the Sesame Maya voice API today

1. Get credentials and confirm access. Sign up for developer access, generate a key, and put it in your environment rather than your source tree.

   export SESAME_API_KEY="sk-sesame-..."
   
   curl https://api.sesame.com/v1/voices \
     -H "Authorization: Bearer $SESAME_API_KEY"

2. Run a non-realtime generation first. Before you wire up streaming, verify the key works and hear what the voice sounds like on your own copy. This is the cheapest possible smoke test.

   curl https://api.sesame.com/v1/speech \
     -H "Authorization: Bearer $SESAME_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "model": "csm-1",
       "voice": "maya",
       "input": "We are open until six today. Want me to check if that slot is still free?",
       "format": "wav"
     }' --output test.wav

3. Open a streaming session. The real capability is the WebSocket. You send audio frames up, you get audio frames down, and both directions stay open — that is what full-duplex means in practice.

   import asyncio, json, websockets
   
   URL = "wss://api.sesame.com/v1/realtime?model=csm-1"
   
   async def main():
       async with websockets.connect(
           URL, additional_headers={"Authorization": f"Bearer {KEY}"}
       ) as ws:
           await ws.send(json.dumps({
               "type": "session.update",
               "session": {
                   "voice": "maya",
                   "input_audio_format": "pcm16",
                   "output_audio_format": "pcm16",
                   "sample_rate": 24000,
                   "turn_detection": {"type": "server_vad", "silence_ms": 200},
                   "instructions": "You are the front desk for Rivera Dental. "
                                   "Be brief. Never quote prices. If asked about "
                                   "billing, offer to take a message."
               }
           }))
           async for raw in ws:
               evt = json.loads(raw)
               if evt["type"] == "response.audio.delta":
                   play(evt["delta"])          # base64 PCM chunk
               elif evt["type"] == "input.speech_started":
                   stop_playback()             # user barged in — yield the floor
   
   asyncio.run(main())

4. Handle barge-in correctly or nothing else matters. The most common failure in conversational voice AI API integrations is a client that keeps playing buffered audio after the user starts talking. Clear your playback buffer on the speech-started event. Do not wait for a server confirmation.

   def stop_playback():
       audio_out.stop()
       audio_out.clear_buffer()   # drop everything already queued
       ws_send({"type": "response.cancel"})

5. Measure your own latency, not the vendor’s. Marketing latency is model inference time. User-perceived latency includes your network hop, your audio buffer, and your playback stack. Log the delta between last-user-audio-frame and first-agent-audio-frame.

   t_user_stop = None
   t_agent_start = None
   
   # on input.speech_stopped
   t_user_stop = time.perf_counter()
   
   # on first response.audio.delta of a turn
   t_agent_start = time.perf_counter()
   print(f"turnaround: {(t_agent_start - t_user_stop) * 1000:.0f} ms")

   Run this over fifty real turns on real network conditions, then look at p95 rather than the median. A 180ms median with a 900ms p95 feels worse to users than a flat 400ms.

6. Constrain the persona in system instructions, tightly. A speech-native model with a warm voice will improvise if you let it. Write instructions the way you would write a script for a temp worker on day one.

   You are the scheduling assistant for {business}.
   Scope: hours, location, availability, and taking messages.
   Out of scope: pricing, medical advice, refunds, anything legal.
   When out of scope, say: "I'll have someone call you back on that."
   Keep responses under two sentences unless reading back an appointment.
   If the caller asks whether you are a person, say you are an AI assistant.

7. Add a fallback path before you launch. Voice failures are loud — a dead line is worse than a slow chatbot. Health-check the socket, and route to voicemail or a human number on disconnect.

How it compares

Capability	Sesame CSM	OpenAI Realtime	ElevenLabs Agents	STT + LLM + TTS chain
Architecture	Speech-to-speech, single model	Speech-to-speech	Orchestrated pipeline with proprietary TTS	Three vendors, three hops
Typical turnaround	Sub-300ms target	~300-500ms	~500-800ms	800ms-1.5s
Barge-in quality	Native, model-level	Native, server VAD	Good, orchestration-level	You build it
Voice variety	Limited persona set	Small fixed set	Very large library plus cloning	Whatever your TTS offers
Emotional prosody	Strongest — carries context across turns	Good	Good on synthesis, lost at STT	Poor
Tool calling / functions	Early, limited	Mature	Mature, built for agents	Full control
Self-hosting	1B open checkpoint only	No	No	Yes, if you use open models

The honest read: if your product’s value is how the conversation feels — companionship, coaching, front-desk warmth, anything where a stiff agent kills the experience — Sesame is now the strongest option. If your value is what the agent does — booking, lookups, multi-step tool use across your backend — OpenAI’s Realtime API and ElevenLabs still offer more mature function-calling and integration surface. Sesame vs ElevenLabs is not a winner-take-all comparison yet; it is a question about which half of the problem is harder for you.

What’s next

Watch tool calling first. A speech-to-speech model that cannot reliably check your calendar is a conversation partner, not an agent, and the gap between those two covers most of the commercial market. Sesame’s function-calling story is early. If it matures over the next two quarters, the company competes for every voice-agent deal. If it does not, Sesame becomes the voice layer that other orchestration platforms wrap — still valuable, much less defensible.

The second thing is hardware. Sesame has been open about building lightweight always-on glasses, and the API is not the endgame — it is distribution and a data flywheel for a companion device. That matters to you as a developer in a specific way: the pricing and rate limits of a company subsidizing an ecosystem look very different from those of a company monetizing an API as its core business. Build with a thin abstraction layer over the transport so you are not rewriting your audio stack if terms shift.

Third, expect the regulatory surface to arrive fast. Emotionally convincing synthetic voices sit directly in the path of disclosure rules, consent-to-record laws that vary by state, and platform policies on companion AI. Build the “I’m an AI assistant” disclosure into your system instructions now, log consent where calls are recorded, and keep your persona definitions in config rather than hard-coded — the compliance requirement you have not read yet will be easier to satisfy if the answer is a config change.

Frequently Asked Questions

Is the Sesame Maya voice API the same model as the open-weights release?

No. The open checkpoint is a smaller research version of CSM that demonstrates the architecture but does not reproduce the hosted demo’s quality, latency, or conversational context handling. If you cloned the repo and were disappointed, that is why. The hosted API is the production model.

What does sub-300ms latency actually mean for users?

It is roughly the turn-taking gap in relaxed human conversation. Above about 500ms, listeners register a pause and adjust their speech to accommodate the machine — slower, more clipped, more “command-like.” Below 300ms, most people stop compensating and just talk. This figure covers model turnaround only; your network and audio buffers add to it, which is why you should measure your own p95.

Can I clone a specific person’s voice with it?

Not through the current developer offering, which ships defined personas rather than arbitrary cloning. If voice cloning is a hard requirement — matching a founder’s voice, localizing a brand — ElevenLabs remains the more direct path. Either way, get written consent from the person whose voice you are reproducing.

How does per-minute pricing compare to token pricing?

Per-minute billing is easier to forecast and generally friendlier to conversational workloads, because a long thoughtful reply and a long rambling one cost the same. Model your costs on average call duration times call volume, then add a buffer for silence and hold time, which still bills. For most voice AI for small business use cases this lands well below the cost of the human hours it displaces.

Do I need to rebuild my existing STT-LLM-TTS agent?

Not immediately. Run both in parallel on a slice of real traffic and compare completion rates and call duration rather than trusting a demo. If your agent’s job is transactional and it already works, the case for migration is weak. If users hang up on it, the case is strong — and a speech-native model addresses the specific reason they hang up.

What is the biggest implementation mistake to avoid?

Mishandled barge-in. Teams ship a low latency voice agent that generates fast but keeps playing queued audio after the user interrupts, which makes the agent feel like it is talking over people. Clear the playback buffer on the speech-started event immediately and cancel the in-flight response. That single fix accounts for most of the difference between a demo that impresses and one that annoys.

Disclosure: some tool names above are affiliate links. If you sign up through one we may earn a commission at no extra cost to you. We only link tools we actually cover in our guides.