Baseten Puts the Whole Voice-Agent Cascade in One Cluster and Prices Its Realtime API Near a Cent a Minute

Baseten opened a private preview of its Voice Agent Stack and Baseten Realtime Voice on November 16, 2026. The stack co-locates speech-to-text, the language model, text-to-speech and the orchestrator on Baseten's dedicated inference, while Realtime Voice speaks the OpenAI Realtime protocol and is priced at about $0.01 per audio minute at scale. All benchmarks are company-reported.

Three switchboard operators in dusty pink, bottle green and ivory sit shoulder to shoulder at one compact switchboard, passing a single patch cord hand to hand.
Three switchboard operators in dusty pink, bottle green and ivory sit shoulder to shoulder at one compact switchboard, passing a single patch cord hand to hand.

Baseten is betting that the fastest way to build a phone-grade voice agent is to keep every model in the same room. On November 16, 2026, the inference company launched the Baseten Voice Agent Stack, infrastructure for running production voice agents, together with Baseten Realtime Voice, a ready-made voice agent API built on top of it. Both are in private preview with a waitlist.

Most voice agents today are cascades: speech-to-text (STT) turns the caller’s audio into words, a language model decides what to say, and text-to-speech (TTS) speaks the reply. When those three models, and the orchestrator that coordinates them, live with different vendors or in different regions, each turn picks up network delay. Baseten says that even with all three models hosted on its platform, a remote orchestrator “still adds 100s of milliseconds of network round trips per turn.” Its answer is to run the orchestrator alongside the models on a tight in-cluster network.

Each model scales on its own concurrency target

The Voice Agent Stack runs on Baseten Dedicated Inference. Baseten lists inference runtime tuning, networking between models, autoscaling, cross-cloud failover and post-training support as the building blocks. Each component scales on its own concurrency target, which the company says helps because STT, the language model and TTS often saturate at different rates. Scaling only the busy component keeps GPU use high and cost per minute lower.

The stack is model-agnostic. Developers can pick any STT, language model and TTS combination they can host on Baseten, mix in third-party APIs, and switch models during a call. The documentation says a session can change the LLM, voice or STT route without a redeploy, and that the LLM can be a Baseten deployment, a Model APIs model or an outside OpenAI-compatible endpoint.

Because the models are owned rather than rented, they can also be post-trained. Baseten describes tuning STT on industry jargon for fields like healthcare and financial services, post-training the language model on real call behaviour, and supervised fine-tuning recipes for TTS voices.

Realtime Voice, the reference agent

Realtime Voice is Baseten’s demonstration of the stack. In the setup measured in the launch post, it uses Qwen3-ASR for transcription on one RTX 6000, Gemma-4-31B as the conversational model on one B200, Qwen3-TTS for speech on another RTX 6000, and a GLM-5.3 Fast “delegate” model on Baseten’s Model APIs. The orchestrator is a custom build based on Pipecat.

The delegate setup splits the work. A fast, co-located “talker” model keeps the conversation going while a stronger “thinker” model takes over reasoning, web search and tool calls in the background. Before a slow tool call, the docs say, the agent speaks a short acknowledgement so the line does not go quiet.

Turn-taking combines voice activity detection, an end-of-turn model and speculative replies: the agent starts preparing an answer when the caller pauses and sends it if the turn really has ended. Callers can interrupt, and short sounds like “mm-hmm” do not cancel a reply.

The API implements the OpenAI Realtime event protocol over a single WebSocket, and the documentation includes a working example using the OpenAI Agents SDK. Baseten says this lets existing Realtime agents move over with minimal changes, while adding features it says many closed APIs lack, such as keyword boosting, configurable acknowledgements and voice cloning. The docs carry two caveats: the openai Python SDK cannot connect directly because it appends /realtime to the URL, and browsers must relay through a backend because they cannot set the Authorization header.

The numbers Baseten reports

All of the following are Baseten’s own measurements:

  • Latency: with context up to 16K tokens, median turn latency stays around one second from 1 to 56 concurrent sessions; median time to first audio rose from 0.94 to 1.14 seconds over that range.
  • Turn-taking: 93% weighted on the Artificial Analysis split of Full-Duplex-Bench, which Baseten says is the highest among open-source full-duplex systems.
  • Task success: 73.5% on τ²-Bench Voice across airline, retail and telecom, against 31.2% for a cascaded baseline. Baseten notes this score is self-reported on Sierra’s harness and not directly comparable with Artificial Analysis results.
  • Cost: about $0.012 per audio minute at 32 concurrent sessions on a fully utilised dedicated deployment, excluding per-token delegation costs. At 56 sessions, Baseten estimates $0.008 per call-minute including delegation.

Baseten says that from 32 concurrent streams upward, its cost per minute is lower than every hosted speech-to-speech API it compared. Those figures assume a dedicated deployment kept fully busy.

Next on Baseten’s list are more complex orchestration flows, easier migration of existing pipelines, and exploration beyond cascades, including native speech-to-speech models and omni models paired with dedicated TTS heads.

Topics
  • #Baseten
  • #Voice Agents
  • #Speech-to-Text
  • #Text-to-Speech
  • #OpenAI Realtime
Raj M

Author

Raj M

Contributor

AI Systems Architect is a seasoned technology leader with over 15 years of experience in the IT industry working with Fortune 500 companies. With a solid foundation in multi-agent systems, open-source LLM infrastructure, and enterprise deployment, he excels at building scalable production-grade AI platforms.