Rime's Prism Hears the Caller but Writes Its Reply Down First, So Guardrails Can Read It

Voice-agent builders have had to choose between two designs. A cascade of speech recognition, a language model and text-to-speech can be inspected and controlled but loses tone of voice. A fused speech-to-speech model hears everything but gives no chance to read a reply before it is spoken. Rime, a San Francisco voice-AI company founded in 2022, says Prism, launched on October 28, 2026, removes that trade-off. The model listens to raw caller audio, writes out what it intends to say, and then speaks while it keeps listening.

In her launch post, Rime CEO and co-founder Lily Clifford admits that 18 months ago she said “speech-to-speech models were a decade out from real use.” She says Rime then spent six months building Prism and has “staked Rime’s entire roadmap” on it.

A text checkpoint inside one model

Rime describes Prism as trained end-to-end to do three jobs: listen, write what it is going to say, and keep hearing the user while it speaks. “In Prism the text is a checkpoint,” Clifford writes. The decision is made over the audio, so frustration, laughter or background noise can still shape the reply, but business rules can read every word before it becomes speech.

Rime says the same checkpoint carries rewritable text for action and content tags, phonemic conditioning for names and account terms, and tool management. Its documentation says each spoken reply is sent with its text, for captions and logs. Tool calls run alongside speech, so the agent can say something like “Let me check that” while the backend does the lookup.

According to the docs, one WebSocket connection replaces separate speech-to-text, language-model and text-to-speech services. Prism decides for itself when a caller has finished, and stops when it is interrupted. Rime tells builders not to add their own turn detector or voice-activity detection.

Rime’s benchmark and trial numbers

All performance claims here are Rime’s. The company says it ran Prism on the harness Artificial Analysis uses for its speech-to-speech leaderboard, with no reasoning step before speaking. On τ-Voice, a benchmark of simulated customer-service calls in airline, retail and telecom, Rime reports a score of 56.6. It says that puts Prism 17 to 30 points ahead of every non-reasoning realtime model from OpenAI and Google, and ahead of GPT-Realtime-2.1 even with reasoning turned up. In the telecom domain, Rime reports 82.5 against 45.6 for GPT-Live-1.

Rime puts time to first audio at 0.57 seconds, which it says is about half the wait of GPT-Live-1, GPT-Realtime-2.1 and Gemini 3.8 Live. The two models it says score higher on τ-Voice reason before speaking and take 1.24 to 1.35 seconds to start.

Fine-tuning is the other pillar. Rime says it took 10 hours of production calls, tool traces and transcripts from Revmo, which runs an ordering agent on OpenAI today, built a reinforcement-learning environment around them and trained a custom Prism. In Clifford’s post, 94% of orders were placed only after the caller confirmed, against 57% for OpenAI. Rime’s press release describes the same trial with different measures: order and pickup-method confirmations rising from 68% to 100%, and calls without dead air from 66% to 88%. Self-serve RL and supervised fine-tuning APIs are still being built with beta customers.

Limits listed in the docs

Prism’s documentation sets out what the launch version does and does not do:

• Language and voices: English only, with four Prism-specific voices (aurora, wawona, brando and astra). Rime’s text-to-speech voices do not work with it.
• Audio format: headerless 16-bit PCM, mono, 16 kHz in and 24 kHz out. Phone audio needs a bridge, because Prism does not accept mu-law or G.711.
• Session limits: a 65,536-token context, a 55-second cap on one continuous caller turn, a 5-minute idle timeout, and no automatic resume after a dropped connection.
• Not configurable: text-only replies, sampling settings such as temperature, or a choice of underlying language model.
• Endpoint: a WebSocket API in US East, with SDKs for Python, Node.js, Go and Rust, plus LiveKit and Pipecat quickstarts.

Prism uses the event names of OpenAI’s Realtime beta interface, so Rime says existing Realtime clients need only a few changes.

The press release goes further than the docs on deployment. It says brands can run Prism “entirely within their own infrastructure,” on-premises or in a private cloud, while remaining SOC 2 and HIPAA compliant. The documentation describes only the hosted endpoint. Dialpad is named as a design partner planning to roll Prism out across its agentic products.