Voice AI Agents in 2026: Realtime vs Cascaded, 1.3s Latency, $0.05 a Minute

Most voice agent demos I watched this year sounded fine in a quiet room and failed the same way on a phone line: the agent talked over the caller, or left a silence long enough for them to say "hello?" twice. Voice AI agents in 2026 are mostly a latency and turn-taking problem wearing a model costume. The models got much better this year (OpenAI shipped three new realtime model generations and a full-duplex model, Google took Gemini Live to GA), but the hard engineering is still the 1.3 seconds between the caller stopping and the agent starting. This post explains the three architectures, where the milliseconds go, how turn detection works, what telephony adds, how LiveKit Agents and Pipecat compare, and what a minute actually costs.

What is a voice AI agent, and what changed in 2026?

A voice agent is an LLM agent (the same loop I describe in what AI agents are) with audio on both ends and a hard real-time deadline. There are now three ways to build one, and LiveKit's pipeline types page names them cleanly:

• Cascaded (STT, LLM, TTS): speech-to-text, a text LLM, then text-to-speech, streamed so the stages overlap.
• Turn-based realtime (speech-to-speech): one model hears audio and speaks audio, but waits for the user's turn to end. OpenAI's Realtime API and Gemini Live work this way.
• Full-duplex realtime: one model listens while it speaks and decides for itself when to take or yield the turn. OpenAI's GPT-Live is the first widely available one.

The pace of releases is the main thing that changed. From OpenAI's API changelog: the Realtime API went GA on 28 August 2025, gpt-realtime-mini arrived on 6 October 2025, GPT-Realtime-1.5 on 23 February 2026, GPT-Realtime-2 (with a streaming GPT-Realtime-Whisper transcriber) on 7 May 2026, the old Realtime beta was removed on 12 May, and GPT-Realtime-2.1 plus a 2.1 mini shipped on 6 July. GPT-Live launched inside ChatGPT on 8 July (VentureBeat) and went GA in the API on 10 September at $0.05 a minute. The current gpt-realtime-2.1 model page lists a 128,000-token context, 32,000 output tokens and configurable reasoning effort, with the honest caveat that higher effort "can increase latency".

Google moved on a similar curve. The Gemini API changelog shows gemini-2.5-flash-native-audio previews in September and December 2025, gemini-3.1-flash-live-preview on 26 March 2026, and Gemini 3.8 Live and 3.8 Live Extended Thinking generally available on 15 September 2026. The 3.8 Live model page gives 131,072 input tokens, makes asynchronous function calling the default and keeps proactive audio always on.

Cascaded STT-LLM-TTS vs speech-to-speech: which should you use?

The surprising answer from the vendor with the most to gain from realtime models is "cascaded, mostly". LiveKit's own docs say that "for most production agents, a cascaded pipeline is the right default", and the trade-offs they list match what I would write from first principles:

Dimension	Cascaded STT-LLM-TTS	Turn-based realtime	Full-duplex (GPT-Live)
End-to-end latency	Moderate	Fastest	Fastest
Listens while speaking	No	No	Yes
Turn detection	You configure it	You configure it	Model decides
Tool calling	Mature	Less mature	Delegated to a backend model
Scripted speech (exact text)	Yes	No	No
Hears tone and prosody	No	Yes	Yes
Audit trail	Full text at every stage	Limited	Limited

Sources: LiveKit pipeline types and LiveKit's GPT-Live docs.

The cascaded pipeline wins whenever you need to prove what was said. A bank, a clinic or anyone recording consent needs the transcript the LLM actually acted on, and the ability to make the agent read a disclosure word for word. Swapping one stage (a better STT for accented English, a cheaper TTS) is a config change, not a migration. You lose the caller's tone, and you pay for three hand-offs.

Speech-to-speech wins on feel. The model hears hesitation and sarcasm, and the output carries emphasis a TTS engine has to guess at. GPT-Live goes further: per LiveKit's integration notes it is server-driven (no client event can create or cancel a response), it decides its own barge-in, its context is append-only, and its reasoning and tool calls are delegated to a separate Responses model with its own instructions. You trade control for the most natural turn-taking available. GPT-Live also has no text-only output, so it cannot be paired with a separate TTS.

How much latency is acceptable for a voice agent?

Humans are fast. Stivers and colleagues timed how fast people answer questions across ten languages in PNAS in 2009: the overall mean gap was 208 ms, from 7 ms in Japanese to 469 ms in Danish, with a mode of zero. People start planning their reply while the other person is still talking. A pipeline that only starts working after the caller stops is already behind.

The most useful public breakdown of a real pipeline is the latency table in the Voice AI and Voice Agents primer, first written in February 2025 and updated in June 2026. Its sixteen stages sum to 1,293 ms, and it names 1,500 ms voice to voice as the target worth aiming for. Grouped, it looks like this:

Two stages are half the budget. LLM time to first token is the largest single number, which is why voice stacks favour small, fast models and why "reasoning effort" is a latency dial, not a free quality upgrade. Transcription plus endpointing is the second, and it is mostly waiting: the system cannot know the caller has finished until some silence has passed. The network rows look small, but you control them least once a call crosses a carrier, and they grow when the agent runs far from the caller. The same discipline I argue for in SRE for LLM apps applies: set a p95 voice-to-voice SLO per region, not an average.

Turn detection is the latency you choose

Voice activity detection (VAD) tells you when sound stops. It cannot tell you whether "I need to think about that for a moment" is the end of a turn. Every framework now layers a semantic or acoustic end-of-turn model on top, and the defaults are worth knowing because they are where your agent spends its silence.

In LiveKit Agents, the session waits between 0.5 and 3.0 seconds after speech by default. With LiveKit's audio turn detector, which hears intonation and rhythm as well as the words, without needing a transcript, it commits sooner, with a 0.3 s minimum and 2.5 s maximum. The full v1 model runs on LiveKit Inference; v1-mini runs locally on CPU and is free anywhere; both cover 14 languages. Silero VAD's default min_silence_duration is 0.55 s, and the turn detector refuses anything below 0.25 s.

Pipecat's default end-of-turn strategy is its own Smart Turn model, now v3.2: about 8M parameters, an 8 MB int8 CPU build, 23 languages, BSD-2-Clause, and roughly 65 ms per inference on a Pipecat Cloud instance. Pipecat's speech input docs pair it with Silero VAD at a 0.2 s stop_secs, short on purpose so the STT can finalise early while Smart Turn decides whether the caller is really done.

The other half of turn-taking is interruption. A naive agent stops talking whenever it hears "uh-huh". LiveKit's adaptive interruption handling is a model trained to tell a real barge-in from a backchannel, on by default for agents running on LiveKit Cloud. Full-duplex models make this the model's job entirely, which is either a relief or a loss of control depending on how much you trust it.

What does a voice AI agent cost per minute?

Per-minute numbers from vendors are honest but incomplete, because each one prices a different slice. Here is what I could verify on 31 October 2026:

Option	What you pay for	Rough cost per conversation minute	Source
OpenAI gpt-realtime-2.1	$32 per 1M audio tokens in, $64 out, $0.40 cached	$0.048 floor for a minute split 50/50; about $0.08 with history resent and 90% cached	OpenAI pricing, token rates
OpenAI gpt-realtime-2.1-mini	$10 in, $20 out per 1M audio tokens	about $0.015 floor	OpenAI pricing
OpenAI GPT-Live 1	$0.05 per minute, billed per second	$0.05 plus the backend model and tools	OpenAI changelog
Gemini 3.8 Live	$0.005 per minute audio in, $0.018 out	about $0.014 if the agent speaks half the minute	Gemini pricing
Deepgram Voice Agent API	Connection time, standard tier	$0.075	Deepgram pricing
Cascaded on LiveKit Cloud, phone call	Session, telephony, STT, LLM, TTS, observability	$0.0479 in LiveKit's own calculator example	LiveKit pricing

The "floor" rows deserve a warning. OpenAI counts user audio at one token per 100 ms and assistant audio at one per 50 ms, so a minute split evenly is 300 input and 600 output tokens. But the Realtime cost guide is explicit that the whole conversation is sent with every response, so later turns cost more. This script models a five-minute call at four exchanges a minute:

# realtime_cost.py: rough cost of one speech-to-speech call (Python 3.12, no deps)
# Rates: OpenAI pricing page, checked 2026-10-31 (USD per 1M tokens)
AUDIO_IN, AUDIO_CACHED, AUDIO_OUT = 32.00, 0.40, 64.00   # gpt-realtime-2.1
TEXT_IN, TEXT_CACHED = 4.00, 0.40
USER_TOK_PER_S, BOT_TOK_PER_S = 10, 20   # 1 token/100 ms in, 1 token/50 ms out

def call_cost(minutes=5, turns_per_min=4, user_s=7.5, bot_s=7.5,
              instructions_tok=1_000, cache_hit=0.9):
    total, history = 0.0, 0
    for _ in range(int(minutes * turns_per_min)):
        new_user = user_s * USER_TOK_PER_S
        reused = history                      # the whole conversation is resent each turn
        cached, fresh = reused * cache_hit, reused * (1 - cache_hit)
        total += (fresh + new_user) * AUDIO_IN + cached * AUDIO_CACHED
        total += instructions_tok * (TEXT_CACHED if history else TEXT_IN)
        total += bot_s * BOT_TOK_PER_S * AUDIO_OUT
        history += new_user + bot_s * BOT_TOK_PER_S
    return total / 1e6

for hit in (0.9, 0.0):
    c = call_cost(cache_hit=hit)
    print(f"cache_hit={hit:.0%}: ${c:.3f} per 5-min call, ${c/5:.4f} per minute")

# Output when I ran it:
# cache_hit=90%: $0.404 per 5-min call, $0.0808 per minute
# cache_hit=0%: $1.620 per 5-min call, $0.3239 per minute

The assumptions are mine, but the shape is the point: cache hit rate is the biggest cost lever in a long realtime call, and anything that rewrites early context (changing instructions mid-call, aggressive truncation) quietly multiplies the bill. On the cascaded side, LiveKit's example breaks down as $0.0100 agent session, $0.0100 telephony, $0.0014 LLM, $0.0075 STT, $0.0090 TTS and $0.0100 observability. Platform fees are a third of it. If you are modelling margins, put the transport and session line items in the spreadsheet, not just the models; my FinOps for AI post goes through why per-unit costs hide in exactly those rows.

Telephony: SIP, phone numbers and the 8 kHz problem

Most commercial voice agents answer phones, and the phone network still runs on SIP and 8 kHz audio. LiveKit's telephony stack turns each caller into a SIP participant in a room, with inbound and outbound trunks, dispatch rules that decide which agent joins, DTMF, cold and warm transfers, and SRTP. LiveKit lists it as tested with Twilio, Telnyx, Exotel, Plivo, Wavix, Sinch, didlogic and DIDWW. LiveKit also sells numbers directly, but only US numbers and only inbound for now, with one free local number on every plan. Pipecat's telephony docs cover dial-in and dial-out over PSTN and SIP through Daily, or media-stream WebSockets from Twilio, Telnyx, Plivo and Exotel.

You can also skip the framework. OpenAI's Realtime SIP guide has you point a trunk at sip:$PROJECT_ID@sip.api.openai.com;transport=tls, receive a realtime.call.incoming webhook, and accept the call with a POST that sets the model and voice. Signalling is TLS on port 5061 and media is SRTP. It is the shortest path to a working phone agent and the least portable.

Narrowband audio is the quiet tax. A G.711 call carries 8 kHz audio, so STT accuracy drops and the turn detector hears less intonation. Silero supports 8 and 16 kHz sampling; check that every stage in your chain accepts what the carrier sends rather than resampling twice.

LiveKit Agents vs Pipecat in 2026

Both are open source and Python-first. They differ in shape, not quality:

• LiveKit Agents (GitHub, Apache-2.0, about 15k stars) is at 1.8.10, released 28 October 2026, with a Node.js SDK alongside. An agent is a worker that gets dispatched into a WebRTC room; telephony, the turn detector, LiveKit Inference (one key for many STT, LLM and TTS providers) and LiveKit Cloud hosting are all part of one product.
• Pipecat (GitHub, BSD-2-Clause, about 17k stars) reached 1.0.0 on 14 April 2026 and 1.15.0 on 29 October 2026. It is a frame pipeline you assemble yourself, transport-agnostic (Daily, LiveKit, raw WebRTC, telephony WebSockets), with Pipecat Cloud as an optional host. On 29 October 2026 Deepgram announced it is acquiring Daily, Pipecat's creator, and says Pipecat stays BSD-licensed under a new independent Pipecat Foundation.

If I want phone numbers, rooms and hosting from one vendor, LiveKit is less glue. If I want to own the transport or run inside an existing telephony stack, Pipecat's pipeline is easier to reshape. The step-by-step guide builds the same agent in both.

FAQ

What is the difference between a cascaded voice agent and speech-to-speech?

A cascaded agent chains speech-to-text, a text LLM and text-to-speech, so every stage produces inspectable text. A speech-to-speech model such as gpt-realtime-2.1 or Gemini 3.8 Live consumes and produces audio directly, which is faster and more expressive but harder to audit and script.

What latency should a voice AI agent target?

Humans average about 208 ms between turns. A realistic cascaded pipeline lands around 1.2 to 1.3 seconds voice to voice, and the primer's target is 1,500 ms. Measure p95 per region, since the network and endpointing stages vary most.

How much does a voice AI agent cost per minute in 2026?

At list prices, roughly $0.014 to $0.08 a minute for speech-to-speech models depending on model and cache hit rate, $0.05 a minute plus backend model for GPT-Live, and about $0.05 all-in for a cascaded phone call in LiveKit's own example. Telephony and platform fees are often a third of the total.

Is LiveKit or Pipecat better for voice agents?

Neither is better in general. LiveKit bundles rooms, SIP, phone numbers, turn detection and hosting; Pipecat is a transport-agnostic pipeline that is easier to embed in your own infrastructure. Both are open source and current.

This quarter I would build cascaded first, with LiveKit's turn detector or Smart Turn, a small fast LLM, and per-turn latency metrics exported from day one. I would run a realtime model behind a flag on the same agent, compare p95 voice-to-voice latency and cost per call on real traffic, and only switch where callers measurably finish more tasks. The models will change again before Christmas; the latency SLO, the transcript and the cost per call are what I would actually keep.

Related: SRE for LLM Apps in 2026: SLOs for TTFT, Provider Outages and Evals, AI Agent Frameworks Compared (2026): LangGraph, CrewAI, ADK and 7 More and LLM Inference in 2026: Prefill Is a Different Machine Now, and the KV Cache Has a Disk Tier.