Krisp Voice Isolation Review: Fixing the Second-Voice Problem in Voice AI Agents

Speech-to-text handles noise well, but a second person talking still ends up in the transcript.

Emmanuel Mumba

Emmanuel Mumba

19 November 2026

Krisp Voice Isolation Review: Fixing the Second-Voice Problem in Voice AI Agents

Apidog for Enterprise

On-Premises Deploy

SSO & RBAC

SOC 2 Compliant

Explore Apidog Enterprise

When a voice agent fails on real calls, the LLM or the voice is often not the cause. The problem starts one step earlier, in the audio the agent hears, whether it goes into speech-to-text (STT) or straight into a speech-to-speech model. Modern STT handles traffic and air conditioning fine. What it can't handle is a second person talking: a colleague, a TV, the agent at the next desk in a call center. It listens to them too, and the agent acts on words the caller never said.

Krisp Voice Isolation sits in front of the STT and removes every voice except the main speaker's. For this review I read Krisp's benchmark and docs, signed up for the self-serve API, and ran eight real recordings from Krisp's public dataset through it. Then I transcribed the before and after audio with an open-source STT model to see what actually changed.

The short verdict: if your agent takes calls from offices, homes, call centers or anywhere people talk in the background, Voice Isolation is worth putting in your pipeline. In my test, background speech that had become 331 words of fake transcript dropped to zero. The trade-offs are real but small: a slight cost on clean phone audio, a US-only API region, and fewer models on the API than in the SDK.

The problem: background voices, not background noise

STT models are trained to capture speech, not to decide whose speech matters. Noise can be filtered because it doesn't look like speech. A second voice looks exactly like speech, because it is.

Krisp's own example is a good one. A caller asks to cancel a subscription while a colleague nearby asks about lunch. STT returns one stream of words with nothing to mark which ones came from the caller, so the agent gets both as one request.

That breaks a voice agent in three ways:

In each case the agent's logic is working as designed. It is reacting to a voice it should never have heard.

The numbers show how big the gap is. In Krisp's Voice Isolation 2.5 release test, 10 STT systems averaged a 2.10% word error rate (WER) on clean single-speaker audio. With a second person talking over the caller, the same systems averaged 35.92%. Noise suppression doesn't close that gap, because the interference is speech.

What Krisp Voice Isolation is

The Krisp Voice Isolation API & SDK removes background voices and noise from a live audio stream and passes on only the main speaker. It's built for voice AI agents, and it's the same core technology Krisp ships for human calls under the name Background Voice Cancellation (BVC).

It goes first in the agent's pipeline, so every model after it hears one voice:

Where Voice Isolation sits in a voice agent pipeline

Where Voice Isolation sits in a voice agent pipeline

That placement matters. Without it, VAD and turn detection hear the bystander too, which is where the false interruptions come from.

Krisp has spent about nine years on voice AI and says it processes more than 1 trillion minutes of voice a year. Voice Isolation is one of the products in its voice AI API and SDK lineup. Voice Isolation is part of VIVA, Krisp's set of real-time models for voice agents, alongside Turn-Taking, Interrupt Prediction and VAD. It works with turn-taking rather than replacing it.

Spec

Voice Isolation 2.5

Compute

CPU only, no GPU needed

Algorithmic latency

15 ms (Krisp's SDK docs put added latency at about 20 ms)

STT engines

Any. It works on the audio signal, so there's no per-language or per-engine setup

Phone audio

Narrowband telephony, G.711, G.722, G.729, Opus, Bluetooth headsets

Voice enrollment

None needed

Mic distance

Near-field; roughly within 30 cm (1 ft) as a general guide, depending on acoustics 

Models

Default, Balanced, Lite, HD

The four models give you control over how aggressively background voices and noise are removed. With full aggressiveness (the default configuration), Default removes the most. Balanced processes the main voice more lightly, at the same compute. Lite uses about a third of Default’s compute. HD is for 32 kHz WebRTC streams rather than phone audio. Aggressiveness is a run-time setting you can tune without changing the compute; in the API it’s the suppression_level parameter, from 0 to 100. 

One limit worth knowing up front: the voice-removing models assume the speaker is close to the mic. As a rough guide, past about 30 cm the main speaker’s own voice can be suppressed. That isn’t a hard rule; it’s a general recommendation, and the actual distance depends on the acoustic conditions.  For far-field devices like smart speakers, Krisp offers a separate model that removes noise only.

Version 2.5 shipped in August 2026, and Krisp published an open benchmark and dataset in September. The model is now available as a self-serve API as well as an SDK.

What Krisp's benchmarks show

On Krisp's open benchmark, Voice Isolation cut average word error rate from 23.24% to 6.24%, a 73% drop.

Krisp's benchmark page summarizes the headline result.

The benchmark uses 265 real recordings from offices, homes, four working call-center floors and cars. Nothing was simulated or mixed in afterwards. Most recordings follow the same script: the main speaker talks, a second person joins, the main speaker goes quiet while the other keeps talking, then the main speaker comes back. That quiet window is the real test, because the correct transcript for it is empty. Each file was run through 11 STT configurations from nine engines, with and without isolation.

Source: Krisp voice isolation benchmark, Oct 2026 · 11 STT configurations, all scenarios pooled

Every engine improved and none got worse. The more useful result is what happens to the gap between engines: on raw audio it's about 20 points wide, and after isolation it shrinks to about 5. If your agent hears background voices, cleaning the input changes your WER more than picking a different STT vendor does.

By scenario, the gain depends on whether a second voice is present:

Scenario

Raw audio

With Voice Isolation

Work (offices, homes, meeting rooms)

31.84%

6.30%

Call center

23.83%

6.86%

Phone calls (no competing voice)

3.48%

3.91%

All 265 files

23.24%

6.24%

Where it doesn't help

Krisp publishes its weak spots, and I think that's the most convincing part of the write-up.

Six sample recordings on Krisp's benchmark page, each playable raw and through Voice Isolation 2.5, including Sample 153 which gets worse

The benchmark page lets you play six recordings raw and processed. Sample 153, a phone call, is labeled "Gets worse".

Two other things to keep in mind. The 73% figure and the 46.4% figure you'll see on Krisp's site come from different tests: 265 recordings in the open benchmark, and 1,685 recordings in the 2.5 release test. And Krisp's results compare audio with and without its own isolation, not against other vendors' isolation products.

The recordings, hand-written transcripts, segment labels and every model's output are public on Hugging Face, so anyone can rerun the test. That's what I did next.

Hands-on: testing the Voice Isolation API

In my test, Voice Isolation removed every word of background speech from the transcripts, and pooled WER fell from 40.1% to 10.2%.

Setup

Signing up on the Developers Portal is quick, with Google, Microsoft or email sign-in. The trial gave me $30 in credits and a limit of two concurrent sessions. From there it's API Keys → Create key, and you get a key you can only copy once.

The Voice Isolation page in the Developers Portal on the trial plan.

The API key I created for this review. One key works across all Krisp API products, including the Voice Translation API. .

The API is a WebSocket. Your backend swaps the API key for a short-lived session token, the client sends a small JSON hello, waits for ready, then streams 16-bit mono PCM and gets cleaned audio back frame by frame:

URL = "wss://streaming.krisp.ai/voice-isolation"async with websockets.connect(URL, additional_headers={"Authorization": f"api-key {session_token}"}) as ws:    await ws.send(json.dumps({"model": "VIVA-tel", "sample_rate": 16000, "suppression_level": 100}))    hello = json.loads(await ws.recv())  # {"type": "ready", ...}    # then send exactly 30 ms of audio per message (960 bytes at 16 kHz)    # and write each binary message you receive to your output

That's close to Krisp's API quickstart, a short Python script that cleans a WAV file. One rule catches people: every audio message must be exactly 30 ms. Anything else ends the session with an invalid_frame_duration error, so pad the last frame with silence.

For the test I picked eight recordings from Krisp's public dataset: three office recordings, three from call-center floors and two phone calls from a car, about 11 minutes in total. I streamed each one through the API at real-time speed, from Asia. Then I transcribed the raw and cleaned audio with NVIDIA's open-source Parakeet TDT 0.6B v2, one of the engines in Krisp's benchmark, and scored each labeled segment against the human transcripts.

Results

Recordings

WER raw

WER with Voice Isolation

Office (3 files)

36.6%

3.6%

Call center (3 files)

53.4%

15.8%

Phone calls, no second voice (2 files)

5.8%

5.2%

All 8 files

40.1%

10.2%

Almost all of the improvement came from one place: the windows where the main speaker was silent and someone else kept talking. On raw audio, Parakeet turned that background speech into 331 words of transcript. After Voice Isolation it produced none. The background voice was 13 to 34 dB quieter in those windows, while the main speaker's level barely moved (within 1 dB).

My test · Krisp API (VIVA-tel), transcribed with Parakeet TDT 0.6B v2 · the 6 recordings with a background-only window

My test · Krisp API (VIVA-tel), transcribed with Parakeet TDT 0.6B v2 · the 6 recordings with a background-only window

The call-center example shows why this matters. While the agent I was testing paused, the agent at the next desk was on a different call. The raw transcript picked up that call word for word:

thank you for calling meridian bank before i begin i will need to confirm your identity for security could you give me your full name your date of birth...

A voice agent would treat that as the caller speaking. With Voice Isolation, the same window came back empty.

When both people talked at once, the change was small: 13.2% raw against 12.5% cleaned. Parakeet already favored the louder, closer voice in those stretches. Most of the remaining errors there are number formatting in my scoring (phone numbers, prices, postcodes), which hit both versions equally.

On the two clean phone calls, one file picked up an extra error and the other lost two. That's in line with Krisp's own finding that the cost on clean audio is small.

Latency and reliability from outside the US

The API currently runs only in the US region, and Krisp's docs warn that clients elsewhere will see extra latency. From Asia, the median round trip for a frame (send to cleaned audio back) was 260–520 ms depending on the session, and setting up a session usually took 1.5–3 seconds. That's network distance, not the model: Krisp states 15 ms of algorithmic latency. But if your agent serves users far from the US, you'll feel it, and the SDK on your own servers avoids it.

One of my nine sessions dropped about 70 seconds in with an abnormal close. Rerunning it worked. I'd put that down to the long route from Asia rather than the service, but it's a reminder to handle reconnects in production.

About this test: eight files is a small sample, not a benchmark. I resampled the 32 kHz recordings to 16 kHz for the telephony model (VIVA-tel), used one STT engine, and kept text normalization simple, so my absolute numbers aren't comparable with Krisp's. What I was checking is the direction and size of the change, and it matched what Krisp reports.

Developer experience: API or SDK

Voice Isolation ships two ways, and which one you can use depends on your plan.


Voice Isolation API

VIVA SDK

Where it runs

Krisp's cloud (US region)

Your own servers, on CPU

How you use it

Stream audio over a WebSocket, get cleaned audio back

Load a model and process audio frames in your own code

Languages

Anything with a WebSocket client

Python, Node.js, Go, Rust, C++ and C

Audio

8 or 16 kHz (VIVA-tel), 32 kHz (VIVA-pro), 16-bit mono PCM

8 kHz up to 32 kHz, depending on the model (HD supports inputs up to 32 kHz) 

Models

Two: VIVA-tel for phone audio, VIVA-pro for 32 kHz wideband

Default, Balanced, Lite, HD

Access

Self-serve, API key from the portal

Commercial license, Enterprise plan

Also includes

Voice Translation API

Turn-Taking and VAD

The API is the easy way in: no model files to ship and nothing to run on your CPU. The trade-offs are that your audio goes to Krisp's cloud, you get two models instead of four: VIVA-tel for 8 or 16 kHz phone audio, and VIVA-pro for 32 kHz wideband audio. VIVA-pro isn't in the API reference yet, but it's in the portal's Playground, the API accepted it in my test, and Krisp confirmed it's available.

The portal's Playground lets you test either API model with your own microphone before writing any code.

The SDK keeps audio on your own servers and only goes online to check its license. It's also where the cost story gets interesting. Krisp's CPU post shows that one SDK flag, enableLowPrecisionExecution, roughly doubles the calls a server can clean on CPUs with FP16 support. A 16-vCPU AWS Graviton3 instance went from 76 live calls to 137, and the Lite model holds about three times as many streams as the full models. On older Intel Ice Lake chips the flag does nothing.

Integrations

The docs are clear and current. The quickstart worked the first time, and the error messages tell you exactly what went wrong, down to the expected and received frame size. My one gripe is naming: the blog talks about Default, Balanced, Lite and HD, the SDK downloads use file names like krisp-viva-vi-tel-v2.5.1, and the API calls its models VIVA-tel and VIVA-pro, with only the first one in the API reference. A mapping table in the docs would help.

Pricing

The API is billed per hour of audio, and it's cheap enough that cost shouldn't decide whether you use it.

Krisp's developer pricing, November 2026.

On the pay-as-you-go Starter plan the Voice Isolation API costs $0.18 per hour of audio, with no monthly fee. The rate is the same for both API models, VIVA-tel and VIVA-pro. Growth costs $400 a month, includes $400 of usage and drops the rate to $0.12 per hour, with higher concurrency and priority support. Enterprise is custom and is the only plan that includes the SDK, an SLA, and HIPAA and SOC 2.

The $30 starting credit covers about 166 hours of audio at the Starter rate, which is plenty for testing. Credits last 12 months and usage is billed by the minute. For scale, a 5-minute call costs about 1.5 cents to clean on Starter.

The SDK is only on the Enterprise plan, at an agreed price, so you'll need to talk to sales if you want to run the models on your own servers. Rates are from Krisp's pricing page, checked in November 2026.

Alternatives

Krisp isn't the only option for removing background voices, but no independent head-to-head test of these products exists yet. Each vendor publishes its own numbers on its own data, so treat the figures below as claims.

Option

What it does

Worth knowing

DeepFilterNet

Real-time noise suppression

Works at 48 kHz in 20 ms windows with about 40 ms of total latency, fast enough for a single CPU thread. It targets noise, not a second voice.

DPDFNet

Real-time noise suppression, built on DeepFilterNet2

Causal, single-channel models from Ceva under Apache-2.0, with a CPU-only ONNX package (pip install dpdfnet). It's noise suppression, not voice removal.

ClearerVoice-Studio

Toolkit for enhancement, separation and target speaker extraction

Its audio-only speaker extraction needs a reference recording of the target speaker and runs at 8 kHz. Other extraction modes use video, gestures or EEG.

SpeechBrain SepFormer (WHAMR!)

Speech separation

Apache-2.0 model for 8 kHz single-channel audio that splits a mix into separate voices. You still have to decide which output is the caller.

No isolation

Rely on the STT's own noise handling

Free, and fine if callers are mostly alone; Deepgram has argued that pre-filtering can hurt STT

Where Krisp stands out is the evidence. It's the only vendor here with an open, reproducible benchmark across 11 STT setups, and it publishes the cases where it loses. It's also the most flexible to deploy: cloud API, on-server SDK, or built into LiveKit and Pipecat.

Verdict

Krisp Voice Isolation is the fix to reach for when a voice agent misbehaves on real calls and works fine in testing. It removes the voices your agent shouldn't hear, it works with whatever STT you already use, and both Krisp's benchmark and my own test show the same thing: background speech stops reaching the transcript.

Pros

Cons

Who should use it: teams whose agents take calls from call centers, offices, homes, cars or anywhere other people talk nearby. Start with the API to measure the effect on your own recordings, then move to the SDK if you need the other models, lower latency outside the US, or audio that stays on your servers.

Who can skip it: agents whose callers are almost always alone in quiet rooms, or far-field devices like smart speakers, where the voice-removing models aren't designed to work.

You can try the Krisp Voice Isolation API with the free credits and run your own recordings through it in an afternoon.

FAQ

Is voice isolation the same as noise cancellation? No. Noise cancellation removes non-speech sounds like traffic and keyboards. Voice isolation also removes other people's voices, which noise cancellation leaves in.

Does Krisp Voice Isolation work with my STT provider? Yes. It processes the audio itself, so it works in front of any STT engine. Krisp tested it with 11 configurations, including Deepgram, AssemblyAI, ElevenLabs, Soniox, Google, Cartesia and NVIDIA models. It can also sit in front of speech-to-speech models. Krisp hasn’t published results for that setup yet, but nothing stops you from trying it. 

Does it replace turn detection? No. It goes before VAD and turn detection and gives them cleaner audio. Krisp provides Turn-Taking as a separate model in the same VIVA SDK.

Do users need to enroll their voice? No. It needs no enrollment and no training on the user's voice.

How much latency does it add? Krisp states 15 ms of algorithmic latency, and its SDK docs put the added latency at about 20 ms. With the cloud API, network time to the US region comes on top of that.

Does it work on phone calls? Yes. It handles narrowband telephony audio, common codecs such as G.711, G.722, G.729 and Opus, and Bluetooth headsets.

What about human-to-human calls? The same technology ships as Background Voice Cancellation (BVC) in Krisp's RTC SDK, which runs on the user's device in browsers, desktop and mobile apps. Contact centers with human agents get it through Krisp’s call center AI product, where it keeps the voices of nearby agents out of customer calls.

Explore more

FinFeedAPI Review: Multi-Exchange Prediction Market Data API

FinFeedAPI Review: Multi-Exchange Prediction Market Data API

FinFeedAPI review: 9 prediction market exchanges through one API. Endpoints, MCP setup, pricing, and limitations for developers explained.

3 November 2026

Best RPC Node Providers in 2027: A Developer Guide

Best RPC Node Providers in 2027: A Developer Guide

A blockchain application can have a great frontend, well-designed smart contracts, and a carefully planned architecture, but it still needs a reliable way to communicate with the network. That communication usually happens through an RPC endpoint. When a wallet retrieves a balance, a dApp reads a smart contract, a trading bot checks the latest block, or an application submits a transaction, the request ultimately needs to reach a blockchain node. Running and maintaining those nodes yourself is

3 September 2026

Best Crypto Swap Exchanges in 2027

Best Crypto Swap Exchanges in 2027

Crypto swaps have become one of the simplest ways for users to exchange digital assets. At first glance, the process looks straightforward: select one token, choose another asset, and complete the swap. However, behind that simple experience is a complex layer of infrastructure involving liquidity providers, routing systems, blockchain networks, and exchange mechanisms. While researching this topic, I looked at how different crypto swap platforms approach this problem from non-custodial swap a

19 July 2026

Practice API Design-first in Apidog

Discover an easier way to build and use APIs

Krisp Voice Isolation Review: Fixing the Second-Voice Problem in Voice AI Agents