What is voice activity detection (VAD)?

How does voice activity detection work?

Voice activity detection looks at short slices of audio, often a few tens of milliseconds each, and classifies every slice as speech or non-speech. The output is a simple running signal: talking, not talking, talking again. Everything a voice agent does with timing hangs off that signal.

Classic VAD algorithms make the decision from a handful of acoustic features. The G.729 Annex B standard, one of the best-known implementations, computes line spectral frequencies, full-band energy, low-band energy below one kilohertz, and the zero-crossing rate of the waveform, then classifies each frame and smooths the result so a single noisy slice does not flip the decision. Speech has energy patterns and a spectral shape that silence and steady background hum do not, and those features are what separate them.

The hard part is not detecting a loud, clear voice in a quiet room. It is holding up in the real world:

• Background noise. A caller in a car, a busy clinic lobby, or a windy street floods the line with sound that is not speech. VAD has to stay locked on the voice and ignore the rest, which gets harder as the signal-to-noise ratio drops.
• Natural pauses. People pause mid-thought. A VAD that treats every half-second gap as “the caller is done” will cut them off; one that waits too long makes the agent feel slow to respond.
• Non-speech vocal noise. Coughs, throat-clears, and “um” are sound from the caller that may or may not count as a turn.

VAD started in telephony for a different reason: bandwidth. VoIP systems use it for discontinuous transmission, sending packets only while someone is actually speaking and filling the gaps with comfort noise, which cuts the data sent over a call. The same detector that saves bandwidth is now what tells a voice agent when to listen.

VAD vs endpointing: detecting speech vs detecting the end of a turn

VAD and endpointing get used interchangeably, but they answer different questions.

• VAD answers “is there speech right now?” It fires the moment the caller starts and clears when they stop.
• Endpointing answers “has the caller finished their turn?” It watches the VAD signal and decides that a pause is long enough to mean the caller is done and expects a reply.

Endpointing is the judgment call that makes or breaks conversational feel. Lean toward short pauses and the agent jumps in before the caller finishes a sentence. Lean toward long pauses and there is a beat of silence after every turn that makes the agent feel laggy. Good voice systems tune this against real speech patterns rather than a fixed timer, and they combine it with barge-in so the caller can always take the turn back.

Why does VAD matter for an AI receptionist?

Every natural-sounding moment in an automated call traces back to knowing who has the floor. VAD is what lets an AI receptionist start transcribing exactly when the caller speaks, stop when they stop, and answer in the rhythm of a real conversation instead of a walkie-talkie.

It also gates the expensive work. Automatic speech recognition does not need to run on silence, so VAD triggers recognition only when there is speech to transcribe. That keeps latency down and avoids the agent “hearing” words in the hum of an empty line.

Get VAD wrong and callers feel it immediately: the agent talks over them, leaves dead air, or answers a cough. Get it right and most callers never think about it, which is the goal.

How Tilk uses voice activity detection

Tilk runs on Google’s Gemini live audio models, which process native audio in real time and handle turn detection as part of the live pipeline rather than as a bolted-on module. Speech recognition and the agent’s response are driven by that live audio stream over Telnyx telephony, so the agent picks up the caller’s turn as it happens.

Because turn detection is native to the audio models, the agent adapts to a caller who trails off, restarts a sentence, or pauses to check a calendar, without a rigid “wait exactly N milliseconds then respond” rule that trips over normal human speech.

Related questions

Is VAD the same as speech recognition? No. VAD only decides whether speech is present. Speech recognition, or ASR, turns that speech into text. VAD usually runs first and tells recognition when there is something worth transcribing.

Does better VAD reduce interruptions? It reduces the wrong kind. Accurate VAD stops the agent from reacting to noise or a brief pause, which cuts accidental interruptions. Intentional interruptions are still handled by barge-in, which lets the caller cut in on purpose whenever they want the floor.