DuplexCascade: Full-Duplex Speech-to-Speech Dialogue with VAD-Free Cascaded ASR-LLM-TTS Pipeline and Micro-Turn Optimization

TL;DR

DuplexCascade introduces a VAD-free, full-duplex speech-to-speech dialogue system that uses micro-turns and control tokens to enable rapid, natural conversations while maintaining LLM capabilities.

Contribution

The paper presents a novel VAD-free cascaded streaming pipeline with micro-turns and control tokens for full-duplex dialogue, enhancing turn-taking and conversational intelligence.

Findings

Achieves state-of-the-art full-duplex turn-taking on benchmarks.

Supports rapid bidirectional exchanges with micro-turns.

Maintains strong conversational intelligence in open-source systems.

Abstract

Spoken dialog systems with cascaded ASR-LLM-TTS modules retain strong LLM intelligence, but VAD segmentation often forces half-duplex turns and brittle control. On the other hand, VAD-free end-to-end model support full-duplex interaction but is hard to maintain conversational intelligence. In this paper, we present DuplexCascade, a VAD-free cascaded streaming pipeline for full-duplex speech-to-speech dialogue. Our key idea is to convert conventional utterance-wise long turns into chunk-wise micro-turn interactions, enabling rapid bidirectional exchange while preserving the strengths of a capable text LLM. To reliably coordinate turn-taking and response timing, we introduce a set of conversational special control tokens that steer the LLM's behavior under streaming constraints. On Full-DuplexBench and VoiceBench, DuplexCascade delivers state-of-the-art full-duplex turn-taking and strong conversational intelligence among open-source speech-to-speech dialogue systems.