Streaming Sequence Transduction through Dynamic Compression

TL;DR

The paper presents STAR, a Transformer-based streaming transduction model that uses dynamic segmentation and compression to improve efficiency and latency in sequence-to-sequence tasks like speech recognition.

Contribution

STAR introduces a novel dynamic segmentation and compression approach for streaming transduction, enhancing efficiency and latency in real-time applications.

Findings

Achieves 12x compression in ASR with near-lossless quality.

Outperforms existing methods in segmentation and latency-quality trade-offs.

Optimizes latency, memory, and quality in simultaneous speech-to-text.

Abstract

We introduce STAR (Stream Transduction with Anchor Representations), a novel Transformer-based model designed for efficient sequence-to-sequence transduction over streams. STAR dynamically segments input streams to create compressed anchor representations, achieving nearly lossless compression (12x) in Automatic Speech Recognition (ASR) and outperforming existing methods. Moreover, STAR demonstrates superior segmentation and latency-quality trade-offs in simultaneous speech-to-text tasks, optimizing latency, memory footprint, and quality.