DASH: Dynamic Audio-Driven Semantic Chunking for Efficient Omnimodal Token Compression

TL;DR

DASH is a training-free, semantic-aware token compression framework for multimodal models that dynamically segments audio-visual data based on semantic cues, improving efficiency without sacrificing accuracy.

Contribution

DASH introduces a novel, semantic-driven, dynamic segmentation method that aligns token compression with the inherent structure of audio-visual signals, outperforming fixed-window approaches.

Findings

Maintains higher accuracy at increased compression ratios.

Effectively aligns token segmentation with semantic boundaries.

Outperforms prior compression methods on multiple datasets.

Abstract

Omnimodal large language models (OmniLLMs) jointly process audio and visual streams, but the resulting long multimodal token sequences make inference prohibitively expensive. Existing compression methods typically rely on fixed window partitioning and attention-based pruning, which overlook the piecewise semantic structure of audio-visual signals and become fragile under aggressive token reduction. We propose Dynamic Audio-driven Semantic cHunking (DASH), a training-free framework that aligns token compression with semantic structure. DASH treats audio embeddings as a semantic anchor and detects boundary candidates via cosine-similarity discontinuities, inducing dynamic, variable-length segments that approximate the underlying piecewise-coherent organization of the sequence. These boundaries are projected onto video tokens to establish explicit cross-modal segmentation. Within each segment, token retention is determined by a tri-signal importance estimator that fuses structural boundary cues, representational distinctiveness, and attention-based salience, mitigating the sparsity bias of attention-only selection. This structure-aware allocation preserves transition-critical tokens while reducing redundant regions. Extensive experiments on AVUT, VideoMME, and WorldSense demonstrate that DASH maintains superior accuracy while achieving higher compression ratios compared to prior methods. Code is available at: https://github.com/laychou666/DASH.