ASCMamba: Multimodal Time-Frequency Mamba for Acoustic Scene Classification

TL;DR

This paper introduces ASCMamba, a multimodal neural network that combines audio and textual data for improved acoustic scene classification, achieving state-of-the-art results in a challenging competition setting.

Contribution

We propose a novel multimodal network architecture, ASCMamba, integrating hierarchical spectral features and long-range dependencies for enhanced acoustic scene understanding.

Findings

Outperforms all participating teams in the challenge

Achieves 6.2% improvement over baseline

Demonstrates effectiveness of multimodal integration

Abstract

Acoustic Scene Classification (ASC) is a fundamental problem in computational audition, which seeks to classify environments based on the distinctive acoustic features. In the ASC task of the APSIPA ASC 2025 Grand Challenge, the organizers introduce a multimodal ASC task. Unlike traditional ASC systems that rely solely on audio inputs, this challenge provides additional textual information as inputs, including the location where the audio is recorded and the time of recording. In this paper, we present our proposed system for the ASC task in the APSIPA ASC 2025 Grand Challenge. Specifically, we propose a multimodal network, ASCMamba, which integrates audio and textual information for fine-grained acoustic scene understanding and effective multimodal ASC. The proposed ASCMamba employs a DenseEncoder to extract hierarchical spectral features from spectrograms, followed by a dual-path Mamba blocks that capture long-range temporal and frequency dependencies using Mamba-based state space models. In addition, we present a two-step pseudo-labeling mechanism to generate more reliable pseudo-labels. Results show that the proposed system outperforms all the participating teams and achieves a 6.2% improvement over the baseline. Code, model and pre-trained checkpoints are available at https://github.com/S-Orion/ASCMamba.git.