Towards Fusion of Neural Audio Codec-based Representations with Spectral for Heart Murmur Classification via Bandit-based Cross-Attention Mechanism

TL;DR

This paper introduces BAOMI, a novel fusion framework using a bandit-based cross-attention mechanism to combine neural audio codec representations with spectral features, significantly improving heart murmur classification accuracy.

Contribution

The paper presents a new fusion method with a bandit-based cross-attention mechanism that effectively combines NACRs and spectral features for heart murmur classification.

Findings

Achieved state-of-the-art performance in heart murmur classification.

Demonstrated superior results over individual features and baseline fusion methods.

Validated the effectiveness of the bandit-based attention mechanism.

Abstract

In this study, we focus on heart murmur classification (HMC) and hypothesize that combining neural audio codec representations (NACRs) such as EnCodec with spectral features (SFs), such as MFCC, will yield superior performance. We believe such fusion will trigger their complementary behavior as NACRs excel at capturing fine-grained acoustic patterns such as rhythm changes, spectral features focus on frequency-domain properties such as harmonic structure, spectral energy distribution crucial for analyzing the complex of heart sounds. To this end, we propose, BAOMI, a novel framework banking on novel bandit-based cross-attention mechanism for effective fusion. Here, a agent provides more weightage to most important heads in multi-head cross-attention mechanism and helps in mitigating the noise. With BAOMI, we report the topmost performance in comparison to individual NACRs, SFs, and baseline fusion techniques and setting new state-of-the-art.