I Guess That's Why They Call it the Blues: Causal Analysis for Audio Classifiers

TL;DR

This paper introduces a causal analysis method and tool, FreqReX, to identify critical frequency features in audio classifiers, revealing their reliance on non-musical cues and enabling targeted manipulation of model outputs.

Contribution

It presents a novel causal reasoning approach and implementation for discovering necessary and sufficient features in audio classification, enhancing interpretability and robustness.

Findings

58% of classifications change with minimal frequency modifications

Small frequency changes can be practically inaudible yet cause misclassification

Causal analysis helps understand and manipulate audio classifier decisions

Abstract

It is well-known that audio classifiers often rely on non-musically relevant features and spurious correlations to classify audio. Hence audio classifiers are easy to manipulate or confuse, resulting in wrong classifications. While inducing a misclassification is not hard, until now the set of features that the classifiers rely on was not well understood. In this paper we introduce a new method that uses causal reasoning to discover features of the frequency space that are sufficient and necessary for a given classification. We describe an implementation of this algorithm in the tool FreqReX and provide experimental results on a number of standard benchmark datasets. Our experiments show that causally sufficient and necessary subsets allow us to manipulate the outputs of the models in a variety of ways by changing the input very slightly. Namely, a change to one out of 240,000 frequencies results in a change in classification 58% of the time, and the change can be so small that it is practically inaudible. These results show that causal analysis is useful for understanding the reasoning process of audio classifiers and can be used to successfully manipulate their outputs.