A Methodology for Exploring Deep Convolutional Features in Relation to Hand-Crafted Features with an Application to Music Audio Modeling

TL;DR

This paper introduces a methodology to analyze deep convolutional features by relating them to known hand-crafted features, demonstrated on music audio data, to enhance understanding and trust in deep models.

Contribution

The paper proposes a systematic approach for exploring deep features in relation to hand-crafted features, applicable across various domains, with a case study on music audio classification.

Findings

Deep features can be related to meaningful hand-crafted features.

The methodology reveals which features are useful and robust for classification.

Deep features show varying degrees of similarity to hand-crafted features.

Abstract

Understanding the features learned by deep models is important from a model trust perspective, especially as deep systems are deployed in the real world. Most recent approaches for deep feature understanding or model explanation focus on highlighting input data features that are relevant for classification decisions. In this work, we instead take the perspective of relating deep features to well-studied, hand-crafted features that are meaningful for the application of interest. We propose a methodology and set of systematic experiments for exploring deep features in this setting, where input feature importance approaches for deep feature understanding do not apply. Our experiments focus on understanding which hand-crafted and deep features are useful for the classification task of interest, how robust these features are for related tasks and how similar the deep features are to the meaningful hand-crafted features. Our proposed method is general to many application areas and we demonstrate its utility on orchestral music audio data.