Evaluating CNN with Stacked Feature Representations and Audio Spectrogram Transformer Models for Sound Classification

TL;DR

This study compares CNNs with stacked acoustic features and transformer models for sound classification, showing CNNs are more efficient with limited data and resources, especially in edge applications.

Contribution

It introduces a comprehensive analysis of feature-stacked CNNs versus transformer models across multiple datasets and training regimes, highlighting their relative advantages.

Findings

Feature-stacked CNNs perform well with limited data.

Transformer models excel with large-scale pretraining.

CNNs are more resource-efficient for edge scenarios.

Abstract

Environmental sound classification (ESC) has gained significant attention due to its diverse applications in smart city monitoring, fault detection, acoustic surveillance, and manufacturing quality control. To enhance CNN performance, feature stacking techniques have been explored to aggregate complementary acoustic descriptors into richer input representations. In this paper, we investigate CNN-based models employing various stacked feature combinations, including Log-Mel Spectrogram (LM), Spectral Contrast (SPC), Chroma (CH), Tonnetz (TZ), Mel-Frequency Cepstral Coefficients (MFCCs), and Gammatone Cepstral Coefficients (GTCC). Experiments are conducted on the widely used ESC-50 and UrbanSound8K datasets under different training regimes, including pretraining on ESC-50, fine-tuning on UrbanSound8K, and comparison with Audio Spectrogram Transformer (AST) models pretrained on large-scale corpora such as AudioSet. This experimental design enables an analysis of how feature-stacked CNNs compare with transformer-based models under varying levels of training data and pretraining diversity. The results indicate that feature-stacked CNNs offer a more computationally and data-efficient alternative when large-scale pretraining or extensive training data are unavailable, making them particularly well suited for resource-constrained and edge-level sound classification scenarios.