Contrastive Environmental Sound Representation Learning

TL;DR

This paper introduces a self-supervised contrastive learning approach using a shallow 1D CNN to extract robust environmental sound representations from raw audio and spectrograms, improving recognition accuracy.

Contribution

It proposes a novel contrastive learning method with multi-input fusion via CCA for environmental sound representation without annotations.

Findings

Achieved 12.8% improvement on ESC-50 dataset.

Achieved 0.9% improvement on UrbanSound8K dataset.

Demonstrated robustness of fused features over individual representations.

Abstract

Machine hearing of the environmental sound is one of the important issues in the audio recognition domain. It gives the machine the ability to discriminate between the different input sounds that guides its decision making. In this work we exploit the self-supervised contrastive technique and a shallow 1D CNN to extract the distinctive audio features (audio representations) without using any explicit annotations.We generate representations of a given audio using both its raw audio waveform and spectrogram and evaluate if the proposed learner is agnostic to the type of audio input. We further use canonical correlation analysis (CCA) to fuse representations from the two types of input of a given audio and demonstrate that the fused global feature results in robust representation of the audio signal as compared to the individual representations. The evaluation of the proposed technique is done on both ESC-50 and UrbanSound8K. The results show that the proposed technique is able to extract most features of the environmental audio and gives an improvement of 12.8% and 0.9% on the ESC-50 and UrbanSound8K datasets respectively.