Temporal Feature Learning in Weakly Labelled Bioacoustic Cetacean Datasets via a Variational Autoencoder and Temporal Convolutional Network: An Interdisciplinary Approach

TL;DR

This paper introduces an interdisciplinary framework combining Variational Autoencoders and Temporal Convolutional Networks to improve classification of weakly labelled bioacoustic cetacean datasets, effectively capturing complex temporal features without extensive manual labelling.

Contribution

It presents a novel combination of VAE-based feature extraction and TCN classification tailored for weakly labelled, lengthy bioacoustic data, enhancing robustness and generalisability.

Findings

VAE-based features outperform expert handpicked features in classification.

TCN achieves AUC scores over 0.9 on sperm whale click train data.

Framework reduces need for manual threshold setting and strong labels.

Abstract

Bioacoustics data from Passive acoustic monitoring (PAM) poses a unique set of challenges for classification, particularly the limited availability of complete and reliable labels in datasets due to annotation uncertainty, biological complexity due the heterogeneity in duration of cetacean vocalizations, and masking of target sounds due to environmental and anthropogenic noise. This means that data is often weakly labelled, with annotations indicating presence/absence of species over several minutes. In order to effectively capture the complex temporal patterns and key features of lengthy continuous audio segments, we propose an interdisciplinary framework comprising dataset standardisation, feature extraction via Variational Autoencoders (VAE) and classification via Temporal Convolutional Networks (TCN). This approach eliminates the necessity for manual threshold setting or time-consuming strong labelling. To demonstrate the effectiveness of our approach, we use sperm whale (<i>Physeter macrocephalus</i>) click trains in 4-minute recordings as a case study, from a dataset comprising diverse sources and deployment conditions to maximise generalisability. The value of feature extraction via the VAE is demonstrated by comparing classification performance against the traditional and explainable approach of expert handpicking of features. The TCN demonstrated robust classification capabilities achieving AUC scores exceeding 0.9.