Automatic Detection and Compression for Passive Acoustic Monitoring of the African Forest Elephant

TL;DR

This paper presents a machine learning approach for detecting and compressing African Forest Elephant calls in passive acoustic monitoring, enabling efficient, real-time wildlife surveillance in remote habitats.

Contribution

It introduces a large labeled dataset and a novel end-to-end differentiable compression method tailored for low-frequency animal calls, advancing passive acoustic monitoring techniques.

Findings

Improved classification and segmentation accuracy for elephant calls.

Significant reduction in data size with the new compression method.

Enhanced real-time detection capabilities in bandwidth-constrained environments.

Abstract

In this work, we consider applying machine learning to the analysis and compression of audio signals in the context of monitoring elephants in sub-Saharan Africa. Earth's biodiversity is increasingly under threat by sources of anthropogenic change (e.g. resource extraction, land use change, and climate change) and surveying animal populations is critical for developing conservation strategies. However, manually monitoring tropical forests or deep oceans is intractable. For species that communicate acoustically, researchers have argued for placing audio recorders in the habitats as a cost-effective and non-invasive method, a strategy known as passive acoustic monitoring (PAM). In collaboration with conservation efforts, we construct a large labeled dataset of passive acoustic recordings of the African Forest Elephant via crowdsourcing, compromising thousands of hours of recordings in the wild. Using state-of-the-art techniques in artificial intelligence we improve upon previously proposed methods for passive acoustic monitoring for classification and segmentation. In real-time detection of elephant calls, network bandwidth quickly becomes a bottleneck and efficient ways to compress the data are needed. Most audio compression schemes are aimed at human listeners and are unsuitable for low-frequency elephant calls. To remedy this, we provide a novel end-to-end differentiable method for compression of audio signals that can be adapted to acoustic monitoring of any species and dramatically improves over naive coding strategies.

Tables

Table 1. Table 1: The statistics of the datasets by location. The Apx. percentage of calls refer to what portion of the audio recordings contained elephant calls. The dates are given in YY/MM format.

Location	Dates Collected	Labelled hours	Num. calls	Apx. % Calls
Ceb1	09/04 - 11/06	1870	52810	0.784 %
Ceb4	08/06 - 11/03	1280	23038	0.500 %
Jobo	09/05 - 11/06	1437	28609	0.553 %
Dzan	11/04 - 12/02	312	63792	21.8 %

Table 2. Table 2: The classification accuracy on the test-set for different algorithms at different locations.

Location	SVM	RF	ADA-grad	DNN ((?))	Densenet + rnd crop
Ceb1	77.22	76.73	77.01	91.11	93.40
Ceb4	70.21	69.82	71.50	90.15	93.68
Jobo	76.92	76.49	76.86	91.67	94.30
Dzan	72.21	69.79	70.86	75.86	77.51
Avg.	74.14	73.21	74.06	87.20	89.72

Table 3. Table 3: The classification accuracy on the test-set for the segmentation task, given for different algorithms at different locations.

Location	LSTM	conv-LSTM
Ceb1	70.50	95.24
Ceb4	70.43	90.54
Jobo	67.49	92.12
Dzan	70.54	88.95
Avg.	69.74	91.71

Table 4. Table 4: The classification accuracy on the test-set for the given bit-rates at various locations.

Method / Bit-rate	Ceb1	Ceb4	Jobo	Dzan
Ours / 47	84.57	83.98	86.31	78.43
Human / 47	83.62	81.31	85.76	69.44
Ours / 141	92.81	92.21	93.19	77.96
Human / 141	86.61	91.90	90.32	73.51
Ours / 235	93.05	93.11	93.84	77.46
Human / 235	90.25	92.34	91.64	76.93

Table 5. Table 5: Hyper-parameters used for training.

Parameter	Value
init. learning rate	0.1
SGD momentum	0.9
batch size	64
initialization	kaiming
weight decay	0.0001
loss function	cross-entropy

Equations

$$\underset{\begin{subarray}{c}\beta\sim N\\ (\hat{X},y)\sim D\end{subarray}}{\mathbb{E}}\bigg{[}L\bigg{(}y,\textrm{DNN}_{\omega}\big{(}\exp(-\lambda)\odot\beta+\hat{X}\big{)}\bigg{)}\bigg{]}+\mu\sum_{i}\lambda_{i}$$

Full text

Introduction

Poaching, illegal logging, and infrastructure expansions are some of many current threats to biodiversity, and large mammals are particularly susceptible. To effectively allocate conservation resources and develop conservation strategies, endangered animal populations need to be accurately and economically surveyed, but for species that roam large or inaccessible areas monitoring by humans becomes intractable. A promising approach for species communicating via acoustic signals is passive acoustic monitoring (PAM), which involves the use of autonomous recording devices scattered throughout habitats that record animal vocalizations. Compared to video monitoring, acoustic monitoring is not limited by line of sight, is typically considerably cheaper and requires less bandwidth for transferring the data. However, extracting useful data from these soundscapes is non-trivial and automatic approaches are necessary.

In this work, we consider PAM in the context of monitoring the African forest elephant. To enable real-time acoustic monitoring one needs to quickly and accurately detect elephants and potential threats to them – a classical challenge of classification and segmentation. Leveraging recent advances in neural networks, we improve upon previous methods in automating PAM. In real-time threat-detection and population monitoring the bandwidth of the wireless networks becomes a bottleneck, and one additionally has to use efficient data representations to only communicate the necessary information. In many lossy compression schemes, signal components inaudible to humans such as low frequencies are given low bit-rates, which in the context of low-frequency elephant calls is a poor strategy. Using a differentiable proxy for non-differentiable bit truncation, we are able to cast this problem as an end-to-end differentiable setup, which can be trained via stochastic gradient descent (SGD) to get improved compression.

We focus on the African Forest Elephant both due to its biological importance and the loud calls by which it communicates. This elephant is a keystone species in the rainforests of the Congo Basin, the second largest expanse of rainforest on earth and among the most speciose. Conserving viable populations of forest elephants protects local biodiversity, but the expansiveness of the rainforest and the difficulty of monitoring animals within it makes manual monitoring problematic. Since elephants communicate over long distances via infrasonic signals referred to as rumbles (?) they are particularly suited to an acoustic approach. These characteristic vocalizations provide information on occupancy, landscape use, population size, and the effects of anthropogenic disturbances (?).

The contributions of our work are to 1) construct a large dataset of real-world elephant vocalizations from central Africa via passive acoustic monitoring, 2) surpass previously proposed methods for automatic PAM in the context of elephant vocalizations via state-of-the-art artificial intelligence techniques, and 3) introduce a novel end-to-end differentiable technique for audio-compression, that can balance the bit-rates between different frequency channels to the unique characteristics of the monitored species.

The Dataset

Data collection

Established in 2000, the Elephant Listening Project (ELP) uses acoustic methods to study the ecology and behavior of forest elephants in order to improve evidence-based decision making concerning their conservation. ELP has recorded sounds from over 150 different locations, amassing more than 700,000 hours of recordings. These varying environments provide the source material for generating training data for algorithm development. The dataset we consider in this work was collected between 2007 and 2012 from three sites in Gabon and one in the Central African Republic, which will be referred to as Ceb1, Ceb4, Dzanga, and Jobo. A map showing these locations is given in Figure 7 of the Appendix. At all locations, a single recording device was placed in a tree 7-10 meters above the ground near forest clearings (25 to 50ha) where elephants congregate for multiple purposes. The recording devices sample audio signals at a rate of 2000 (12-bit resolution) or 4000Hz (16-bit) and can detect elephant calls up to approximately 0.8 km away. As is typical in bioacoustic applications the animals are detected infrequently, and different locations have variable density, see Table 1. Additionally, multiple other sources of sound are recorded, both man-made and natural. For example, Ceb4 is close to a road and the recordings include signals associated with logging and gunshots.

Acoustic Characteristics of the Dataset

The primary mode of communication among elephants is a low-frequency vocalization known as a rumble, typically lasting between 2 and 8 seconds. These sounds have distinct frequency characteristics, with a low fundamental frequency (8 - 34Hz), often several higher harmonics, and slight frequency modulation. A typical recording is shown in Figure 2. At large gatherings, multiple elephants often make simultaneous or overlapping calls (see for example Figure 2 where two calls overlap). Other complications are the variability of the dataset, for example, some recording sites are close to logging concessions which are often visited by motorized vehicles which become recorded. Natural sources of noise include heavy wind, rainfall, insects chirping and thunderstorms, see Figure 3 for further examples.

Labeling

The labeling of rumbles to be used in the training and testing of detection algorithms was done by both experts and trained volunteers at the Elephant Listening Project. The volunteers were recruited by a combination of work-study positions and information spread via word-of-mouth and were asked to identify individual elephant calls and their temporal extent in the recording. Positive labeling was based on a set of criteria developed by experts with more than ten years of experience with forest elephant vocalizations and experience with potentially confusing environmental sounds. Volunteers followed a detailed training program that concluded with them labeling rumbles in two 24 hour long test sound files. The labels generated by the volunteers for the test files were compared to those of an expert. If the results were within 5% of each other, the volunteer was considered trained; if not, he/she repeated the process on other sound files until the 5% or less difference was achieved. The occasional further review of volunteer labeling efforts by the experts maintained reasonable consistency among all labelers (reliability $>98\%$). Statistics about the dataset and the labeling can be seen in Table 1. To facilitate online crowdsourcing, we have created an online labeling application for labeling. The website contains a tutorial where participants can first learn about the characteristics and variations of elephant calls and other sounds that might occur in recordings. The tutorial is publicly available at www.udiscover.it/applications/elp/tutorial.php. Once trained, participants can then label elephant calls in audio segments by using the application’s annotation tool, see Figure 8 in the Appendix. By giving the same spectrum to multiple participants one can gauge the accuracy of individual users and can encourage truthful responses. These issues will be addressed further in future work.

Classification and Segmentation

The simplest and most straightforward problem for passive acoustic monitoring is that of detection. Given a short audio-clip we want to classify it as containing a signal produced by the species of interest (in this case an elephant rumble) or not. This setting has been considered by many previous authors (?; ?; ?), and is a crucial stepping stone toward using PAM for population surveying and monitoring. A similar setting we consider is one of segmentation where we want to classify each discrete time step as belonging to an elephant call or not.

Data Processing

Given a specific location where a recording device is placed, say ”Ceb1” in Table 1, we extract all unique elephant rumbles recorded. The calls will in some cases overlap and if so we consider them as two or more unique calls. To facilitate a homogeneous dataset, we extract signals of the fixed length 25.5 sec and remove the handful of calls that are longer than this. We then extract empty regions of the same length, that does not overlap with any elephant calls, uniformly at random, we extract as many empty regions as there are elephant calls. The combined dataset of calls and empty frames is then split uniformly and randomly into a testing and training set, where additional augmentation might be performed on the training set. Given these fixed sized windows of audio recording, we transform them into the frequency domain via FFT. Using the signals down-sampled to 1000 Hz, we use a window size of 512 and hop-length of 384 we use FFT to transform the audio signal into the frequency domain. All frequency bands above 100Hz are removed, as the set of recorded elephant calls rarely have significant signals above such frequency because of signal attenuation. This gives us a 64 time-steps by 47 frequency bands tensor, and we specifically choose time-steps to have the shape be a power of two which can be beneficial for training on GPUs. One has to be slightly careful when normalizing the dataset as the signals are very sparse, we have found that subtracting the mean of all frames containing no calls and then dividing the signal by the median call intensity works well.

Neural Network Architectures and Training

Audio-clips are approximately time-invariant, i.e., an elephant call will sound the same no matter if it starts after 1s or 3s. This approximate symmetry suggests the use of convolutions over the time dimension would be successful, we have however found it beneficial to additionally perform convolutions over the frequency dimension. This corresponds to an approximate pitch-invariance, meaning that elephant calls from different elephants sound similar except for a uniform pitch change. Given this two-way convolution, we adopt the state-of-the-art network architecture Densenet (?), which is a standard convolutional network with skip connections between all layers. For training this architecture we use best practices from neural network training (?), we consider SGD with momentum and weight decay, iteratively lowering the learning rate as performance plateaus and using cross-entropy as the loss. We further introduce data augmentation by adapting the technique of randomized cropping, which is typically used on image classification, to sound classification. The 64 time-steps are padded by 8 on both sides, and we feed a random 64 step long subsequence. Exact parameters are given in table 5 in the Appendix. For our segmentation setting we train a standard long short-term memory (LSTM) network (?), additionally, we use the same LSTM network with an initial one-dimensional convolutional layer on the frequency dimension. This convolution layers uses 25 filters and thus outputs a feature vector of length 25 for each time step, which is then fed into the LSTM network. For training this architecture we use the ADAM optimizer and cross-entropy as the loss.

Results

We compare our approach to previously proposed methods in bioacoustic monitoring, both modern and classical. From the latter category, we consider the method of ?, where rich features based upon the MFCC coefficients are extracted and then fed into an SVM (?). The MFCC coefficients are similar to the FFT, where the signal is decomposed into frequency parts, however, the filters and masks used are much more sophisticated. Given these coefficients for each time step, the features for the entire audio clip are the mean, variance and derivative of the coefficients for each time step across the entire audio clip. Additionally, we consider feeding these features into a Random Forest classifier (?) and an ADAGrad classifier, essentially emulating the approach of ? although with slightly different features. The second type of baselines we consider are convolutional neural networks, where we use the architecture (and training parameters/schedule) of ? proposed for classifying the vocalizations of bats. The architecture is classical, meaning no skip-connections are used and dropout is not used (?) for regularization, see ? for details. Results can be viewed in Table 2 where we see that our methods consistently outperform baselines across every location. A more nuanced picture over the precision and recall is given in Figure 4. For the segmentation setting, we compare our convolution-LSTM hybrid network’s performance to that of only an LSTM network and show how the hybrid methods perform much better in terms of accuracy, see Table 3.

Compression

Background

The ultimate aim of passive acoustic monitoring is to provide accurate real-time detections of elephant vocalizations and threats. It is infeasible to perform neural network computations on the recording devices, and hence the devices need to send their data over the wireless networks of sub-Saharan Africa. Unfortunately, wireless infrastructure is largely relatively poor or absent in this area of the world (?), available bandwidth is small and data-transfer is expensive. To make real-time passive acoustic monitoring cost efficient, one has to transfer only the most relevant information across the wireless network.

A natural strategy for reducing the data-transfers across the wireless network is to compress the acoustic data. Most lossy compression codecs crucially rely on the specifics of the human auditory system to remove data that are irrelevant to the experience of a human listener. For example, it is well known that the sensitivity of the human auditory system varies with frequency (?), and hence many lossy compression algorithms remove low-frequency components or simply use a low bit-rate for them. In the context of elephant monitoring, this is a poor strategy since the elephants communicate by low-frequency rumbles. It is clear that we need to develop compression strategies uniquely suited for the elephant calls and for the neural networks that will analyze them. As neural networks are well known to be resistant to minor random perturbations (?) lossy compression is a promising avenue. Additionally, as passive acoustic monitoring has applications to many species, from small birds (?) to marine mammals (?), data-driven approaches such as ours avoids the laborious process of manually crafting audio codecs and can easily be adapted to new species. It does not require any hand-crafted features or any specific information regarding the structure of animal vocalization (save for an approximate frequency range, information that is easily obtainable for most species), and one would “only” need training data to adapt our framework to other species.

End-to-end differentiable compression codecs

As opposed to typical audio compression applications, the listener in our setup is not a human, additionally, the frequency spectrum is vastly different. To study this phenomenon in isolation and achieve a simple setup we only consider compression in terms of the different frequency bands. Other aspects of lossy compression, for example, lossless compression on top of lossy strategies, can be added to all methods we consider. We assume that the one-dimensional $X$ that describes the sound waves has been transformed via FFT as a pre-processing step into $\hat{X}$, and consider the problem of assigning bit-rates to the different frequency bands. Simple operations such as FFT and bit-truncation can easily be implemented on the rudimentary hardware of the recording devices. We propose a method that jointly optimizes for low bit-rates of the frequency channels and high classification accuracy.

Our algorithmic setup is illustrated in Figure 5. We want to assign different bit-rates to different frequency-channels, which we achieve by simply truncating the bit representation of elements of the channels, which lowers the precision. Our key insight is to exchange a non-differentiable bit-truncation by a differentiable proxy – we simply model truncation as additive Gaussian noise, a common model of quantization error (?). We let the components of the vector $\lambda$ denote the bit-rates of various frequency channels, and let $\beta$ be a matrix with dimensions ${t\times f}$ with independent standard Gaussian entries, where there are $t$ time-steps and $f$ frequency bands. The truncation error is the proportional to by the matrix $\exp(-\lambda)\odot\beta$, where the entries $(i,j)$ are equal to $\exp(-\lambda_{j})\beta_{ij}$. This ensures that the additive errors in the original elephant spectrogram $\hat{X}$, which models bit-truncation, are independent but that each frequency band has its own error scale. The input to the neural networks is thus $\hat{X}+\exp(-\lambda)\odot\beta$, and we simultaneously optimize the network parameters $\omega$ for large classification accuracy and the total bit-rate which is simply expressed as $\sum_{i}\lambda_{i}$, balancing these two objectives with the hyper-parameter $\mu$. The loss can be written as

[TABLE]

Here the dataset $D$ contains tuples $(\hat{X},y)$ of data $\hat{X}$ and labels $y$, $L(y,\hat{y})$ denotes the loss function used to measure goodness of fit between ground-truth label $y$ and estimated label $\hat{y}$. The function $\textrm{DNN}_{\omega}$ gives the output of the trained neural network with network parameters $\omega$. We again use the cross-entropy for the loss function. This function can be optimized via SGD, where we exchange the expectation $\mathbb{E}[\;\cdot\;]$ by sample averages.

Experiments

We compare different compression strategies by how well they transmit the important information as measured by how well a classifier can be trained to classify compressed elephant spectrograms given a fixed bit-rate. For all compression strategies, we will use the Densenet model of earlier sections. The original Fourier signal has elements put into one of the $2^{32}$ bins represented as 32 bit signed integers, lowering the bit-rate simply corresponds to removing the least significant bits with the sign bit is removed last. This has the effect of quantizing the signal and removing small variations in signal strength while keeping the large variations (see Figure 6). We enforce that no less than 5 bits are used for each frequency band as the dynamic range of the audio signal has the effect of completely erasing the signal for smaller bit-rates. For assigning bit-rates via optimizing (1) we use the same Densenet architecture as for evaluating the compression quality, and train it with the same parameters as in earlier sections and with $\mu=10^{-7}$. To ensure specific total bit-rates we assign bit-rates to various frequency bands proportional to the values of the components of $\lambda$. We compare our method against the method of assigning bit-rates proportional to the sensitivity of human hearing, using the well-known model of how human auditory sensitivity vary with frequency of (?). The proportional allocation excludes the 5 bits needed for the dynamic range of the signal. The results for various locations and bit-rates are given in Table 4, where we can clearly see that our proposed method achieves superior performance for the same bit-rates. For very small and very large bit-rates the difference becomes smaller. Implementing our method leads to data compression of a factor roughly 116 compared to näively storing the 1000Hz signal in 32-bit floating point numbers while achieving little performance degradation. These savings are significant for the often poor wireless networks of sub-Saharan Africa.

Related Work

Bioacoustics

The field of bioacoustics has for a long time been interested in automatic approaches towards detecting and classifying animal vocalizations with the ultimate goal to accurately survey population size and behavior (?). As sound waves attenuate less in water, passive acoustic monitoring can cover vast underwater areas. Much effort has been in terms of large marine animals with characteristic vocalizations – predominately various whale species (Humpback, right (?), Baleen (?), Blue and Fin (?)) and dolphins (?). Acoustic signals are the primary mode of communication for many marine species and for large gatherings vocalizations typically overlap which together with long reverberation times becomes challenging. Techniques used to overcome these issues include blind source separation (?), pitch-tracking via dynamic programming (?) and kernel methods (?).

On land, efforts towards bioacoustics have primarily focused on various bird species, owing to the characteristic songs many of them use for mating and communication. As bird species typically have unique songs, PAM makes it possible to accurately survey populations of endangered species, whereas using direct visual observations becomes problematic for species that are small and/or occupy canopies (?). Popular strategies include SVMs based upon MFCC (?), segmentation via deep learning (?) and dictionary learning (?). Beyond birds, insects (?), bats (?) and monkeys (?) have all been considered. Elephants have been studied from a similar perspective to ours by ?. For many of these species, especially many birds, the vocalizations occupy a relatively small frequency band making models less sensitive to noise and intra-population variability in vocalizations, hence making them unsuitable for elephant monitoring.

Machine-learning for Audio

Machine learning for audio-signals has primarily focused on human speech due to applications such as virtual assistants, automatic transcription, and translation. For a long time, mainstream research was primarily propelled by using the EM-algorithms for training Hidden-Markov-Models (?). Features for audio input could often be encoded via MFCC (?), and rich distributions could be represented via Gaussian-Mixture-Models (?). While using neural networks for acoustic applications was conceived more than 25 years ago (?), it was in only 2009 that deep learning approaches were shown to be competitive with more traditional “hand-crafted” machine learning approaches (?). Deep learning has now gained mainstream traction and it has become the dominant paradigm. State-of-the-art speech recognition often relies on recurrent neural networks (?) (?), where convolutional layers can automatically extract features (?). Beyond speech recognition, deep learning for acoustic sensing in smartphones has been investigated (?).

Compression

Compression for acoustic signals has been studied for a long time due to applications such as storing music on handheld devices and sending human conversations across networks, and many audio compression methods rely on essentially handcrafted features, for example, wavelets (?). Most methods for lossy compression of audio has the goal of ensuring signals are audible to humans, and hence most models are based upon the models of human hearing, so-called psychoacoustic models. A salient feature of human hearing is that its sensitivity varies with frequency (?), a common strategy is to transform the audio-signal with the modified discrete cosine transform (MDCT) and address frequency bands individually. Another phenomenon of human hearing is called simultaneous masking where signal $A$ can make signal $B$ (which is of a different frequency and intensity) inaudible (?).

While traditional compression schemes have typically relied on handcrafted features, the advent of deep learning has spurred interest in data-driven approaches to compression. Previous research has primarily focused on images and video, proposing various continuous and differentiable proxies for entropy and quantization, see for example (?) and (?). The only work on audio compression known to the authors is on human speech (?) which has is different in terms of frequency distribution, complexity, and dataset cleanliness; the proposed architecture relies on softmax quantization.

Future Work and Conclusions

Managers of protected areas designed for the forest elephants are interested in better conservation tools but need to see definitive proof of their efficacy. If useful information about elephant populations and human encroachments can reach managers within a reasonable timeframe, the potential to expand acoustic monitoring across the Congo Basin becomes a reality. Collaboration with managers is thus instrumental in developing a rapid work-flow for the current acoustic monitoring project in northern Congo, which covers 1500 square km of rainforest and generates seven terabytes of sound data quarterly. We hope that these proof-of-concept demonstrations of how various AI techniques can inspire future work on PAM, with the ultimate goal of real-world implementation.

In this work, we have introduced a dataset of elephant calls recorded in the wild aimed at promoting interest and progress in automatic methods for passive acoustic monitoring and discussed our methods for labeling it. Using modern neural network architecture, state-of-the-art training regimes and data-augmentation techniques we have shown how to improve upon previously proposed method for passive acoustic monitoring. Additionally, we have addressed how wireless network infrastructure is often lacking in sub-Saharan Africa data transfer quickly becomes a bottleneck for real-time systems. To circumvent this issue, we have introduced a novel scheme for jointly optimizing bit-rates and prediction accuracy, which beats a baseline based upon models of human hearing.

Acknowledgements

We would like to thanks the Elephant project, the Cornell Lab of Ornithology its volunteers and the Wildlife Conservation Society. This work is supported by NSF Expedition CCF-1522054 and ARO DURIP W911NF-17-1-0187. The work of PHW was supported by the U.S. Fish and Wildlife Service and the Robert G. and Jane V. Engel Foundation.

Appendix A Appendix