ConvNets for Counting: Object Detection of Transient Phenomena in Steelpan Drums

TL;DR

This paper develops a convolutional neural network-based object detector to count interference fringes in steelpan drum videos, aiding the analysis of transient vibrations and sympathetic oscillations, with implications for understanding drum acoustics.

Contribution

It introduces a novel CNN-based detection system trained on real and synthetic images to analyze transient phenomena in steelpan drums, combining crowdsourced annotations and style transfer techniques.

Findings

Detected oscillations align with audio recordings

Sympathetic oscillations precede sound intensity rise

Model performs well on real and synthetic images

Abstract

We train an object detector built from convolutional neural networks to count interference fringes in elliptical antinode regions in frames of high-speed video recordings of transient oscillations in Caribbean steelpan drums illuminated by electronic speckle pattern interferometry (ESPI). The annotations provided by our model aim to contribute to the understanding of time-dependent behavior in such drums by tracking the development of sympathetic vibration modes. The system is trained on a dataset of crowdsourced human-annotated images obtained from the Zooniverse Steelpan Vibrations Project. Due to the small number of human-annotated images and the ambiguity of the annotation task, we also evaluate the model on a large corpus of synthetic images whose properties have been matched to the real images by style transfer using a Generative Adversarial Network. Applying the model to thousands of unlabeled video frames, we measure oscillations consistent with audio recordings of these drum strikes. One unanticipated result is that sympathetic oscillations of higher-octave notes significantly precede the rise in sound intensity of the corresponding second harmonic tones; the mechanism responsible for this remains unidentified. This paper primarily concerns the development of the predictive model; further exploration of the steelpan images and deeper physical insights await its further application.