Fourier-based Action Recognition for Wildlife Behavior Quantification with Event Cameras

TL;DR

This paper introduces Fourier Transform-based methods for wildlife behavior recognition using event cameras, effectively identifying oscillating motions like penguin wing flaps with minimal parameters, offering a simple alternative to neural networks.

Contribution

The paper presents a novel Fourier-based approach for action recognition in event camera data, focusing on oscillating behaviors and demonstrating effectiveness with significantly fewer parameters than deep neural networks.

Findings

Effective recognition of penguin wing-flapping behavior

Achieves comparable results to deep neural networks with far fewer parameters

Works well on diverse, uncontrolled wildlife data

Abstract

Event cameras are novel bio-inspired vision sensors that measure pixel-wise brightness changes asynchronously instead of images at a given frame rate. They offer promising advantages, namely a high dynamic range, low latency, and minimal motion blur. Modern computer vision algorithms often rely on artificial neural network approaches, which require image-like representations of the data and cannot fully exploit the characteristics of event data. We propose approaches to action recognition based on the Fourier Transform. The approaches are intended to recognize oscillating motion patterns commonly present in nature. In particular, we apply our approaches to a recent dataset of breeding penguins annotated for "ecstatic display", a behavior where the observed penguins flap their wings at a certain frequency. We find that our approaches are both simple and effective, producing slightly lower results than a deep neural network (DNN) while relying just on a tiny fraction of the parameters compared to the DNN (five orders of magnitude fewer parameters). They work well despite the uncontrolled, diverse data present in the dataset. We hope this work opens a new perspective on event-based processing and action recognition.