Stereo Co-capture System for Recording and Tracking Fish with Frame- and Event Cameras

TL;DR

This paper presents a co-capture system combining conventional and event cameras to record and track fast-moving fish, leveraging event cameras' high temporal resolution and developing a new multi-animal tracking algorithm.

Contribution

It introduces a novel multi-animal recording system and an event-based tracking algorithm, demonstrating the potential of combining event and conventional cameras for aquatic animal monitoring.

Findings

Event cameras effectively capture rapid fish movements.

The tracking algorithm establishes a baseline for multi-animal tracking.

The system demonstrates feasibility for high-speed aquatic animal monitoring.

Abstract

This work introduces a co-capture system for multi-animal visual data acquisition using conventional cameras and event cameras. Event cameras offer multiple advantages over frame-based cameras, such as a high temporal resolution and temporal redundancy suppression, which enable us to efficiently capture the fast and erratic movements of fish. We furthermore present an event-based multi-animal tracking algorithm, which proves the feasibility of the approach and sets the baseline for further exploration of combining the advantages of event cameras and conventional cameras for multi-animal tracking.