Leaving Flatland: Advances in 3D behavioral measurement

TL;DR

This paper discusses recent advances in 3D behavioral measurement techniques that leverage deep learning and computer vision to improve tracking of animal movement in natural environments, enabling new biological and artificial intelligence applications.

Contribution

It highlights progress in 3D animal tracking methods that reduce hardware dependence and expand applicability across species and environments.

Findings

Enhanced 3D tracking accuracy in natural settings

Reduced training data requirements for deep learning models

Broader applicability across species and environments

Abstract

Animals move in three dimensions (3D). Thus, 3D measurement is necessary to report the true kinematics of animal movement. Existing 3D measurement techniques draw on specialized hardware, such as motion capture or depth cameras, as well as deep multi-view and monocular computer vision. Continued advances at the intersection of deep learning and computer vision will facilitate 3D tracking across more anatomical features, with less training data, in additional species, and within more natural, occlusive environments. 3D behavioral measurement enables unique applications in phenotyping, investigating the neural basis of behavior, and designing artificial agents capable of imitating animal behavior.