Tracking Mouse from Incomplete Body-Part Observations and Deep-Learned Deformable-Mouse Model Motion-Track Constraint for Behavior Analysis

TL;DR

This paper presents a method for tracking mouse body parts in videos with occlusions by integrating multiple camera views, 3D reconstruction, and a deep-learned deformable model to improve behavior analysis accuracy.

Contribution

It introduces a novel approach combining multi-view integration, 3D triangulation, and a deep-learned deformable mouse model for more complete and accurate body part tracking.

Findings

Enhanced 3D body and part tracking accuracy

More complete body part observations than single-frame detection

Improved animal behavior analysis capabilities

Abstract

Tracking mouse body parts in video is often incomplete due to occlusions such that - e.g. - subsequent action and behavior analysis is impeded. In this conceptual work, videos from several perspectives are integrated via global exterior camera orientation; body part positions are estimated by 3D triangulation and bundle adjustment. Consistency of overall 3D track reconstruction is achieved by introduction of a 3D mouse model, deep-learned body part movements, and global motion-track smoothness constraint. The resulting 3D body and body part track estimates are substantially more complete than the original single-frame-based body part detection, therefore, allowing improved animal behavior analysis.