A Primer on Motion Capture with Deep Learning: Principles, Pitfalls and Perspectives

TL;DR

This paper reviews the emerging field of deep learning-based motion capture, discussing its principles, potential, pitfalls, and future perspectives to aid researchers in neuroscience and biology.

Contribution

It provides a comprehensive overview of deep learning methods for motion capture, highlighting their principles, challenges, and future directions.

Findings

Deep learning has significantly advanced video-based posture prediction.

Current algorithms offer promising tools but have notable pitfalls for experimental use.

Future research directions include improving accuracy and addressing limitations.

Abstract

Extracting behavioral measurements non-invasively from video is stymied by the fact that it is a hard computational problem. Recent advances in deep learning have tremendously advanced predicting posture from videos directly, which quickly impacted neuroscience and biology more broadly. In this primer we review the budding field of motion capture with deep learning. In particular, we will discuss the principles of those novel algorithms, highlight their potential as well as pitfalls for experimentalists, and provide a glimpse into the future.