The Last Mile to Production Readiness: Physics-Based Motion Refinement for Video-Based Capture

TL;DR

This paper introduces a physics-based motion refinement framework that improves the physical realism and production readiness of video-based motion capture data, reducing manual cleanup and supporting integration into animator workflows.

Contribution

It presents a novel physics-based approach for motion refinement that enhances realism and can be integrated into existing animation workflows, addressing a key challenge in motion capture.

Findings

Reduces manual cleanup effort in motion capture workflows.

Enhances physical realism and visual quality of captured motions.

Supports both single- and multi-character sequences.

Abstract

High-quality motion data underpins games, film, XR, and robotics. Vision-based motion capture tools have made significant progress, offering accessible and visually convincing results, yet often fall short in the final stretch -- the last mile -- when it comes to physical realism and production readiness, due to various artifacts introduced during capture. In this paper, we summarize key issues through case studies and feedback from professional animators to set a stepping stone for future research in motion cleanup. We then present a physics-based motion refinement framework to bridge the gap, with the goal of reducing labor-intensive manual cleanup and enhancing visual quality and physical realism. Our framework supports both single- and multi-character sequences and can be integrated into animator workflows for further refinement, such as stylizing motions via keyframe editing.