Humans as Checkerboards: Calibrating Camera Motion Scale for World-Coordinate Human Mesh Recovery

TL;DR

This paper introduces HAC, an optimization-free framework that uses human body geometry to calibrate camera motion scale, significantly improving global human mesh recovery accuracy and efficiency from monocular videos.

Contribution

HAC leverages human body geometry as a calibration reference to estimate camera scale without optimization, enabling fast and accurate global human motion estimation.

Findings

Reduces motion errors by 50% compared to prior methods.

Achieves 100× faster inference than optimization-based approaches.

Sets new state-of-the-art in global human mesh estimation.

Abstract

Accurate camera motion estimation is essential for recovering global human motion in world coordinates from RGB video inputs. SLAM is widely used for estimating camera trajectory and point cloud, but monocular SLAM does so only up to an unknown scale factor. Previous works estimate the scale factor through optimization, but this is unreliable and time-consuming. This paper presents an optimization-free scale calibration framework, Human as Checkerboard (HAC). HAC innovatively leverages the human body predicted by human mesh recovery model as a calibration reference. Specifically, it uses the absolute depth of human-scene contact joints as references to calibrate the corresponding relative scene depth from SLAM. HAC benefits from geometric priors encoded in human mesh recovery models to estimate the SLAM scale and achieves precise global human motion estimation. Simple yet powerful, our method sets a new state-of-the-art performance for global human mesh estimation tasks, reducing motion errors by 50% over prior local-to-global methods while using 100$\times$ less inference time than optimization-based methods. Project page: https://martayang.github.io/HAC.