Metric, inertially aligned monocular state estimation via kinetodynamic priors

TL;DR

This paper presents a novel monocular state estimation method for non-rigid robotic systems that combines deformation modeling and continuous-time kinematic representations to improve accuracy and recover inertial properties.

Contribution

It introduces a physics-based approach integrating deformation-force models with B-spline kinematics for robust non-rigid pose estimation from monocular vision.

Findings

Enables accurate pose estimation on deformable platforms

Allows recovery of inertial sensing properties from monocular data

Successfully resolves scale and gravity in visual odometry

Abstract

Accurate state estimation for flexible robotic systems poses significant challenges, particularly for platforms with dynamically deforming structures that invalidate rigid-body assumptions. This paper addresses this problem and enables the extension of existing rigid-body pose estimation methods to non-rigid systems. Our approach integrates two core components: first, we capture elastic properties using a deformation-force model, efficiently learned via a Multi-Layer Perceptron; second, we resolve the platform's inherently smooth motion using continuous-time B-spline kinematic models. By continuously applying Newton's Second Law, our method formulates the relationship between visually-derived trajectory acceleration and predicted deformation-induced acceleration. We demonstrate that our approach not only enables robust and accurate pose estimation on non-rigid platforms, but also shows that the properly modeled platform physics allow for the recovery of inertial sensing properties. We validate this feasibility on a simple spring-camera system, showing how it robustly resolves the typically ill-posed problem of metric scale and gravity recovery in monocular visual odometry.