Multi-Agent Pose Uncertainty: A Differentiable Rendering Cram\'er-Rao Bound

TL;DR

This paper introduces a differentiable rendering-based Cramér-Rao bound to quantify uncertainty in camera pose estimates, extending classical methods to multi-agent systems and applications like cooperative perception.

Contribution

It derives a closed-form lower bound on pose covariance using differentiable rendering, bridging classical bundle adjustment and multi-agent uncertainty quantification.

Findings

Provides a render-aware Cramér-Rao bound for pose uncertainty

Extends uncertainty quantification to multi-agent camera systems

Enables applications like cooperative perception without keypoint correspondences

Abstract

Pose estimation is essential for many applications within computer vision and robotics. Despite its uses, few works provide rigorous uncertainty quantification for poses under dense or learned models. We derive a closed-form lower bound on the covariance of camera pose estimates by treating a differentiable renderer as a measurement function. Linearizing image formation with respect to a small pose perturbation on the manifold yields a render-aware Cram\'er-Rao bound. Our approach reduces to classical bundle-adjustment uncertainty, ensuring continuity with vision theory. It also naturally extends to multi-agent settings by fusing Fisher information across cameras. Our statistical formulation has downstream applications for tasks such as cooperative perception and novel view synthesis without requiring explicit keypoint correspondences.