Perception with Guarantees: Certified Pose Estimation via Reachability Analysis

TL;DR

This paper introduces a method for certified 3D pose estimation from camera images using reachability analysis, ensuring safety guarantees in critical applications without relying on external trustworthy sensors.

Contribution

It presents a novel approach combining reachability analysis and neural network verification to provide formal safety guarantees in pose estimation from visual data.

Findings

Efficient and accurate localization demonstrated in synthetic experiments.

Achieved safety guarantees without external trustworthy sensors.

Applicable to real-world scenarios with complex geometries.

Abstract

Agents in cyber-physical systems are increasingly entrusted with safety-critical tasks. Ensuring safety of these agents often requires localizing the pose for subsequent actions. Pose estimates can, e.g., be obtained from various combinations of lidar sensors, cameras, and external services such as GPS. Crucially, in safety-critical domains, a rough estimate is insufficient to formally determine safety, i.e., guaranteeing safety even in the worst-case scenario, and external services might additionally not be trustworthy. We address this problem by presenting a certified pose estimation in 3D solely from a camera image and a well-known target geometry. This is realized by formally bounding the pose, which is computed by leveraging recent results from reachability analysis and formal neural network verification. Our experiments demonstrate that our approach efficiently and accurately localizes agents in both synthetic and real-world experiments.