# AI-Based Multi-Object Relative State Estimation with Self-Calibration Capabilities

## TL;DR

This paper presents a novel AI-based multi-object relative state estimation method that fuses visual and inertial data, enabling accurate, self-calibrating localization for mobile robots in complex environments.

## Contribution

It introduces a combined AI pose estimator with sensor fusion that is capable of self-calibration for multi-object relative state estimation in real-world scenarios.

## Key findings

- Reliable multi-object relative state estimation demonstrated in experiments.
- Self-calibration improves robustness and reproducibility.
- Effective fusion of AI-based visual poses with IMU data.

## Abstract

The capability to extract task specific, semantic information from raw sensory data is a crucial requirement for many applications of mobile robotics. Autonomous inspection of critical infrastructure with Unmanned Aerial Vehicles (UAVs), for example, requires precise navigation relative to the structure that is to be inspected. Recently, Artificial Intelligence (AI)-based methods have been shown to excel at extracting semantic information such as 6 degree-of-freedom (6-DoF) poses of objects from images. In this paper, we propose a method combining a state-of-the-art AI-based pose estimator for objects in camera images with data from an inertial measurement unit (IMU) for 6-DoF multi-object relative state estimation of a mobile robot. The AI-based pose estimator detects multiple objects of interest in camera images along with their relative poses. These measurements are fused with IMU data in a state-of-the-art sensor fusion framework. We illustrate the feasibility of our proposed method with real world experiments for different trajectories and number of arbitrarily placed objects. We show that the results can be reliably reproduced due to the self-calibrating capabilities of our approach.