Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping

TL;DR

Kimera is an open-source C++ library that enables real-time metric-semantic SLAM with mesh reconstruction and semantic labeling, integrating visual-inertial odometry, pose graph optimization, and 3D reconstruction.

Contribution

It introduces a modular, real-time, open-source SLAM library that combines metric and semantic mapping with mesh reconstruction, surpassing existing SLAM tools.

Findings

Runs in real-time on CPU

Produces 3D metric-semantic meshes

Flexible modular architecture

Abstract

We provide an open-source C++ library for real-time metric-semantic visual-inertial Simultaneous Localization And Mapping (SLAM). The library goes beyond existing visual and visual-inertial SLAM libraries (e.g., ORB-SLAM, VINS- Mono, OKVIS, ROVIO) by enabling mesh reconstruction and semantic labeling in 3D. Kimera is designed with modularity in mind and has four key components: a visual-inertial odometry (VIO) module for fast and accurate state estimation, a robust pose graph optimizer for global trajectory estimation, a lightweight 3D mesher module for fast mesh reconstruction, and a dense 3D metric-semantic reconstruction module. The modules can be run in isolation or in combination, hence Kimera can easily fall back to a state-of-the-art VIO or a full SLAM system. Kimera runs in real-time on a CPU and produces a 3D metric-semantic mesh from semantically labeled images, which can be obtained by modern deep learning methods. We hope that the flexibility, computational efficiency, robustness, and accuracy afforded by Kimera will build a solid basis for future metric-semantic SLAM and perception research, and will allow researchers across multiple areas (e.g., VIO, SLAM, 3D reconstruction, segmentation) to benchmark and prototype their own efforts without having to start from scratch.