FutureMapping: The Computational Structure of Spatial AI Systems

TL;DR

This paper explores the future evolution of Spatial AI, emphasizing the need for co-designed algorithms, hardware, and sensors to achieve advanced perception capabilities in embodied devices.

Contribution

It analyzes the computational structure of current and future Spatial AI algorithms within the context of hardware development and co-design strategies.

Findings

Spatial AI will evolve from SLAM to more general perception capabilities.

Co-design of algorithms, hardware, and sensors is essential for future Spatial AI.

Current hardware constraints limit perception performance in consumer devices.

Abstract

We discuss and predict the evolution of Simultaneous Localisation and Mapping (SLAM) into a general geometric and semantic `Spatial AI' perception capability for intelligent embodied devices. A big gap remains between the visual perception performance that devices such as augmented reality eyewear or comsumer robots will require and what is possible within the constraints imposed by real products. Co-design of algorithms, processors and sensors will be needed. We explore the computational structure of current and future Spatial AI algorithms and consider this within the landscape of ongoing hardware developments.