The Semantic Lifecycle in Embodied AI: Acquisition, Representation and Storage via Foundation Models

TL;DR

This paper introduces the Semantic Lifecycle framework to understand how foundation models enhance semantic knowledge processing in embodied AI, covering acquisition, representation, and storage for more robust perception-action systems.

Contribution

It proposes a holistic Semantic Lifecycle framework that unifies semantic knowledge evolution in embodied AI driven by foundation models, moving beyond isolated modules.

Findings

Analyzes recent advances in semantic acquisition, representation, and storage.

Highlights challenges and future directions in semantic processing for embodied AI.

Abstract

Semantic information in embodied AI is inherently multi-source and multi-stage, making it challenging to fully leverage for achieving stable perception-to-action loops in real-world environments. Early studies have combined manual engineering with deep neural networks, achieving notable progress in specific semantic-related embodied tasks. However, as embodied agents encounter increasingly complex environments and open-ended tasks, the demand for more generalizable and robust semantic processing capabilities has become imperative. Recent advances in foundation models (FMs) address this challenge through their cross-domain generalization abilities and rich semantic priors, reshaping the landscape of embodied AI research. In this survey, we propose the Semantic Lifecycle as a unified framework to characterize the evolution of semantic knowledge within embodied AI driven by foundation models. Departing from traditional paradigms that treat semantic processing as isolated modules or disjoint tasks, our framework offers a holistic perspective that captures the continuous flow and maintenance of semantic knowledge. Guided by this embodied semantic lifecycle, we further analyze and compare recent advances across three key stages: acquisition, representation, and storage. Finally, we summarize existing challenges and outline promising directions for future research.