World Models: The Safety Perspective

TL;DR

This paper reviews the role of World Models in AI safety, emphasizing their importance in predicting environmental states for safe decision-making, and discusses current challenges and future research directions.

Contribution

It provides a comprehensive survey and analysis of current World Models, highlighting safety concerns and proposing research challenges to enhance trustworthiness.

Findings

Current WMs are crucial for safe AI planning.

Safety and trustworthiness are key challenges in WM development.

The paper calls for collaborative research to improve WM safety.

Abstract

With the proliferation of the Large Language Model (LLM), the concept of World Models (WM) has recently attracted a great deal of attention in the AI research community, especially in the context of AI agents. It is arguably evolving into an essential foundation for building AI agent systems. A WM is intended to help the agent predict the future evolution of environmental states or help the agent fill in missing information so that it can plan its actions and behave safely. The safety property of WM plays a key role in their effective use in critical applications. In this work, we review and analyze the impacts of the current state-of-the-art in WM technology from the point of view of trustworthiness and safety based on a comprehensive survey and the fields of application envisaged. We provide an in-depth analysis of state-of-the-art WMs and derive technical research challenges and their impact in order to call on the research community to collaborate on improving the safety and trustworthiness of WM.