Real-World Robot Applications of Foundation Models: A Review

TL;DR

This review explores how foundation models like LLMs and VLMs are practically applied in robotics, focusing on replacing components and enhancing perception, planning, and control in real-world robotic systems.

Contribution

It provides a comprehensive overview of the application of foundation models in robotics, highlighting their role in component replacement and system enhancement.

Findings

Foundation models are increasingly used to replace traditional robot components.

They improve perception, motion planning, and control in robotic systems.

Future challenges include integration and real-world deployment issues.

Abstract

Recent developments in foundation models, like Large Language Models (LLMs) and Vision-Language Models (VLMs), trained on extensive data, facilitate flexible application across different tasks and modalities. Their impact spans various fields, including healthcare, education, and robotics. This paper provides an overview of the practical application of foundation models in real-world robotics, with a primary emphasis on the replacement of specific components within existing robot systems. The summary encompasses the perspective of input-output relationships in foundation models, as well as their role in perception, motion planning, and control within the field of robotics. This paper concludes with a discussion of future challenges and implications for practical robot applications.