Large Language Models for Robotics: A Survey

TL;DR

This survey reviews how large language models are increasingly integrated into robotics, enhancing capabilities like control, perception, and interaction, and discusses recent advancements, applications, and future challenges in this rapidly evolving field.

Contribution

It provides a comprehensive overview of LLM applications in robotics, highlighting recent developments, techniques, and potential challenges for embodied intelligence.

Findings

LLMs improve robot-human interaction and autonomy.

Recent advancements enable better perception and decision-making.

Challenges include integration complexity and real-world deployment issues.

Abstract

The human ability to learn, generalize, and control complex manipulation tasks through multi-modality feedback suggests a unique capability, which we refer to as dexterity intelligence. Understanding and assessing this intelligence is a complex task. Amidst the swift progress and extensive proliferation of large language models (LLMs), their applications in the field of robotics have garnered increasing attention. LLMs possess the ability to process and generate natural language, facilitating efficient interaction and collaboration with robots. Researchers and engineers in the field of robotics have recognized the immense potential of LLMs in enhancing robot intelligence, human-robot interaction, and autonomy. Therefore, this comprehensive review aims to summarize the applications of LLMs in robotics, delving into their impact and contributions to key areas such as robot control, perception, decision-making, and planning. This survey first provides an overview of the background and development of LLMs for robotics, followed by a discussion of their benefits and recent advancements in LLM-based robotic models. It then explores various techniques, employed in perception, decision-making, control, and interaction, as well as cross-module coordination in practical tasks. Finally, we review current applications of LLMs in robotics and outline potential challenges they may face in the near future. Embodied intelligence represents the future of intelligent systems, and LLM-based robotics is one of the most promising yet challenging paths toward achieving it.