Agentic Skill Discovery

TL;DR

This paper presents a novel LLM-driven framework for autonomous skill discovery in robots, enabling the emergence of diverse, meaningful, and reliable skills from zero initial capabilities through iterative task proposals and reinforcement learning.

Contribution

The proposed framework leverages LLMs to autonomously generate task proposals and guide reinforcement learning, facilitating the emergence of a versatile skill library without manual decomposition.

Findings

Skills emerge and expand from zero initial capabilities.

The framework produces meaningful and reliable robotic skills.

Robots can efficiently propose and complete complex tasks.

Abstract

Language-conditioned robotic skills make it possible to apply the high-level reasoning of Large Language Models (LLMs) to low-level robotic control. A remaining challenge is to acquire a diverse set of fundamental skills. Existing approaches either manually decompose a complex task into atomic robotic actions in a top-down fashion, or bootstrap as many combinations as possible in a bottom-up fashion to cover a wider range of task possibilities. These decompositions or combinations, however, require an initial skill library. For example, a ``grasping'' capability can never emerge from a skill library containing only diverse ``pushing'' skills. Existing skill discovery techniques with reinforcement learning acquire skills by an exhaustive exploration but often yield non-meaningful behaviors. In this study, we introduce a novel framework for skill discovery that is entirely driven by LLMs. The framework begins with an LLM generating task proposals based on the provided scene description and the robot's configurations, aiming to incrementally acquire new skills upon task completion. For each proposed task, a series of reinforcement learning processes are initiated, utilizing reward and success determination functions sampled by the LLM to develop the corresponding policy. The reliability and trustworthiness of learned behaviors are further ensured by an independent vision-language model. We show that starting with zero skill, the skill library emerges and expands to more and more meaningful and reliable skills, enabling the robot to efficiently further propose and complete advanced tasks. Project page: \url{https://agentic-skill-discovery.github.io}.