AURA: Autonomous Upskilling with Retrieval-Augmented Agents

TL;DR

AURA is a novel framework that uses retrieval-augmented large language models to autonomously design, validate, and refine reinforcement learning curricula for agile robots, enabling scalable and adaptive policy training from user prompts.

Contribution

AURA introduces an automated, schema-validated curriculum RL framework leveraging LLMs and retrieval feedback, reducing manual tuning and enabling zero-shot deployment on robots.

Findings

Outperforms LLM-guided baselines in success rate, locomotion, and manipulation.

Schema validation and retrieval are crucial for curriculum quality.

Successfully trains and deploys policies directly from user prompts.

Abstract

Designing reinforcement learning curricula for agile robots traditionally requires extensive manual tuning of reward functions, environment randomizations, and training configurations. We introduce AURA (Autonomous Upskilling with Retrieval-Augmented Agents), a schema-validated curriculum reinforcement learning (RL) framework that leverages Large Language Models (LLMs) as autonomous designers of multi-stage curricula. AURA transforms user prompts into YAML workflows that encode full reward functions, domain randomization strategies, and training configurations. All files are statically validated before any GPU time is used, ensuring efficient and reliable execution. A retrieval-augmented feedback loop allows specialized LLM agents to design, execute, and refine curriculum stages based on prior training results stored in a vector database, enabling continual improvement over time. Quantitative experiments show that AURA consistently outperforms LLM-guided baselines in generation success rate, humanoid locomotion, and manipulation tasks. Ablation studies highlight the importance of schema validation and retrieval for curriculum quality. AURA successfully trains end-to-end policies directly from user prompts and deploys them zero-shot on a custom humanoid robot in multiple environments - capabilities that did not exist previously with manually designed controllers. By abstracting the complexity of curriculum design, AURA enables scalable and adaptive policy learning pipelines that would be complex to construct by hand. Project page: https://aura-research.org/