Interactive LLM-assisted Curriculum Learning for Multi-Task Evolutionary Policy Search

TL;DR

This paper introduces an interactive, real-time LLM-assisted curriculum learning framework for multi-task policy search, demonstrating improved performance over static methods through adaptive feedback in a robot navigation case study.

Contribution

The work presents a novel interactive LLM-based curriculum generation approach that leverages real-time feedback, enhancing multi-task policy search beyond static and expert-designed curricula.

Findings

Interactive curricula outperform static LLM-generated ones.

Multimodal feedback improves curriculum quality.

Performance is competitive with expert-designed curricula.

Abstract

Multi-task policy search is a challenging problem because policies are required to generalize beyond training cases. Curriculum learning has proven to be effective in this setting, as it introduces complexity progressively. However, designing effective curricula is labor-intensive and requires extensive domain expertise. LLM-based curriculum generation has only recently emerged as a potential solution, but was limited to operate in static, offline modes without leveraging real-time feedback from the optimizer. Here we propose an interactive LLM-assisted framework for online curriculum generation, where the LLM adaptively designs training cases based on real-time feedback from the evolutionary optimization process. We investigate how different feedback modalities, ranging from numeric metrics alone to combinations with plots and behavior visualizations, influence the LLM ability to generate meaningful curricula. Through a 2D robot navigation case study, tackled with genetic programming as optimizer, we evaluate our approach against static LLM-generated curricula and expert-designed baselines. We show that interactive curriculum generation outperforms static approaches, with multimodal feedback incorporating both progression plots and behavior visualizations yielding performance competitive with expert-designed curricula. This work contributes to understanding how LLMs can serve as interactive curriculum designers for embodied AI systems, with potential extensions to broader evolutionary robotics applications.