Human Decision Makings on Curriculum Reinforcement Learning with Difficulty Adjustment

TL;DR

This paper introduces an interactive platform for human-in-the-loop curriculum reinforcement learning, enabling personalized difficulty adjustment to match human preferences and improve learning efficiency.

Contribution

It presents a portable, parallelizable system that allows humans to guide RL training by manipulating difficulty, demonstrating effective personalized curriculum adaptation.

Findings

Reinforcement learning performance aligns with human difficulty preferences.

The system enables large-scale RL training without a server.

Interactive curriculum improves learning efficiency and personalization.

Abstract

Human-centered AI considers human experiences with AI performance. While abundant research has been helping AI achieve superhuman performance either by fully automatic or weak supervision learning, fewer endeavors are experimenting with how AI can tailor to humans' preferred skill level given fine-grained input. In this work, we guide the curriculum reinforcement learning results towards a preferred performance level that is neither too hard nor too easy via learning from the human decision process. To achieve this, we developed a portable, interactive platform that enables the user to interact with agents online via manipulating the task difficulty, observing performance, and providing curriculum feedback. Our system is highly parallelizable, making it possible for a human to train large-scale reinforcement learning applications that require millions of samples without a server. The result demonstrates the effectiveness of an interactive curriculum for reinforcement learning involving human-in-the-loop. It shows reinforcement learning performance can successfully adjust in sync with the human desired difficulty level. We believe this research will open new doors for achieving flow and personalized adaptive difficulties.