Learning Versatile Skills with Curriculum Masking
Yao Tang, Zhihui Xie, Zichuan Lin, Deheng Ye, Shuai Li

TL;DR
CurrMask is a curriculum-based masking pretraining method that dynamically adjusts masking schemes during offline RL training, enabling the model to learn versatile skills of varying complexity and perform well on multiple downstream tasks.
Contribution
We introduce CurrMask, a novel curriculum masking approach that improves skill learning in offline RL by dynamically adjusting masking schemes during pretraining.
Findings
Superior zero-shot performance on skill prompting tasks
Effective goal-conditioned planning results
Competitive finetuning performance on offline RL tasks
Abstract
Masked prediction has emerged as a promising pretraining paradigm in offline reinforcement learning (RL) due to its versatile masking schemes, enabling flexible inference across various downstream tasks with a unified model. Despite the versatility of masked prediction, it remains unclear how to balance the learning of skills at different levels of complexity. To address this, we propose CurrMask, a curriculum masking pretraining paradigm for sequential decision making. Motivated by how humans learn by organizing knowledge in a curriculum, CurrMask adjusts its masking scheme during pretraining for learning versatile skills. Through extensive experiments, we show that CurrMask exhibits superior zero-shot performance on skill prompting tasks, goal-conditioned planning tasks, and competitive finetuning performance on offline RL tasks. Additionally, our analysis of training dynamics reveals…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsEducation Systems and Policy
