GSL-PCD: Improving Generalist-Specialist Learning with Point Cloud Feature-based Task Partitioning

TL;DR

This paper introduces GSL-PCD, a method that enhances generalist-specialist learning in deep reinforcement learning by using point cloud features for task partitioning, leading to better performance and efficiency.

Contribution

It proposes a novel point cloud feature-based clustering method for task partitioning in GSL, improving specialization and reducing computational costs.

Findings

Point cloud feature-based partitioning outperforms vanilla methods by 9.4%.

GSL-PCD reduces computational and sample costs by 50%.

Achieves comparable performance with fewer resources.

Abstract

Generalization in Deep Reinforcement Learning (DRL) across unseen environment variations often requires training over a diverse set of scenarios. Many existing DRL algorithms struggle with efficiency when handling numerous variations. The Generalist-Specialist Learning (GSL) framework addresses this by first training a generalist model on all variations, then creating specialists from the generalist's weights, each focusing on a subset of variations. The generalist then refines its learning with assistance from the specialists. However, random task partitioning in GSL can impede performance by assigning vastly different variations to the same specialist, often resulting in each specialist focusing on only one variation, which raises computational costs. To improve this, we propose Generalist-Specialist Learning with Point Cloud Feature-based Task Partitioning (GSL-PCD). Our approach clusters environment variations based on features extracted from object point clouds and uses balanced clustering with a greedy algorithm to assign similar variations to the same specialist. Evaluations on robotic manipulation tasks from the ManiSkill benchmark demonstrate that point cloud feature-based partitioning outperforms vanilla partitioning by 9.4%, with a fixed number of specialists, and reduces computational and sample requirements by 50% to achieve comparable performance.