Projected Task-Specific Layers for Multi-Task Reinforcement Learning

TL;DR

This paper introduces Projected Task-Specific Layers (PTSL), a new architecture for multi-task reinforcement learning that improves task generalization and reduces interference by combining shared policies with task-specific corrections, outperforming previous methods on Meta-World benchmarks.

Contribution

The paper proposes PTSL, a novel architecture that effectively captures shared and task-specific information using dense corrections, advancing multi-task reinforcement learning capabilities.

Findings

PTSL outperforms state-of-the-art methods on Meta-World MT10 and MT50 benchmarks.

The architecture effectively captures shared and task-specific features.

Improved task generalization and reduced interference demonstrated.

Abstract

Multi-task reinforcement learning could enable robots to scale across a wide variety of manipulation tasks in homes and workplaces. However, generalizing from one task to another and mitigating negative task interference still remains a challenge. Addressing this challenge by successfully sharing information across tasks will depend on how well the structure underlying the tasks is captured. In this work, we introduce our new architecture, Projected Task-Specific Layers (PTSL), that leverages a common policy with dense task-specific corrections through task-specific layers to better express shared and variable task information. We then show that our model outperforms the state of the art on the MT10 and MT50 benchmarks of Meta-World consisting of 10 and 50 goal-conditioned tasks for a Sawyer arm.