Towards Exploiting Geometry and Time for Fast Off-Distribution Adaptation in Multi-Task Robot Learning

TL;DR

This paper investigates methods for rapid adaptation in multi-task robot learning by leveraging shared physical structures, temporal coherence, and base policies to efficiently learn new off-distribution tasks with minimal data.

Contribution

It introduces simple architectural approaches for reusing policies as priors and exploiting geometry and time to improve off-distribution task adaptation in robotics.

Findings

Combining low-complexity policies with base priors enables quick adaptation.

Alignment of observation or action spaces facilitates transfer across tasks.

Temporal switching policies improve adaptation efficiency.

Abstract

We explore possible methods for multi-task transfer learning which seek to exploit the shared physical structure of robotics tasks. Specifically, we train policies for a base set of pre-training tasks, then experiment with adapting to new off-distribution tasks, using simple architectural approaches for re-using these policies as black-box priors. These approaches include learning an alignment of either the observation space or action space from a base to a target task to exploit rigid body structure, and methods for learning a time-domain switching policy across base tasks which solves the target task, to exploit temporal coherence. We find that combining low-complexity target policy classes, base policies as black-box priors, and simple optimization algorithms allows us to acquire new tasks outside the base task distribution, using small amounts of offline training data.