State Representation Learning from Demonstration

TL;DR

This paper introduces SRLfD, a method for learning compact, task-relevant state representations from demonstrations using a multi-head neural network, improving robot adaptability and reinforcement learning efficiency.

Contribution

The paper presents SRLfD, a novel imitation learning approach that trains shared representations across multiple tasks to enhance robot state understanding and learning efficiency.

Findings

SRLfD produces more compact and relevant state representations.

Controllers using SRLfD outperform other strategies in experiments.

SRLfD facilitates more efficient reinforcement learning.

Abstract

Robots could learn their own state and world representation from perception and experience without supervision. This desirable goal is the main focus of our field of interest, state representation learning (SRL). Indeed, a compact representation of such a state is beneficial to help robots grasp onto their environment for interacting. The properties of this representation have a strong impact on the adaptive capability of the agent. In this article we present an approach based on imitation learning. The idea is to train several policies that share the same representation to reproduce various demonstrations. To do so, we use a multi-head neural network with a shared state representation feeding a task-specific agent. If the demonstrations are diverse, the trained representation will eventually contain the information necessary for all tasks, while discarding irrelevant information. As such, it will potentially become a compact state representation useful for new tasks. We call this approach SRLfD (State Representation Learning from Demonstration). Our experiments confirm that when a controller takes SRLfD-based representations as input, it can achieve better performance than with other representation strategies and promote more efficient reinforcement learning (RL) than with an end-to-end RL strategy.