Learning to Actively Reduce Memory Requirements for Robot Control Tasks

TL;DR

This paper introduces a reinforcement learning framework that synthesizes policies with low-dimensional, task-specific memory representations, significantly reducing memory usage in robot control tasks while maintaining performance.

Contribution

It presents a novel method combining group LASSO regularization with reinforcement learning to actively generate memory-efficient policies for robotics.

Findings

Policies use low-dimensional memory representations

Improved generalization in navigation tasks

Active reduction of memory requirements

Abstract

Robots equipped with rich sensing modalities (e.g., RGB-D cameras) performing long-horizon tasks motivate the need for policies that are highly memory-efficient. State-of-the-art approaches for controlling robots often use memory representations that are excessively rich for the task or rely on hand-crafted tricks for memory efficiency. Instead, this work provides a general approach for jointly synthesizing memory representations and policies; the resulting policies actively seek to reduce memory requirements. Specifically, we present a reinforcement learning framework that leverages an implementation of the group LASSO regularization to synthesize policies that employ low-dimensional and task-centric memory representations. We demonstrate the efficacy of our approach with simulated examples including navigation in discrete and continuous spaces as well as vision-based indoor navigation set in a photo-realistic simulator. The results on these examples indicate that our method is capable of finding policies that rely only on low-dimensional memory representations, improving generalization, and actively reducing memory requirements.