Tackling Visual Control via Multi-View Exploration Maximization

TL;DR

MEM introduces a novel multi-view reinforcement learning approach that combines representation learning and intrinsic reward-driven exploration, significantly improving sample efficiency and generalization in complex visual control tasks.

Contribution

It is the first method to integrate multi-view representation learning with entropy-based exploration rewards in RL.

Findings

MEM outperforms existing methods on DeepMind Control Suite tasks.

MEM demonstrates higher sample efficiency and better generalization.

The approach is effective in high-dimensional, sparse-reward environments.

Abstract

We present MEM: Multi-view Exploration Maximization for tackling complex visual control tasks. To the best of our knowledge, MEM is the first approach that combines multi-view representation learning and intrinsic reward-driven exploration in reinforcement learning (RL). More specifically, MEM first extracts the specific and shared information of multi-view observations to form high-quality features before performing RL on the learned features, enabling the agent to fully comprehend the environment and yield better actions. Furthermore, MEM transforms the multi-view features into intrinsic rewards based on entropy maximization to encourage exploration. As a result, MEM can significantly promote the sample-efficiency and generalization ability of the RL agent, facilitating solving real-world problems with high-dimensional observations and spare-reward space. We evaluate MEM on various tasks from DeepMind Control Suite and Procgen games. Extensive simulation results demonstrate that MEM can achieve superior performance and outperform the benchmarking schemes with simple architecture and higher efficiency.