Hadamax Encoding: Elevating Performance in Model-Free Atari

TL;DR

The paper introduces Hadamax, a novel encoder architecture using Hadamard max-pooling that significantly improves performance in model-free Atari reinforcement learning benchmarks.

Contribution

It presents a new encoder architecture, Hadamax, which enhances model-free RL performance without hyperparameter tuning, achieving state-of-the-art results.

Findings

Achieves 80% performance gain over vanilla PQN.

Surpasses Rainbow-DQN in Atari-57 benchmark.

State-of-the-art results with no hyperparameter modifications.

Abstract

Neural network architectures have a large impact in machine learning. In reinforcement learning, network architectures have remained notably simple, as changes often lead to small gains in performance. This work introduces a novel encoder architecture for pixel-based model-free reinforcement learning. The Hadamax (\textbf{Hada}mard \textbf{max}-pooling) encoder achieves state-of-the-art performance by max-pooling Hadamard products between GELU-activated parallel hidden layers. Based on the recent PQN algorithm, the Hadamax encoder achieves state-of-the-art model-free performance in the Atari-57 benchmark. Specifically, without applying any algorithmic hyperparameter modifications, Hadamax-PQN achieves an 80\% performance gain over vanilla PQN and significantly surpasses Rainbow-DQN. For reproducibility, the full code is available on \href{https://github.com/Jacobkooi/Hadamax}{GitHub}.