A Quadratic Actor Network for Model-Free Reinforcement Learning

TL;DR

This paper introduces quadratic neurons into policy networks for model-free reinforcement learning, demonstrating improved performance and efficiency in continuous control tasks.

Contribution

It is the first to incorporate quadratic neurons into actor-critic networks, showing enhanced performance and parameter efficiency over traditional MLP policies.

Findings

Quadratic neurons outperform baseline MLP policies in MuJoCo tasks.

Added quadratic neurons increase sample efficiency by 21%.

Quadratic networks maintain robustness against noise.

Abstract

In this work we discuss the incorporation of quadratic neurons into policy networks in the context of model-free actor-critic reinforcement learning. Quadratic neurons admit an explicit quadratic function approximation in contrast to conventional approaches where the the non-linearity is induced by the activation functions. We perform empiric experiments on several MuJoCo continuous control tasks and find that when quadratic neurons are added to MLP policy networks those outperform the baseline MLP whilst admitting a smaller number of parameters. The top returned reward is in average increased by $5.8\%$ while being about $21\%$ more sample efficient. Moreover, it can maintain its advantage against added action and observation noise.