Soft Actor-Critic for Discrete Action Settings

TL;DR

This paper introduces a discrete-action version of the Soft Actor-Critic algorithm, making it applicable to important discrete settings and demonstrating competitive performance on Atari games without hyperparameter tuning.

Contribution

The paper develops a novel discrete-action Soft Actor-Critic algorithm and shows its competitive performance on Atari benchmarks without hyperparameter tuning.

Findings

Competitive performance on Atari games

Effective without hyperparameter tuning

Extends Soft Actor-Critic to discrete actions

Abstract

Soft Actor-Critic is a state-of-the-art reinforcement learning algorithm for continuous action settings that is not applicable to discrete action settings. Many important settings involve discrete actions, however, and so here we derive an alternative version of the Soft Actor-Critic algorithm that is applicable to discrete action settings. We then show that, even without any hyperparameter tuning, it is competitive with the tuned model-free state-of-the-art on a selection of games from the Atari suite.