Combining Deep Reinforcement Learning And Local Control For The Acrobot Swing-up And Balance Task

TL;DR

This paper introduces a method that combines traditional control techniques with deep reinforcement learning to improve the acrobot swing-up and balance task, leveraging domain knowledge and learning capabilities.

Contribution

The authors extend the soft actor critic algorithm to integrate classical controllers with neural network policies for enhanced performance.

Findings

Outperforms existing reinforcement learning algorithms on the acrobot task

Effectively combines domain knowledge with learned policies

Demonstrates improved stability and efficiency in control

Abstract

In this work we present a novel extension of soft actor critic, a state of the art deep reinforcement algorithm. Our method allows us to combine traditional controllers with learned neural network policies. This combination allows us to leverage both our own domain knowledge and some of the advantages of model free reinforcement learning. We demonstrate our algorithm by combining a hand designed linear quadratic regulator with a learned controller for the acrobot problem. We show that our technique outperforms other state of the art reinforcement learning algorithms in this setting.