Deep Reinforcement Learning in Applied Control: Challenges, Analysis, and Insights

TL;DR

This paper systematically evaluates deep reinforcement learning methods on real-world control benchmarks, highlighting their potential and current limitations for practical applications.

Contribution

It provides a comprehensive comparative analysis of deep RL algorithms on diverse control benchmarks, offering insights into their real-world applicability.

Findings

Deep RL shows promise but faces challenges in real-world control tasks.

Performance varies significantly across different benchmark problems.

The study identifies key limitations and areas for future improvement.

Abstract

Over the past decade, remarkable progress has been made in adopting deep neural networks to enhance the performance of conventional reinforcement learning. A notable milestone was the development of Deep Q-Networks (DQN), which achieved human-level performance across a range of Atari games, demonstrating the potential of deep learning to stabilise and scale reinforcement learning. Subsequently, extensions to continuous control algorithms paved the way for a new paradigm in control, one that has attracted broader attention than any classical control approach in recent literature. These developments also demonstrated strong potential for advancing data-driven, model-free algorithms for control and for achieving higher levels of autonomy. However, the application of these methods has remained largely confined to simulated and gaming environments, with ongoing efforts to extend them to real-world applications. Before such deployment can be realised, a solid and quantitative understanding of their performance on applied control problems is necessary. This paper conducts a comparative analysis of these approaches on four diverse benchmark problems with implementation results. This analysis offers a scrutinising and systematic evaluation to shed light on the real-world capabilities and limitations of deep reinforcement learning methods in applied control settings.