Comparison of Model Predictive Control and Proximal Policy Optimization for a 1-DOF Helicopter System

TL;DR

This paper compares Model Predictive Control and Proximal Policy Optimization, a Deep Reinforcement Learning method, on a 1-DOF helicopter system, analyzing their performance, computational demands, and suitability for different control tasks.

Contribution

It provides a systematic comparison of MPC and PPO on a 1-DOF helicopter, highlighting their respective strengths, limitations, and practical considerations for control applications.

Findings

PPO shows superior rise-time and adaptability.

LQR achieves the best steady-state accuracy.

PPO offers promising rapid response capabilities.

Abstract

This study conducts a comparative analysis of Model Predictive Control (MPC) and Proximal Policy Optimization (PPO), a Deep Reinforcement Learning (DRL) algorithm, applied to a 1-Degree of Freedom (DOF) Quanser Aero 2 system. Classical control techniques such as MPC and Linear Quadratic Regulator (LQR) are widely used due to their theoretical foundation and practical effectiveness. However, with advancements in computational techniques and machine learning, DRL approaches like PPO have gained traction in solving optimal control problems through environment interaction. This paper systematically evaluates the dynamic response characteristics of PPO and MPC, comparing their performance, computational resource consumption, and implementation complexity. Experimental results show that while LQR achieves the best steady-state accuracy, PPO excels in rise-time and adaptability, making it a promising approach for applications requiring rapid response and adaptability. Additionally, we have established a baseline for future RL-related research on this specific testbed. We also discuss the strengths and limitations of each control strategy, providing recommendations for selecting appropriate controllers for real-world scenarios.