Hybrid-Quantum Neural Architecture Search for The Proximal Policy Optimization Algorithm

TL;DR

This paper introduces a hybrid-quantum neural architecture search method for the PPO reinforcement learning algorithm, aiming to identify effective hybrid models and interpret their design choices amidst quantum hardware limitations.

Contribution

It applies a Regularized Evolution algorithm to optimize hybrid classical-quantum architectures for PPO and provides insights into factors influencing model performance.

Findings

Classical models outperformed hybrid models on the leaderboard.

The best hybrid model ranked eleventh among all models.

Insights into design factors that affect hybrid architecture performance.

Abstract

Recent studies in quantum machine learning advocated the use of hybrid models to assist with the limitations of the currently existing Noisy Intermediate Scale Quantum (NISQ) devices, but what was missing from most of them was the explanations and interpretations of the choices that were made to pick those exact architectures and the differentiation between good and bad hybrid architectures, this research attempts to tackle that gap in the literature by using the Regularized Evolution algorithm to search for the optimal hybrid classical-quantum architecture for the Proximal Policy Optimization (PPO) algorithm, a well-known reinforcement learning algorithm, ultimately the classical models dominated the leaderboard with the best hybrid model coming in eleventh place among all unique models, while we also try to explain the factors that contributed to such results,and for some models to behave better than others in hope to grasp a better intuition about what we should consider good practices for designing an efficient hybrid architecture.