Physics-Informed Model and Hybrid Planning for Efficient Dyna-Style Reinforcement Learning

TL;DR

This paper introduces a physics-informed model combined with hybrid planning to improve sample efficiency, inference time, and performance in Dyna-style reinforcement learning by leveraging partial physical knowledge.

Contribution

It proposes a novel approach that integrates physics-informed modeling with hybrid planning to address key challenges in real-world reinforcement learning applications.

Findings

Improves sample efficiency over state-of-the-art methods

Enhances planning time efficiency with hybrid strategies

Achieves better performance-performance trade-offs in practical tasks

Abstract

Applying reinforcement learning (RL) to real-world applications requires addressing a trade-off between asymptotic performance, sample efficiency, and inference time. In this work, we demonstrate how to address this triple challenge by leveraging partial physical knowledge about the system dynamics. Our approach involves learning a physics-informed model to boost sample efficiency and generating imaginary trajectories from this model to learn a model-free policy and Q-function. Furthermore, we propose a hybrid planning strategy, combining the learned policy and Q-function with the learned model to enhance time efficiency in planning. Through practical demonstrations, we illustrate that our method improves the compromise between sample efficiency, time efficiency, and performance over state-of-the-art methods.