Identifying Ordinary Differential Equations for Data-efficient Model-based Reinforcement Learning

TL;DR

This paper introduces a physics-informed neural network approach for identifying ordinary differential equations from data, enabling more accurate model-based reinforcement learning in complex systems.

Contribution

It presents a novel neural network method that incorporates prior knowledge to identify differential equations, improving data efficiency in model-based reinforcement learning.

Findings

Successfully identified ODEs for a Duffing oscillator and a cascaded tank system.

Enhanced control performance in swinging-up an inverted pendulum.

Demonstrated data-efficient model-based RL using the identified models.

Abstract

The identification of a mathematical dynamics model is a crucial step in the designing process of a controller. However, it is often very difficult to identify the system's governing equations, especially in complex environments that combine physical laws of different disciplines. In this paper, we present a new approach that allows identifying an ordinary differential equation by means of a physics-informed machine learning algorithm. Our method introduces a special neural network that allows exploiting prior human knowledge to a certain degree and extends it autonomously, so that the resulting differential equations describe the system as accurately as possible. We validate the method on a Duffing oscillator with simulation data and, additionally, on a cascaded tank example with real-world data. Subsequently, we use the developed algorithm in a model-based reinforcement learning framework by alternately identifying and controlling a system to a target state. We test the performance by swinging-up an inverted pendulum on a cart.