Actor-Critic Methods using Physics-Informed Neural Networks: Control of a 1D PDE Model for Fluid-Cooled Battery Packs

TL;DR

This paper introduces an actor-critic control algorithm leveraging physics-informed neural networks to solve the continuous-time Hamilton-Jacobi-Bellman equation for optimal control of a 1D PDE model of a fluid-cooled battery pack.

Contribution

It presents a novel approach combining PINNs with actor-critic methods to directly solve the HJB PDE for continuous-time optimal control of PDE systems.

Findings

Hybrid-policy method outperforms other approaches in control accuracy

PINN-based HJB solution effectively guides optimal control

Method demonstrates improved control of fluid-cooled battery PDE model

Abstract

This paper proposes an actor-critic algorithm for controlling the temperature of a battery pack using a cooling fluid. This is modeled by a coupled 1D partial differential equation (PDE) with a controlled advection term that determines the speed of the cooling fluid. The Hamilton-Jacobi-Bellman (HJB) equation is a PDE that evaluates the optimality of the value function and determines an optimal controller. We propose an algorithm that treats the value network as a Physics-Informed Neural Network (PINN) to solve for the continuous-time HJB equation rather than a discrete-time Bellman optimality equation, and we derive an optimal controller for the environment that we exploit to achieve optimal control. Our experiments show that a hybrid-policy method that updates the value network using the HJB equation and updates the policy network identically to PPO achieves the best results in the control of this PDE system.