Neural Network Training Using Closed-Loop Data: Hazards and an Instrumental Variable (IVNN) Solution

TL;DR

This paper identifies the problem of parameter inconsistency in neural network control systems trained with closed-loop data and proposes an instrumental variable method (IVNN) to achieve consistent estimates and optimal control performance.

Contribution

It introduces the IVNN approach that uses instrumental variables to ensure consistent neural network parameter estimation in closed-loop control systems.

Findings

IVNN asymptotically recovers the optimal solution

Pre-existing methods lead to inconsistent estimates

Demonstrated effectiveness on a nonlinear system example

Abstract

An increasing trend in the use of neural networks in control systems is being observed. The aim of this paper is to reveal that the straightforward application of learning neural network feedforward controllers with closed-loop data may introduce parameter inconsistency that degrades control performance, and to provide a solution. The proposed method employs instrumental variables to ensure consistent parameter estimates. A nonlinear system example reveals that the developed instrumental variable neural network (IVNN) approach asymptotically recovers the optimal solution, while pre-existing approaches are shown to lead to inconsistent estimates.