Arrow of Time in Estimation and Control: Duality Theory Beyond the Linear Gaussian Model

TL;DR

This paper explores the limitations of duality between estimation and control in nonlinear stochastic systems like hidden Markov models, and discusses recent advances to extend duality beyond linear Gaussian models.

Contribution

It identifies the challenges in extending duality to HMMs and presents recent solutions that address these difficulties, especially related to time reversal issues.

Findings

Duality is incomplete for nonlinear stochastic systems.

Time reversal is the main obstacle in extending duality.

Recent work provides a resolution for duality in HMMs.

Abstract

Duality between estimation and control is a foundational concept in Control Theory. Most students learn about the elementary duality -- between observability and controllability -- in their first graduate course in linear systems theory. Therefore, it comes as a surprise that for a more general class of nonlinear stochastic systems (hidden Markov models or HMMs), duality is incomplete. Our objective in writing this article is two-fold: (i) To describe the difficulty in extending duality to HMMs; and (ii) To discuss its recent resolution by the authors. A key message is that the main difficulty in extending duality comes from time reversal in going from estimation to control. The reason for time reversal is explained with the aid of the familiar linear deterministic and linear Gaussian models. The explanation is used to motivate the difference between the linear and the nonlinear models. Once the difference is understood, duality for HMMs is described based on our recent work. The article also includes a comparison and discussion of the different types of duality considered in literature.