Range Space or Null Space: Least-Squares Methods for the Realization Problem

TL;DR

This paper compares range-space and null-space least-squares methods for the classical realization problem, analyzing their sensitivities, conditions for preference, and proposing an optimal weighted least-squares approach.

Contribution

It clarifies the relationship between range-space and null-space realization algorithms and introduces an optimal weighted least-squares method for improved accuracy.

Findings

Range-space method is equivalent to total least-squares.

Null-space method corresponds to ordinary least-squares.

Optimal realization is achieved via a weighted least-squares approach.

Abstract

This contribution revisits the classical approximate realization problem, which involves determining matrices of a state-space model based on estimates of a truncated series of Markov parameters. A Hankel matrix built up by these Markov parameters plays a fundamental role in this problem, leveraging the fact that both its range space and left null space encode critical information about the state-space model. We examine two prototype realization algorithms based on the Hankel matrix: the classical range-space-based (SVD-based) method and the more recent null-space-based method. It is demonstrated that the range-space-based method corresponds to a total least-squares solution, whereas the null-space-based method corresponds to an ordinary least-squares solution. By analyzing the differences in sensitivity of the two algorithms, we determine the conditions when one or the other realization algorithm is to be preferred, and identify factors that contribute to an ill-conditioned realization problem. Furthermore, recognizing that both methods are suboptimal, we argue that the optimal realization is obtained through a weighted least-squares approach. A statistical analysis of these methods, including their consistency and asymptotic normality is also provided.