Maximum entropy properties of discrete-time first-order stable spline kernel

TL;DR

This paper explores the maximum entropy properties of the stable spline (SS-1) kernel used in system identification, revealing its optimality under general sampling schemes and its structural characteristics.

Contribution

It formulates the maximum entropy problem solved by the SS-1 kernel without Gaussian or uniform sampling assumptions, and characterizes its structural properties and covariance completion.

Findings

SS-1 kernel solves a maximum entropy problem under general sampling.

The inverse of the SS-1 kernel is tridiagonal, indicating a specific structure.

Similar maximum entropy properties are identified for the Wiener kernel.

Abstract

The first order stable spline (SS-1) kernel is used extensively in regularized system identification. In particular, the stable spline estimator models the impulse response as a zero-mean Gaussian process whose covariance is given by the SS-1 kernel. In this paper, we discuss the maximum entropy properties of this prior. In particular, we formulate the exact maximum entropy problem solved by the SS-1 kernel without Gaussian and uniform sampling assumptions. Under general sampling schemes, we also explicitly derive the special structure underlying the SS-1 kernel (e.g. characterizing the tridiagonal nature of its inverse), also giving to it a maximum entropy covariance completion interpretation. Along the way similar maximum entropy properties of the Wiener kernel are also given.