Sign-Perturbed Sums: A New System Identification Approach for Constructing Exact Non-Asymptotic Confidence Regions in Linear Regression Models

TL;DR

This paper introduces Sign-Perturbed Sums (SPS), a novel system identification method that constructs exact non-asymptotic confidence regions for linear regression models with minimal assumptions, applicable to finite data sets.

Contribution

The paper presents SPS, a new approach for exact confidence regions in linear regression, with a star convex shape and an efficient approximation algorithm, applicable under mild noise assumptions.

Findings

SPS provides exact confidence probabilities for finite data sets.

SPS regions are star convex with the LS estimate as center.

Simulation experiments validate the effectiveness of SPS.

Abstract

We propose a new system identification method, called Sign-Perturbed Sums (SPS), for constructing non-asymptotic confidence regions under mild statistical assumptions. SPS is introduced for linear regression models, including but not limited to FIR systems, and we show that the SPS confidence regions have exact confidence probabilities, i.e., they contain the true parameter with a user-chosen exact probability for any finite data set. Moreover, we also prove that the SPS regions are star convex with the Least-Squares (LS) estimate as a star center. The main assumptions of SPS are that the noise terms are independent and symmetrically distributed about zero, but they can be nonstationary, and their distributions need not be known. The paper also proposes a computationally efficient ellipsoidal outer approximation algorithm for SPS. Finally, SPS is demonstrated through a number of simulation experiments.