Variance of the root mean square value of the residuals of sine fitting in the presence of additive noise

TL;DR

This paper analyzes how additive noise affects the variance of the RMS value in sinusoidal fitting, providing exact and approximate formulas validated through simulations.

Contribution

The paper derives an exact analytical expression and proposes simpler approximations for the variance of RMS residuals under additive noise.

Findings

An exact analytical expression for the variance of RMS residuals in the presence of additive noise is derived.

Two simplified approximations for the variance are proposed and validated through numerical simulations.

The statistical properties of RMS values are shown to be affected by additive noise phenomena.

Abstract

The least-squares fitting of a sinusoidal model to a set of data points is a common procedure in signal processing algorithms. A residual is the difference between the value of one data points and the estimated value of that point given by the sinusoidal model. The root mean square (RMS) value of all the residuals is a common metric used in many applications to quantify the goodness of fit. In analog-to-digital conversion, for example, the RMS value is used to compute the number of effective bits. In other applications the RMS value is used to compute the signal-to-noise ratio which measures the amount of noise generated by an electronic circuit such as an amplifier, for instance. Due to the presence of different random non-ideal phenomena affecting the data points, like stimulus signal phase noise, sampling jitter or quantization error, the estimative of the RMS value is uncertain and whose statistical properties are important to evaluate. In this work we focus on the effect that additive noise has on the variance of the RMS value of the residuals. A first exact analytical expression is derived and two easier to use and simpler approximations are proposed. The results presented are validated using numerical simulations employing a Monte Carlo type procedure.

Figures