Statistical efficiency of curve fitting algorithms

TL;DR

This paper analyzes the statistical properties of curve fitting algorithms, deriving asymptotic bias and covariance, extending bounds, and identifying efficient algebraic fits under certain models.

Contribution

It provides new asymptotic expressions for bias and covariance, extends the Cramer-Rao bound to more algorithms, and identifies the gradient-weighted algebraic fit as statistically efficient.

Findings

Gradient-weighted algebraic fit is statistically efficient.

Extended Cramer-Rao bound applies to popular algorithms.

Derived asymptotic bias and covariance expressions.

Abstract

We study the problem of fitting parametrized curves to noisy data. Under certain assumptions (known as Cartesian and radial functional models), we derive asymptotic expressions for the bias and the covariance matrix of the parameter estimates. We also extend Kanatani's version of the Cramer-Rao lower bound, which he proved for unbiased estimates only, to more general estimates that include many popular algorithms (most notably, the orthogonal least squares and algebraic fits). We then show that the gradient-weighted algebraic fit is statistically efficient and describe all other statistically efficient algebraic fits.