Data boundary fitting using a generalised least-squares method

TL;DR

This paper introduces a flexible, automatic method for determining data boundaries in astronomical datasets using a generalized least-squares approach with adaptive splines and error weighting, improving boundary estimation accuracy.

Contribution

It presents a novel boundary fitting technique employing a generalized least-squares method with adaptive splines and error weighting, applicable to complex data behaviors.

Findings

Effective boundary estimation with simple polynomials for common cases.

Adaptive splines provide excellent results for complex data.

Normalization reduces errors and computational effort.

Abstract

In many astronomical problems one often needs to determine the upper and/or lower boundary of a given data set. An automatic and objective approach consists in fitting the data using a generalised least-squares method, where the function to be minimized is defined to handle asymmetrically the data at both sides of the boundary. In order to minimise the cost function, a numerical approach, based on the popular downhill simplex method, is employed. The procedure is valid for any numerically computable function. Simple polynomials provide good boundaries in common situations. For data exhibiting a complex behaviour, the use of adaptive splines gives excellent results. Since the described method is sensitive to extreme data points, the simultaneous introduction of error weighting and the flexibility of allowing some points to fall outside of the fitted frontier, supplies the parameters that help to tune the boundary fitting depending on the nature of the considered problem. Two simple examples are presented, namely the estimation of spectra pseudo-continuum and the segregation of scattered data into ranges. The normalisation of the data ranges prior to the fitting computation typically reduces both the numerical errors and the number of iterations required during the iterative minimisation procedure.