Bivariate least squares linear regression: towards a unified analytic formalism. II. Extreme structural models

TL;DR

This paper reviews and extends bivariate least squares linear regression models using a new formalism, providing more compact notation and applying it to astronomical data, highlighting the effects of data dispersion and systematic errors.

Contribution

It introduces a unified formalism for extreme structural models in bivariate regression, improving variance estimations and applying the approach to astronomical abundance data.

Findings

Results agree within errors for low-dispersion samples across methods.

Discrepancies increase with large dispersion and different methods.

Asymptotic variance estimators are more accurate, with discrepancies under 10% for large dispersion.

Abstract

Concerning bivariate least squares linear regression, the classical results obtained for extreme structural models in earlier attempts are reviewed using a new formalism in terms of deviation (matrix) traces which, for homoscedastic data, reduce to usual quantities leaving aside an unessential (but dimensional) multiplicative factor. Within the framework of classical error models, the dependent variable relates to the independent variable according to the usual additive model. The classes of linear models considered are regression lines in the limit of uncorrelated errors in X and in Y. For homoscedastic data, the results are taken from earlier attempts and rewritten using a more compact notation. For heteroscedastic data, the results are inferred from a procedure related to functional models. An example of astronomical application is considered, concerning the [O/H]-[Fe/H] empirical relations deduced from five samples related to different stars and/or different methods of oxygen abundance determination. For low-dispersion samples and assigned methods, different regression models yield results which are in agreement within the errors for both heteroscedastic and homoscedastic data, while the contrary holds for large-dispersion samples. In any case, samples related to different methods produce discrepant results, due to the presence of (still undetected) systematic errors, which implies no definitive statement can be made at present. Asymptotic expressions approximate regression line slope and intercept variance estimators, for normal residuals, to a better extent with respect to earlier attempts. Related fractional discrepancies are not exceeding a few percent for low-dispersion data, which grows up to about 10% for large-dispersion data. An extension of the formalism to generic structural models is left to a forthcoming paper.