Three Properties of F-Statistics for Multiple Regression and ANOVA

TL;DR

This paper explores fundamental properties of F-statistics in multiple regression and ANOVA, clarifying how different sums of squares tests relate in degrees of freedom and non-centrality, with implications for model testing.

Contribution

It establishes three key properties of F-statistics, including the relationship between sums of squares, degrees of freedom, and non-centrality parameters in regression and ANOVA.

Findings

Extra SSE tests the estimable part of linear conditions.

All alternative sums of squares have not-lesser degrees of freedom.

Omitting columns in the model matrix corresponds to eliminating effects in ANOVA.

Abstract

This paper establishes three properties of F-statistics for inference about the mean vector in multiple regression and analysis of variance. The extra SSE due to imposing a set of linear conditions on the model tests the estimable part of those conditions. All other possible numerator sums of squares that test the same have not-lesser degrees of freedom and not-greater non-centrality parameters. When factor-level combinations are coded by contrasts, the model restricted to eliminate an ANOVA effect is formulated by omitting that effect's columns from the model matrix.