Information criteria for structured parameter selection in high dimensional tree and graph models

TL;DR

This paper develops refined information criteria, specifically corrected Mallows's Cp, for structured parameter selection in high-dimensional tree and graph models, aiming to better balance false positives and negatives without shrinkage.

Contribution

It introduces corrected Mallows's Cp criteria tailored for structured high-dimensional models, improving parameter selection accuracy in trees and graphical models.

Findings

Corrected criteria reduce false positive overestimation.

Enhanced selection accuracy in high-dimensional models.

Applicable to non-shrinkage estimators in structured models.

Abstract

Parameter selection in high-dimensional models is typically finetuned in a way that keeps the (relative) number of false positives under control. This is because otherwise the few true positives may be dominated by the many possible false positives. This happens, for instance, when the selection follows from a naive optimisation of an information criterion, such as AIC or Mallows's Cp. It can be argued that the overestimation of the selection comes from the optimisation process itself changing the statistics of the selected variables, in a way that the information criterion no longer reflects the true divergence between the selection and the data generating process. In lasso, the overestimation can also be linked to the shrinkage estimator, which makes the selection too tolerant of false positive selections. For these reasons, this paper works on refined information criteria, carefully balancing false positives and false negatives, for use with estimators without shrinkage. In particular, the paper develops corrected Mallows's Cp criteria for structured selection in trees and graphical models.