Model selection and local geometry

TL;DR

This paper investigates the challenges in model selection caused by the local geometric structure of models, especially when models share tangent spaces, and proposes methods focusing on tangent space learning rather than the models themselves.

Contribution

It reveals the geometric limitations affecting model distinguishability and convexity, and introduces a tangent space-based approach for model selection, including a generic algorithm for Bayesian networks.

Findings

Models with identical tangent spaces require larger sample sizes to distinguish.

Certain models cannot be made convex through reparameterization.

Proposed tangent space learning methods improve model selection accuracy.

Abstract

We consider problems in model selection caused by the geometry of models close to their points of intersection. In some cases---including common classes of causal or graphical models, as well as time series models---distinct models may nevertheless have identical tangent spaces. This has two immediate consequences: first, in order to obtain constant power to reject one model in favour of another we need local alternative hypotheses that decrease to the null at a slower rate than the usual parametric $n^{-1/2}$ (typically we will require $n^{-1/4}$ or slower); in other words, to distinguish between the models we need large effect sizes or very large sample sizes. Second, we show that under even weaker conditions on their tangent cones, models in these classes cannot be made simultaneously convex by a reparameterization. This shows that Bayesian network models, amongst others, cannot be learned directly with a convex method similar to the graphical lasso. However, we are able to use our results to suggest methods for model selection that learn the tangent space directly, rather than the model itself. In particular, we give a generic algorithm for learning Bayesian network models.