How to Learn a Star: Binary Classification with Starshaped Polyhedral Sets

TL;DR

This paper explores a class of binary classifiers with starshaped polyhedral decision boundaries, analyzing their expressivity, loss landscape, and geometric properties, providing bounds on VC dimension and conditions for optimality.

Contribution

It introduces a novel class of piecewise linear classifiers with starshaped polyhedral boundaries, analyzing their complexity and geometric structure in detail.

Findings

Explicit bounds on VC dimension of the model

Description of sublevel sets as chambers in a hyperplane arrangement

Conditions for the uniqueness of the log-likelihood optimum

Abstract

We consider binary classification restricted to a class of continuous piecewise linear functions whose decision boundaries are (possibly nonconvex) starshaped polyhedral sets, supported on a fixed polyhedral simplicial fan. We investigate the expressivity of these function classes and describe the combinatorial and geometric structure of the loss landscape, most prominently the sublevel sets, for two loss-functions: the 0/1-loss (discrete loss) and a log-likelihood loss function. In particular, we give explicit bounds on the VC dimension of this model, and concretely describe the sublevel sets of the discrete loss as chambers in a hyperplane arrangement. For the log-likelihood loss, we give sufficient conditions for the optimum to be unique, and describe the geometry of the optimum when varying the rate parameter of the underlying exponential probability distribution.