Piecewise linear functions and neural network expressivity via discriminantal arrangements

TL;DR

This paper extends the mathematical framework for understanding neural network expressivity by using discriminantal arrangements, characterizing piecewise linear functions through circuit relations and matroid theory.

Contribution

It introduces a novel extension from braid to discriminantal arrangements, providing a matroidal description of compatible piecewise linear functions.

Findings

Functions determined by values on small subsets for circuits of size three

Dimension equals the number of independent sets in the matroid

Characterization of functions via circuit relations and Mobius inversion

Abstract

We extend the hyperplane arrangement framework for neural network expressivity from the braid to discriminantal arrangements. Compatible piecewise linear functions are characterized by circuit relations and admit a matroidal description via Mobius inversion, with dimension equal to the number of independent sets. For circuits of size three, functions are determined by values on subsets of size at most two.