How Could Polyhedral Theory Harness Deep Learning?

TL;DR

This paper explores how polyhedral theory and mixed-integer representability could provide an analytical framework for designing optimal deep learning architectures, moving beyond empirical methods.

Contribution

It proposes leveraging polyhedral theory to analytically guide neural network architecture design, offering a new theoretical perspective.

Findings

Identifies potential of polyhedral theory in neural architecture design

Suggests analytical methods as alternatives to empirical tuning

Highlights promising research directions in deep learning theory

Abstract

The holy grail of deep learning is to come up with an automatic method to design optimal architectures for different applications. In other words, how can we effectively dimension and organize neurons along the network layers based on the computational resources, input size, and amount of training data? We outline promising research directions based on polyhedral theory and mixed-integer representability that may offer an analytical approach to this question, in contrast to the empirical techniques often employed.