CantorNet: A Sandbox for Testing Geometrical and Topological Complexity Measures

TL;DR

CantorNet is a neural network framework inspired by the Cantor set, designed to analyze and measure the geometrical and topological complexity of decision boundaries, bridging theoretical complexity and practical robustness insights.

Contribution

Introduces CantorNet, a family of ReLU neural networks that span the entire spectrum of Kolmogorov complexities, serving as a sandbox for studying self-similarity and complexity measures.

Findings

CantorNet decision boundaries can be arbitrarily ragged and are analytically known.

The framework spans from simple to highly complex Kolmogorov descriptions.

Potential to reveal pitfalls in geometry-ignorant data augmentation and adversarial robustness.

Abstract

Many natural phenomena are characterized by self-similarity, for example the symmetry of human faces, or a repetitive motif of a song. Studying of such symmetries will allow us to gain deeper insights into the underlying mechanisms of complex systems. Recognizing the importance of understanding these patterns, we propose a geometrically inspired framework to study such phenomena in artificial neural networks. To this end, we introduce \emph{CantorNet}, inspired by the triadic construction of the Cantor set, which was introduced by Georg Cantor in the $19^\text{th}$ century. In mathematics, the Cantor set is a set of points lying on a single line that is self-similar and has a counter intuitive property of being an uncountably infinite null set. Similarly, we introduce CantorNet as a sandbox for studying self-similarity by means of novel topological and geometrical complexity measures. CantorNet constitutes a family of ReLU neural networks that spans the whole spectrum of possible Kolmogorov complexities, including the two opposite descriptions (linear and exponential as measured by the description length). CantorNet's decision boundaries can be arbitrarily ragged, yet are analytically known. Besides serving as a testing ground for complexity measures, our work may serve to illustrate potential pitfalls in geometry-ignorant data augmentation techniques and adversarial attacks.