Bundle Networks: Fiber Bundles, Local Trivializations, and a Generative Approach to Exploring Many-to-one Maps

TL;DR

This paper introduces Bundle Networks, a novel generative architecture inspired by fiber bundles in topology, designed to better explore and understand the fibers of many-to-one maps in machine learning.

Contribution

The paper proposes Bundle Networks, leveraging local trivializations and invertible components to improve exploration and sampling of fibers in many-to-one models.

Findings

BundleNets enable natural investigation of a network's fibers.

They outperform popular generative architectures in fiber exploration.

The approach is grounded in differential topology concepts.

Abstract

Many-to-one maps are ubiquitous in machine learning, from the image recognition model that assigns a multitude of distinct images to the concept of "cat" to the time series forecasting model which assigns a range of distinct time-series to a single scalar regression value. While the primary use of such models is naturally to associate correct output to each input, in many problems it is also useful to be able to explore, understand, and sample from a model's fibers, which are the set of input values $x$ such that $f(x) = y$, for fixed $y$ in the output space. In this paper we show that popular generative architectures are ill-suited to such tasks. Motivated by this we introduce a novel generative architecture, a Bundle Network, based on the concept of a fiber bundle from (differential) topology. BundleNets exploit the idea of a local trivialization wherein a space can be locally decomposed into a product space that cleanly encodes the many-to-one nature of the map. By enforcing this decomposition in BundleNets and by utilizing state-of-the-art invertible components, investigating a network's fibers becomes natural.