Fixed Integral Neural Networks

TL;DR

This paper introduces Fixed-Integral Neural Networks (FINN), enabling exact analytical integration of neural network functions, which facilitates applications requiring constrained and positive functions, such as probability distributions.

Contribution

The paper presents a novel method for representing the analytical integral of neural networks, allowing exact integration and direct constraint application on the integral and the function.

Findings

Exact integral computation for neural networks achieved.

Method to constrain neural networks to be positive.

Applications demonstrated in probabilistic modeling.

Abstract

It is often useful to perform integration over learned functions represented by neural networks. However, this integration is usually performed numerically, as analytical integration over learned functions (especially neural networks) is generally viewed as intractable. In this work, we present a method for representing the analytical integral of a learned function $f$. This allows the exact integral of a neural network to be computed, and enables constrained neural networks to be parametrised by applying constraints directly to the integral. Crucially, we also introduce a method to constrain $f$ to be positive, a necessary condition for many applications (e.g. probability distributions, distance metrics, etc). Finally, we introduce several applications where our fixed-integral neural network (FINN) can be utilised.