Connections between Operator-splitting Methods and Deep Neural Networks with Applications in Image Segmentation

TL;DR

This paper reveals a mathematical connection between operator-splitting algorithms and deep neural networks, proposing new network architectures inspired by operator-splitting methods for image segmentation, and demonstrating their effectiveness through experiments.

Contribution

It establishes a novel link between operator-splitting methods and neural network structures, and introduces two new networks based on this connection for image segmentation.

Findings

Networks inspired by operator-splitting methods perform effectively in image segmentation tasks.

The proposed networks are structurally equivalent to specific operator-splitting algorithms.

Numerical experiments validate the effectiveness of the new network architectures.

Abstract

Deep neural network is a powerful tool for many tasks. Understanding why it is so successful and providing a mathematical explanation is an important problem and has been one popular research direction in past years. In the literature of mathematical analysis of deep neural networks, a lot of works is dedicated to establishing representation theories. How to make connections between deep neural networks and mathematical algorithms is still under development. In this paper, we give an algorithmic explanation for deep neural networks, especially in their connections with operator splitting. We show that with certain splitting strategies, operator-splitting methods have the same structure as networks. Utilizing this connection and the Potts model for image segmentation, two networks inspired by operator-splitting methods are proposed. The two networks are essentially two operator-splitting algorithms solving the Potts model. Numerical experiments are presented to demonstrate the effectiveness of the proposed networks.