ButterflyNet2D: Bridging Classical Methods and Neural Network Methods in Image Processing

TL;DR

ButterflyNet2D is a neural network architecture that combines classical Fourier methods with deep learning, achieving interpretable and high-performance image processing by leveraging Fourier initialization.

Contribution

The paper introduces ButterflyNet2D, a CNN with sparse connections and Fourier initialization, bridging classical Fourier methods and neural networks for improved image processing.

Findings

ButterflyNet2D accurately approximates Fourier transforms.

Fourier-initialized ButterflyNet2D outperforms randomly initialized networks.

The method enhances interpretability and performance in image tasks.

Abstract

Both classical Fourier transform-based methods and neural network methods are widely used in image processing tasks. The former has better interpretability, whereas the latter often achieves better performance in practice. This paper introduces ButterflyNet2D, a regular CNN with sparse cross-channel connections. A Fourier initialization strategy for ButterflyNet2D is proposed to approximate Fourier transforms. Numerical experiments validate the accuracy of ButterflyNet2D approximating both the Fourier and the inverse Fourier transforms. Moreover, through four image processing tasks and image datasets, we show that training ButterflyNet2D from Fourier initialization does achieve better performance than random initialized neural networks.