Fourier Series Expansion Based Filter Parametrization for Equivariant Convolutions

TL;DR

This paper introduces a novel Fourier series-based filter parametrization for equivariant convolutions, significantly improving filter accuracy and rotation robustness, leading to better performance in rotation-sensitive tasks like image super-resolution.

Contribution

It proposes a new Fourier-based filter parametrization method that achieves zero error for non-rotated filters and reduces quality loss during rotations, enhancing equivariant convolution design.

Findings

F-Conv achieves exact equivariance in the continuous domain.

The method outperforms previous filter parametrization approaches in image super-resolution.

It effectively preserves rotation symmetries in local image features.

Abstract

It has been shown that equivariant convolution is very helpful for many types of computer vision tasks. Recently, the 2D filter parametrization technique plays an important role when designing equivariant convolutions. However, the current filter parametrization method still has its evident drawbacks, where the most critical one lies in the accuracy problem of filter representation. Against this issue, in this paper we modify the classical Fourier series expansion for 2D filters, and propose a new set of atomic basis functions for filter parametrization. The proposed filter parametrization method not only finely represents 2D filters with zero error when the filter is not rotated, but also substantially alleviates the fence-effect-caused quality degradation when the filter is rotated. Accordingly, we construct a new equivariant convolution method based on the proposed filter parametrization method, named F-Conv. We prove that the equivariance of the proposed F-Conv is exact in the continuous domain, which becomes approximate only after discretization. Extensive experiments show the superiority of the proposed method. Particularly, we adopt rotation equivariant convolution methods to image super-resolution task, and F-Conv evidently outperforms previous filter parametrization based method in this task, reflecting its intrinsic capability of faithfully preserving rotation symmetries in local image features.