Metric Convolutions: A Unifying Theory to Adaptive Image Convolutions

TL;DR

This paper introduces metric convolutions, a unifying theoretical framework for adaptive image convolutions based on local and geodesic distances, enabling more interpretable and efficient operators in deep learning.

Contribution

It proposes a novel metric-based convolution framework that unifies existing deformation strategies and improves adaptability and interpretability in image processing.

Findings

Requires fewer parameters than standard convolutions

Provides better generalisation in denoising and classification tasks

Offers a unifying geometric perspective for adaptive convolutions

Abstract

Standard convolutions are prevalent in image processing and deep learning, but their fixed kernels limits adaptability. Several deformation strategies of the reference kernel grid have been proposed. Yet, they lack a unified theoretical framework. By returning to a metric perspective for images, now seen as two-dimensional manifolds equipped with notions of local and geodesic distances, either symmetric (Riemannian) or not (Finsler), we provide a unifying principle: the kernel positions are samples of unit balls of implicit metrics. With this new perspective, we also propose metric convolutions, a novel approach that samples unit balls from explicit signal-dependent metrics, providing interpretable operators with geometric regularisation. This framework, compatible with gradient-based optimisation, can directly replace existing convolutions applied to either input images or deep features of neural networks. Metric convolutions typically require fewer parameters and provide better generalisation. Our approach shows competitive performance in standard denoising and classification tasks.