CUF: Continuous Upsampling Filters

TL;DR

This paper introduces CUF, a neural field-based approach to image upsampling that significantly reduces parameters and improves efficiency over existing super-resolution methods without sacrificing quality.

Contribution

We propose a novel neural field parameterization for learnable upsampling kernels, achieving substantial parameter reduction and efficiency gains in image super-resolution.

Findings

40-fold reduction in parameters compared to existing methods

2x-10x efficiency improvement in arbitrary-scale super-resolution

Maintains super-resolution performance on standard benchmarks

Abstract

Neural fields have rapidly been adopted for representing 3D signals, but their application to more classical 2D image-processing has been relatively limited. In this paper, we consider one of the most important operations in image processing: upsampling. In deep learning, learnable upsampling layers have extensively been used for single image super-resolution. We propose to parameterize upsampling kernels as neural fields. This parameterization leads to a compact architecture that obtains a 40-fold reduction in the number of parameters when compared with competing arbitrary-scale super-resolution architectures. When upsampling images of size 256x256 we show that our architecture is 2x-10x more efficient than competing arbitrary-scale super-resolution architectures, and more efficient than sub-pixel convolutions when instantiated to a single-scale model. In the general setting, these gains grow polynomially with the square of the target scale. We validate our method on standard benchmarks showing such efficiency gains can be achieved without sacrifices in super-resolution performance.