Neural Space-filling Curves

TL;DR

This paper introduces Neural Space-filling Curves, a neural network-based method to learn optimized pixel traversal orders for images, improving applications like image compression over traditional fixed scan methods.

Contribution

It proposes a novel neural network approach to learn data-driven, task-optimized space-filling curves for images, replacing fixed scanning algorithms.

Findings

Neural SFCs outperform traditional scan methods in image compression.

The learned order improves downstream task performance.

Code and results are publicly available.

Abstract

We present Neural Space-filling Curves (SFCs), a data-driven approach to infer a context-based scan order for a set of images. Linear ordering of pixels forms the basis for many applications such as video scrambling, compression, and auto-regressive models that are used in generative modeling for images. Existing algorithms resort to a fixed scanning algorithm such as Raster scan or Hilbert scan. Instead, our work learns a spatially coherent linear ordering of pixels from the dataset of images using a graph-based neural network. The resulting Neural SFC is optimized for an objective suitable for the downstream task when the image is traversed along with the scan line order. We show the advantage of using Neural SFCs in downstream applications such as image compression. Code and additional results will be made available at https://hywang66.github.io/publication/neuralsfc.