Learning Continuous Image Representation with Local Implicit Image Function

TL;DR

This paper introduces LIIF, a neural implicit representation for images that enables continuous, high-resolution image reconstruction and super-resolution by predicting pixel colors from local features at arbitrary coordinates.

Contribution

The paper proposes LIIF, a novel neural implicit model that learns continuous image representations, allowing arbitrary resolution rendering and improved super-resolution performance.

Findings

LIIF can generate images at resolutions up to 30 times higher than training resolution.

It outperforms traditional resizing methods in super-resolution tasks.

The approach bridges discrete and continuous image representations effectively.

Abstract

How to represent an image? While the visual world is presented in a continuous manner, machines store and see the images in a discrete way with 2D arrays of pixels. In this paper, we seek to learn a continuous representation for images. Inspired by the recent progress in 3D reconstruction with implicit neural representation, we propose Local Implicit Image Function (LIIF), which takes an image coordinate and the 2D deep features around the coordinate as inputs, predicts the RGB value at a given coordinate as an output. Since the coordinates are continuous, LIIF can be presented in arbitrary resolution. To generate the continuous representation for images, we train an encoder with LIIF representation via a self-supervised task with super-resolution. The learned continuous representation can be presented in arbitrary resolution even extrapolate to x30 higher resolution, where the training tasks are not provided. We further show that LIIF representation builds a bridge between discrete and continuous representation in 2D, it naturally supports the learning tasks with size-varied image ground-truths and significantly outperforms the method with resizing the ground-truths.