LoFi: Neural Local Fields for Scalable Image Reconstruction

TL;DR

LoFi introduces a local neural field framework for scalable image reconstruction that efficiently processes local information, enabling high-resolution recovery, excellent generalization, and low memory usage, even with small datasets.

Contribution

LoFi presents a novel local neural field approach for image reconstruction that outperforms traditional models in efficiency, resolution flexibility, and data efficiency.

Findings

Achieves comparable or better performance than CNNs and ViTs.

Requires less than 200MB memory for 1024x1024 images.

Can train on datasets with fewer than 10 samples without overfitting.

Abstract

Neural fields or implicit neural representations (INRs) have attracted significant attention in computer vision and imaging due to their efficient coordinate-based representation of images and 3D volumes. In this work, we introduce a coordinate-based framework for solving imaging inverse problems, termed LoFi (Local Field). Unlike conventional methods for image reconstruction, LoFi processes local information at each coordinate separately by multi-layer perceptrons (MLPs), recovering the object at that specific coordinate. Similar to INRs, LoFi can recover images at any continuous coordinate, enabling image reconstruction at multiple resolutions. With comparable or better performance than standard deep learning models like convolutional neural networks (CNNs) and vision transformers (ViTs), LoFi achieves excellent generalization to out-of-distribution data with memory usage almost independent of image resolution. Remarkably, training on 1024x1024 images requires less than 200MB of memory -- much below standard CNNs and ViTs. Additionally, LoFi's local design allows it to train on extremely small datasets with 10 samples or fewer, without overfitting and without the need for explicit regularization or early stopping.