Spatial Functa: Scaling Functa to ImageNet Classification and Generation

TL;DR

This paper introduces spatial functa, a scalable neural field framework that effectively handles complex datasets like ImageNet, achieving competitive results in image classification and generation.

Contribution

The paper proposes spatial functa, a novel approach that overcomes previous limitations and scales neural fields to large datasets like ImageNet-1k.

Findings

Achieves ImageNet classification at 256x256 resolution

Performs competitively with Vision Transformers on classification

Generates images comparable to Latent Diffusion models

Abstract

Neural fields, also known as implicit neural representations, have emerged as a powerful means to represent complex signals of various modalities. Based on this Dupont et al. (2022) introduce a framework that views neural fields as data, termed *functa*, and proposes to do deep learning directly on this dataset of neural fields. In this work, we show that the proposed framework faces limitations when scaling up to even moderately complex datasets such as CIFAR-10. We then propose *spatial functa*, which overcome these limitations by using spatially arranged latent representations of neural fields, thereby allowing us to scale up the approach to ImageNet-1k at 256x256 resolution. We demonstrate competitive performance to Vision Transformers (Steiner et al., 2022) on classification and Latent Diffusion (Rombach et al., 2022) on image generation respectively.