Generalizable Neural Fields as Partially Observed Neural Processes

TL;DR

This paper introduces a novel framework for training neural fields using partially observed neural processes, enabling better generalization and efficiency compared to existing meta-learning and hypernetwork methods.

Contribution

It proposes a new paradigm that applies neural process algorithms to neural fields, improving generalization and performance over prior meta-learning and hypernetwork approaches.

Findings

Outperforms state-of-the-art meta-learning methods

Achieves better generalization across signals

Demonstrates improved efficiency in neural field training

Abstract

Neural fields, which represent signals as a function parameterized by a neural network, are a promising alternative to traditional discrete vector or grid-based representations. Compared to discrete representations, neural representations both scale well with increasing resolution, are continuous, and can be many-times differentiable. However, given a dataset of signals that we would like to represent, having to optimize a separate neural field for each signal is inefficient, and cannot capitalize on shared information or structures among signals. Existing generalization methods view this as a meta-learning problem and employ gradient-based meta-learning to learn an initialization which is then fine-tuned with test-time optimization, or learn hypernetworks to produce the weights of a neural field. We instead propose a new paradigm that views the large-scale training of neural representations as a part of a partially-observed neural process framework, and leverage neural process algorithms to solve this task. We demonstrate that this approach outperforms both state-of-the-art gradient-based meta-learning approaches and hypernetwork approaches.