Modulated Periodic Activations for Generalizable Local Functional Representations

TL;DR

This paper introduces a novel modulated periodic activation method with a dual-MLP architecture for creating generalizable, high-fidelity functional representations of signals like images and shapes, surpassing prior single-signal optimized methods.

Contribution

The paper proposes a dual-MLP architecture with modulation and local-functional representations, enabling generalization across multiple signals with improved reconstruction quality.

Findings

Achieves state-of-the-art fidelity in multi-instance signal representation

Enables high-quality encoding of images, videos, and shapes

Outperforms prior single-signal optimized methods

Abstract

Multi-Layer Perceptrons (MLPs) make powerful functional representations for sampling and reconstruction problems involving low-dimensional signals like images,shapes and light fields. Recent works have significantly improved their ability to represent high-frequency content by using periodic activations or positional encodings. This often came at the expense of generalization: modern methods are typically optimized for a single signal. We present a new representation that generalizes to multiple instances and achieves state-of-the-art fidelity. We use a dual-MLP architecture to encode the signals. A synthesis network creates a functional mapping from a low-dimensional input (e.g. pixel-position) to the output domain (e.g. RGB color). A modulation network maps a latent code corresponding to the target signal to parameters that modulate the periodic activations of the synthesis network. We also propose a local-functional representation which enables generalization. The signal's domain is partitioned into a regular grid,with each tile represented by a latent code. At test time, the signal is encoded with high-fidelity by inferring (or directly optimizing) the latent code-book. Our approach produces generalizable functional representations of images, videos and shapes, and achieves higher reconstruction quality than prior works that are optimized for a single signal.