ASMR: Activation-sharing Multi-resolution Coordinate Networks For Efficient Inference

TL;DR

This paper introduces ASMR, a novel coordinate network that significantly improves inference efficiency by sharing activations across multi-resolution grids, reducing MAC cost while maintaining or improving reconstruction quality.

Contribution

The paper proposes the Activation-Sharing Multi-Resolution (ASMR) coordinate network, which decouples inference cost from network depth, enabling near O(1) complexity regardless of layers.

Findings

ASMR reduces MAC by up to 500x compared to vanilla SIREN.

ASMR achieves higher reconstruction quality than baseline models.

Inference complexity is nearly independent of network depth.

Abstract

Coordinate network or implicit neural representation (INR) is a fast-emerging method for encoding natural signals (such as images and videos) with the benefits of a compact neural representation. While numerous methods have been proposed to increase the encoding capabilities of an INR, an often overlooked aspect is the inference efficiency, usually measured in multiply-accumulate (MAC) count. This is particularly critical in use cases where inference throughput is greatly limited by hardware constraints. To this end, we propose the Activation-Sharing Multi-Resolution (ASMR) coordinate network that combines multi-resolution coordinate decomposition with hierarchical modulations. Specifically, an ASMR model enables the sharing of activations across grids of the data. This largely decouples its inference cost from its depth which is directly correlated to its reconstruction capability, and renders a near O(1) inference complexity irrespective of the number of layers. Experiments show that ASMR can reduce the MAC of a vanilla SIREN model by up to 500x while achieving an even higher reconstruction quality than its SIREN baseline.