NeRV360: Neural Representation for 360-Degree Videos with a Viewport Decoder

TL;DR

NeRV360 is a novel neural video representation framework that efficiently decodes only user-selected viewports in 360-degree videos, significantly reducing memory and decoding time while improving quality.

Contribution

It introduces an end-to-end viewport decoding method with a spatial-temporal affine transform, enabling efficient high-resolution 360-degree video processing.

Findings

7-fold reduction in memory usage

2.5-fold faster decoding speed

Better image quality than prior methods

Abstract

Implicit neural representations for videos (NeRV) have shown strong potential for video compression. However, applying NeRV to high-resolution 360-degree videos causes high memory usage and slow decoding, making real-time applications impractical. We propose NeRV360, an end-to-end framework that decodes only the user-selected viewport instead of reconstructing the entire panoramic frame. Unlike conventional pipelines, NeRV360 integrates viewport extraction into decoding and introduces a spatial-temporal affine transform module for conditional decoding based on viewpoint and time. Experiments on 6K-resolution videos show that NeRV360 achieves a 7-fold reduction in memory consumption and a 2.5-fold increase in decoding speed compared to HNeRV, a representative prior work, while delivering better image quality in terms of objective metrics.