Structure-Preserving Patch Decoding for Efficient Neural Video Representation

TL;DR

This paper introduces a structure-preserving patch decoding method for neural video representations that enhances reconstruction quality and reduces boundary artifacts by maintaining spatial coherence through a global-to-local decoding strategy.

Contribution

It proposes a novel patch-based INR approach that preserves spatial structure and improves high-frequency detail reconstruction in neural video modeling.

Findings

Higher reconstruction quality compared to existing INR methods

Better compression performance on standard video datasets

Reduced boundary artifacts and spatial discontinuities

Abstract

Implicit neural representations (INRs) are the subject of extensive research, particularly in their application to modeling complex signals by mapping spatial and temporal coordinates to corresponding values. When handling videos, mapping compact inputs to entire frames or spatially partitioned patch images is an effective approach. This strategy better preserves spatial relationships, reduces computational overhead, and improves reconstruction quality compared to coordinate-based mapping. However, predicting entire frames often limits the reconstruction of high-frequency visual details. Additionally, conventional patch-based approaches based on uniform spatial partitioning tend to introduce boundary discontinuities that degrade spatial coherence. We propose a neural video representation method based on Structure-Preserving Patches (SPPs) to address such limitations. Our method separates each video frame into patch images of spatially aligned frames through a deterministic pixel-based splitting similar to PixelUnshuffle. This operation preserves the global spatial structure while allowing patch-level decoding. We train the decoder to reconstruct these structured patches, enabling a global-to-local decoding strategy that captures the global layout first and refines local details. This effectively reduces boundary artifacts and mitigates distortions from naive upsampling. Experiments on standard video datasets demonstrate that our method achieves higher reconstruction quality and better compression performance than existing INR-based baselines.