N4MC: Neural 4D Mesh Compression

TL;DR

N4MC is a novel neural framework for efficiently compressing 4D time-varying mesh sequences by leveraging temporal redundancy, motion compensation, and transformer-based interpolation, achieving superior rate-distortion performance and real-time decoding.

Contribution

It introduces the first 4D neural mesh compression method that exploits temporal redundancy and uses a transformer-based model for intermediate frame prediction.

Findings

Outperforms state-of-the-art in rate-distortion metrics

Enables real-time decoding of 4D mesh sequences

Effectively captures spatial and temporal correlations

Abstract

We present N4MC, the first 4D neural compression framework to efficiently compress time-varying mesh sequences by exploiting their temporal redundancy. Unlike prior neural mesh compression methods that treat each mesh frame independently, N4MC takes inspiration from inter-frame compression in 2D video codecs, and learns motion compensation in long mesh sequences. Specifically, N4MC converts consecutive irregular mesh frames into regular 4D tensors to provide a uniform and compact representation. These tensors are then condensed using an auto-decoder, which captures both spatial and temporal correlations for redundancy removal. To enhance temporal coherence, we introduce a transformer-based interpolation model that predicts intermediate mesh frames conditioned on latent embeddings derived from tracked volume centers, eliminating motion ambiguities. Extensive evaluations show that N4MC outperforms state-of-the-art in rate-distortion performance, while enabling real-time decoding of 4D mesh sequences. The implementation of our method is available at: https://github.com/frozzzen3/N4MC.