A Mamba-based Perceptual Loss Function for Learning-based UGC Transcoding

TL;DR

This paper introduces a perceptual loss function based on a neural quality model using Mamba, improving the perceptual quality of user-generated content transcoding in neural video codecs.

Contribution

It proposes a novel perceptual loss function utilizing a Mamba-based neural quality model, shifting the reference from pixel accuracy to perceptual quality in UGC video transcoding.

Findings

Achieves 8.46% BD-rate savings over autoencoder baselines.

Achieves 12.89% BD-rate savings over implicit neural representation baselines.

Improves perceptual quality in neural video transcoding tasks.

Abstract

In user-generated content (UGC) transcoding, source videos typically suffer various degradations due to prior compression, editing, or suboptimal capture conditions. Consequently, existing video compression paradigms that solely optimize for fidelity relative to the reference become suboptimal, as they force the codec to replicate the inherent artifacts of the non-pristine source. To address this, we propose a novel perceptually inspired loss function for learning-based UGC video transcoding that redefines the role of the reference video, shifting it from a ground-truth pixel anchor to an informative contextual guide. Specifically, we train a lightweight neural quality model based on a Selective Structured State-Space Model (Mamba) optimized using a weakly-supervised Siamese ranking strategy. The proposed model is then integrated into the rate-distortion optimization (RDO) process of two neural video codecs (DCVC and HiNeRV) as a loss function, aiming to generate reconstructed content with improved perceptual quality. Our experiments demonstrate that this framework achieves substantial coding gains over both autoencoder and implicit neural representation-based baselines, with 8.46% and 12.89% BD-rate savings, respectively.