Sparse Video Representation Using Steered Mixture-of-Experts With Global Motion Compensation

TL;DR

This paper introduces a global motion compensation technique into the Steered-Mixtures-of-Experts framework, significantly improving video compression by exploiting inter-frame correlations with minimal additional parameters.

Contribution

It integrates a global motion model into SMoE for videos, enabling better temporal steering and compression efficiency with only a few added motion parameters.

Findings

Reduces the number of kernels needed by 54.25% on average.

Maintains reconstruction quality while increasing compression gains.

Enhances SMoE's capability for video data with nonlinear motion.

Abstract

Steered-Mixtures-of Experts (SMoE) present a unified framework for sparse representation and compression of image data with arbitrary dimensionality. Recent work has shown great improvements in the performance of such models for image and light-field representation. However, for the case of videos the straight-forward application yields limited success as the SMoE framework leads to a piece-wise linear representation of the underlying imagery which is disrupted by nonlinear motion. We incorporate a global motion model into the SMoE framework which allows for higher temporal steering of the kernels. This drastically increases its capabilities to exploit correlations between adjacent frames by only adding 2 to 8 motion parameters per frame to the model but decreasing the required amount of kernels on average by 54.25%, respectively, while maintaining the same reconstruction quality yielding higher compression gains.