Restoration of User Videos Shared on Social Media

TL;DR

This paper introduces VOTES, a novel video restoration framework that explicitly models degradation features to improve the quality of user videos shared on social media, outperforming existing methods.

Contribution

The paper proposes a new degradation feature map (DFM) concept and an adaptive estimation process to guide video restoration, along with a large real-world user video dataset.

Findings

VOTES outperforms state-of-the-art methods quantitatively.

VOTES achieves superior qualitative restoration results.

A large-scale real-world user video database is provided.

Abstract

User videos shared on social media platforms usually suffer from degradations caused by unknown proprietary processing procedures, which means that their visual quality is poorer than that of the originals. This paper presents a new general video restoration framework for the restoration of user videos shared on social media platforms. In contrast to most deep learning-based video restoration methods that perform end-to-end mapping, where feature extraction is mostly treated as a black box, in the sense that what role a feature plays is often unknown, our new method, termed Video restOration through adapTive dEgradation Sensing (VOTES), introduces the concept of a degradation feature map (DFM) to explicitly guide the video restoration process. Specifically, for each video frame, we first adaptively estimate its DFM to extract features representing the difficulty of restoring its different regions. We then feed the DFM to a convolutional neural network (CNN) to compute hierarchical degradation features to modulate an end-to-end video restoration backbone network, such that more attention is paid explicitly to potentially more difficult to restore areas, which in turn leads to enhanced restoration performance. We will explain the design rationale of the VOTES framework and present extensive experimental results to show that the new VOTES method outperforms various state-of-the-art techniques both quantitatively and qualitatively. In addition, we contribute a large scale real-world database of user videos shared on different social media platforms. Codes and datasets are available at https://github.com/luohongming/VOTES.git