Transcoded Video Restoration by Temporal Spatial Auxiliary Network
Li Xu, Gang He, Jinjia Zhou, Jie Lei, Weiying Xie, Yunsong Li, Yu-Wing, Tai

TL;DR
This paper introduces TSAN, a novel neural network architecture designed to effectively restore videos that have undergone multiple encoding and transcoding processes, addressing a common real-world challenge in video platform workflows.
Contribution
The paper presents a new transcoded video restoration method that leverages self-supervised attention, multi-frame information, and novel alignment and fusion techniques, improving upon previous methods.
Findings
Outperforms previous video restoration techniques
Utilizes self-supervised attention with intermediate labels
Employs temporal deformable alignment and pyramidal spatial fusion
Abstract
In most video platforms, such as Youtube, and TikTok, the played videos usually have undergone multiple video encodings such as hardware encoding by recording devices, software encoding by video editing apps, and single/multiple video transcoding by video application servers. Previous works in compressed video restoration typically assume the compression artifacts are caused by one-time encoding. Thus, the derived solution usually does not work very well in practice. In this paper, we propose a new method, temporal spatial auxiliary network (TSAN), for transcoded video restoration. Our method considers the unique traits between video encoding and transcoding, and we consider the initial shallow encoded videos as the intermediate labels to assist the network to conduct self-supervised attention training. In addition, we employ adjacent multi-frame information and propose the temporal…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsAdvanced Image Processing Techniques · Generative Adversarial Networks and Image Synthesis · Video Coding and Compression Technologies
