Zoom in to the details of human-centric videos

TL;DR

This paper introduces a human-centric video super-resolution method that leverages high-level human appearance priors and motion analysis to produce high-quality, detailed HR human sequences from LR videos, outperforming traditional methods.

Contribution

The proposed algorithm uniquely incorporates high-level human appearance priors and motion analysis to enhance super-resolution of human-centric videos.

Findings

Outperforms traditional super-resolution methods in visual quality.

Effectively utilizes HR reference frames for better detail reconstruction.

Demonstrates superior results on real-world and dataset videos.

Abstract

Presenting high-resolution (HR) human appearance is always critical for the human-centric videos. However, current imagery equipment can hardly capture HR details all the time. Existing super-resolution algorithms barely mitigate the problem by only considering universal and low-level priors of im-age patches. In contrast, our algorithm is under bias towards the human body super-resolution by taking advantage of high-level prior defined by HR human appearance. Firstly, a motion analysis module extracts inherent motion pattern from the HR reference video to refine the pose estimation of the low-resolution (LR) sequence. Furthermore, a human body reconstruction module maps the HR texture in the reference frames onto a 3D mesh model. Consequently, the input LR videos get super-resolved HR human sequences are generated conditioned on the original LR videos as well as few HR reference frames. Experiments on an existing dataset and real-world data captured by hybrid cameras show that our approach generates superior visual quality of human body compared with the traditional method.