Frame Rate Up-Conversion Using Key Point Agnostic Frequency-Selective Mesh-to-Grid Resampling

TL;DR

This paper introduces an advanced mesh-to-grid resampling method, AFSMR, for high frame rate video up-conversion that improves contrast significantly but requires longer processing time.

Contribution

The paper proposes a novel key point agnostic frequency-selective mesh-to-grid resampling method (AFSMR) that enhances contrast in frame rate up-conversion tasks with irregular mesh densities.

Findings

AFSMR outperforms FSMR with up to 3.2 dB contrast gain.

AFSMR increases processing time by a factor of 11.

AFSMR is effective for irregular meshes with varying densities.

Abstract

High frame rates are desired in many fields of application. As in many cases the frame repetition rate of an already captured video has to be increased, frame rate up-conversion (FRUC) is of high interest. We conduct a motion compensated approach. From two neighboring frames, the motion is estimated and the neighboring pixels are shifted along the motion vector into the frame to be reconstructed. For displaying, these irregularly distributed mesh pixels have to be resampled onto regularly spaced grid positions. We use the model-based key point agnostic frequency-selective mesh-to-grid resampling (AFSMR) for this task and show that AFSMR works best for applications that contain irregular meshes with varying densities. AFSMR gains up to 3.2 dB in contrast to the already high performing frequency-selective mesh-to-grid resampling (FSMR). Additionally, AFSMR increases the run time by a factor of 11 relative to FSMR.