GFFE: G-buffer Free Frame Extrapolation for Low-latency Real-time Rendering

TL;DR

GFFE introduces a novel G-buffer free frame extrapolation method that generates future frames in real-time without additional latency, improving efficiency and ease of integration in high-resolution, high-frame-rate rendering.

Contribution

The paper presents a new G-buffer free extrapolation framework with a neural network that handles disocclusions and shading correction, enabling low-latency real-time rendering without G-buffer dependency.

Findings

Achieves comparable or better results than previous methods

More efficient performance and easier game integration

Handles disocclusions and shading correction effectively

Abstract

Real-time rendering has been embracing ever-demanding effects, such as ray tracing. However, rendering such effects in high resolution and high frame rate remains challenging. Frame extrapolation methods, which don't introduce additional latency as opposed to frame interpolation methods such as DLSS 3 and FSR 3, boost the frame rate by generating future frames based on previous frames. However, it is a more challenging task because of the lack of information in the disocclusion regions, and recent methods also have a high engine integration cost due to requiring G-buffers as input. We propose a \emph{G-buffer free} frame extrapolation, GFFE, with a novel heuristic framework and an efficient neural network, to plausibly generate new frames in real-time without introducing additional latency. We analyze the motion of dynamic fragments and different types of disocclusions, and design the corresponding modules of the extrapolation block to handle them. After filling disocclusions, a light-weight shading correction network is used to correct shading and improve overall quality. GFFE achieves comparable or better results compared to previous interpolation as well as G-buffer-dependent extrapolation methods, with more efficient performance and easier game integration.