Noise-based Enhancement for Foveated Rendering

TL;DR

This paper introduces a perceptually-inspired noise replacement method for foveated rendering that improves performance and visual quality by selectively replacing undetectable high-frequency details with procedural noise, enabling real-time VR and AR applications.

Contribution

It presents a novel noise-based enhancement technique that calibrates noise parameters based on human perception, allowing more aggressive foveation and real-time performance at 4K resolution.

Findings

Runs at over 200FPS at 4K resolution

Improves rendering performance and perceptual quality

Validated through user experiments comparing to existing methods

Abstract

Human visual sensitivity to spatial details declines towards the periphery. Novel image synthesis techniques, so-called foveated rendering, exploit this observation and reduce the spatial resolution of synthesized images for the periphery, avoiding the synthesis of high-spatial-frequency details that are costly to generate but not perceived by a viewer. However, contemporary techniques do not make a clear distinction between the range of spatial frequencies that must be reproduced and those that can be omitted. For a given eccentricity, there is a range of frequencies that are detectable but not resolvable. While the accurate reproduction of these frequencies is not required, an observer can detect their absence if completely omitted. We use this observation to improve the performance of existing foveated rendering techniques. We demonstrate that this specific range of frequencies can be efficiently replaced with procedural noise whose parameters are carefully tuned to image content and human perception. Consequently, these frequencies do not have to be synthesized during rendering, allowing more aggressive foveation, and they can be replaced by noise generated in a less expensive post-processing step, leading to improved performance of the rendering system. Our main contribution is a perceptually-inspired technique for deriving the parameters of the noise required for the enhancement and its calibration. The method operates on rendering output and runs at rates exceeding 200FPS at 4K resolution, making it suitable for integration with real-time foveated rendering systems for VR and AR devices. We validate our results and compare them to the existing contrast enhancement technique in user experiments.