PQDAST: Depth-Aware Arbitrary Style Transfer for Games via Perceptual Quality-Guided Distillation

TL;DR

PQDAST introduces a depth-aware, real-time style transfer method for games that maintains high stylisation quality and temporal stability while significantly reducing memory and processing requirements.

Contribution

It presents the first depth-aware, arbitrary style transfer framework integrated into game pipelines using perceptual quality-guided distillation and synthetic training data.

Findings

Achieves superior temporal consistency in style transfer for games.

Reduces memory usage and processing time compared to existing methods.

Maintains comparable stylisation quality with state-of-the-art approaches.

Abstract

Artistic style transfer is concerned with the generation of imagery that combines the content of an image with the style of an artwork. In the realm of computer games, most work has focused on post-processing video frames. Some recent work has integrated style transfer into the game pipeline, but it is limited to single styles. Integrating an arbitrary style transfer method into the game pipeline is challenging due to the memory and speed requirements of games. We present PQDAST, the first solution to address this. We use a perceptual quality-guided knowledge distillation framework and train a compressed model using the FLIP evaluator, which substantially reduces both memory usage and processing time with limited impact on stylisation quality. For better preservation of depth and fine details, we utilise a synthetic dataset with depth and temporal considerations during training. The developed model is injected into the rendering pipeline to further enforce temporal stability and avoid diminishing post-process effects. Quantitative and qualitative experiments demonstrate that our approach achieves superior performance in temporal consistency, with comparable style transfer quality, to state-of-the-art image, video and in-game methods.