UniFixer: A Universal Reference-Guided Fixer for Diffusion-Based View Synthesis

TL;DR

UniFixer is a universal, reference-guided framework that improves diffusion-based view synthesis by fixing artifacts through a coarse-to-fine, plug-and-play approach, enhancing detail and structural fidelity.

Contribution

It introduces UniFixer, a novel universal refiner that addresses diffusion degradation artifacts across spatial, temporal, and backbone dimensions in a zero-shot manner.

Findings

Achieves state-of-the-art results on view synthesis and stereo conversion.

Effectively fixes diverse diffusion artifacts with a plug-and-play design.

Demonstrates superior quality in recovering details and structures.

Abstract

With the recent surge of generative models, diffusion-based approaches have become mainstream for view synthesis tasks, either in an explicit depth-warp-inpaint or in an implicit end-to-end manner. Despite their success, both paradigms often suffer from noticeable quality degradation, e.g., blurred details and distorted structures, caused by pixel-to-latent compression and diffusion hallucination. In this paper, we investigate diffusion degradation from three key dimensions (i.e., spatial, temporal, and backbone-related) and propose UniFixer, a universal reference-guided framework that fixes diverse degradation artifacts via a coarse-to-fine strategy. Specifically, a reference pre-alignment module is first designed to perform coarse alignment between the reference view and the degraded novel view. A global structure anchoring mechanism then rectifies geometric distortions to ensure structural fidelity, followed by a local detail injection module that recovers fine-grained texture details for high-quality view synthesis. Our UniFixer serves as a plug-and-play refiner that achieves zero-shot fixing across different types of diffusion degradation, and extensive experiments verify our state-of-the-art performance on novel view synthesis and stereo conversion.