Blur2Sharp: Human Novel Pose and View Synthesis with Generative Prior Refinement

TL;DR

Blur2Sharp introduces a novel framework combining 3D-aware neural rendering and diffusion models to generate sharp, consistent, and realistic human images from a single reference view, advancing pose and view synthesis.

Contribution

It presents a dual-conditioning approach with Human NeRF and diffusion models, integrating hierarchical feature fusion and priors for improved visual quality and consistency.

Findings

Outperforms state-of-the-art in pose and view synthesis

Generates sharper, more realistic human images

Handles challenging scenarios with clothing and occlusions

Abstract

The creation of lifelike human avatars capable of realistic pose variation and viewpoint flexibility remains a fundamental challenge in computer vision and graphics. Current approaches typically yield either geometrically inconsistent multi-view images or sacrifice photorealism, resulting in blurry outputs under diverse viewing angles and complex motions. To address these issues, we propose Blur2Sharp, a novel framework integrating 3D-aware neural rendering and diffusion models to generate sharp, geometrically consistent novel-view images from only a single reference view. Our method employs a dual-conditioning architecture: initially, a Human NeRF model generates geometrically coherent multi-view renderings for target poses, explicitly encoding 3D structural guidance. Subsequently, a diffusion model conditioned on these renderings refines the generated images, preserving fine-grained details and structural fidelity. We further enhance visual quality through hierarchical feature fusion, incorporating texture, normal, and semantic priors extracted from parametric SMPL models to simultaneously improve global coherence and local detail accuracy. Extensive experiments demonstrate that Blur2Sharp consistently surpasses state-of-the-art techniques in both novel pose and view generation tasks, particularly excelling under challenging scenarios involving loose clothing and occlusions.