Physically Aware 360$^\circ$ View Generation from a Single Image using Disentangled Scene Embeddings

TL;DR

Disentangled360 is a novel 3D-aware view synthesis framework that generates realistic 360° views from a single image, integrating direction-aware rendering with applications in medical imaging and scene reconstruction.

Contribution

It introduces a dual-branch, disentangled scene embedding approach with hybrid pose anchoring, enabling rapid, photorealistic 360° view synthesis without scene-specific fine-tuning.

Findings

Outperforms existing methods in SSIM and LPIPS metrics.

Achieves real-time, photorealistic 360° view synthesis.

Applicable to medical imaging and immersive scene reconstruction.

Abstract

We introduce Disentangled360, an innovative 3D-aware technology that integrates the advantages of direction disentangled volume rendering with single-image 360{\deg} unique view synthesis for applications in medical imaging and natural scene reconstruction. In contrast to current techniques that either oversimplify anisotropic light behavior or lack generalizability across various contexts, our framework distinctly differentiates between isotropic and anisotropic contributions inside a Gaussian Splatting backbone. We implement a dual-branch conditioning framework, one optimized for CT intensity driven scattering in volumetric data and the other for real-world RGB scenes through normalized camera embeddings. To address scale ambiguity and maintain structural realism, we present a hybrid pose agnostic anchoring method that adaptively samples scene depth and material transitions, functioning as stable pivots during scene distillation. Our design integrates preoperative radiography simulation and consumer-grade 360{\deg} rendering into a singular inference pipeline, facilitating rapid, photorealistic view synthesis with inherent directionality. Evaluations on the Mip-NeRF 360, RealEstate10K, and DeepDRR datasets indicate superior SSIM and LPIPS performance, while runtime assessments confirm its viability for interactive applications. Disentangled360 facilitates mixed-reality medical supervision, robotic perception, and immersive content creation, eliminating the necessity for scene-specific finetuning or expensive photon simulations.