UnMix-NeRF: Spectral Unmixing Meets Neural Radiance Fields
Fabian Perez, Sara Rojas, Carlos Hinojosa, Hoover Rueda-Chac\'on, Bernard Ghanem

TL;DR
UnMix-NeRF combines spectral unmixing with neural radiance fields to enable joint hyperspectral view synthesis and unsupervised material segmentation, improving material perception in 3D scenes.
Contribution
It introduces a novel framework integrating spectral unmixing into NeRF for unsupervised material segmentation and scene editing capabilities.
Findings
Superior spectral reconstruction compared to existing methods.
Effective unsupervised material segmentation demonstrated.
Enables flexible scene editing through endmember modification.
Abstract
Neural Radiance Field (NeRF)-based segmentation methods focus on object semantics and rely solely on RGB data, lacking intrinsic material properties. This limitation restricts accurate material perception, which is crucial for robotics, augmented reality, simulation, and other applications. We introduce UnMix-NeRF, a framework that integrates spectral unmixing into NeRF, enabling joint hyperspectral novel view synthesis and unsupervised material segmentation. Our method models spectral reflectance via diffuse and specular components, where a learned dictionary of global endmembers represents pure material signatures, and per-point abundances capture their distribution. For material segmentation, we use spectral signature predictions along learned endmembers, allowing unsupervised material clustering. Additionally, UnMix-NeRF enables scene editing by modifying learned endmember…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Computer Graphics and Visualization Techniques · Advanced Neural Network Applications
MethodsFocus
