MZEN: Multi-Zoom Enhanced NeRF for 3-D Reconstruction with Unknown Camera Poses

TL;DR

MZEN introduces a novel NeRF framework that effectively incorporates multi-zoom images with unknown camera poses, enabling detailed 3D reconstruction crucial for industrial inspection tasks.

Contribution

It proposes a learnable zoom scalar and a pose strategy that aligns wide-field and zoom-in images, extending NeRF to handle multi-zoom scenarios with unknown poses.

Findings

Outperforms pose-free baselines and high-res variants in PSNR, SSIM, and LPIPS.

Successfully reconstructs micro-structures and objects in industrial settings.

Enhances 3D reconstruction accuracy with micron-level detail.

Abstract

Neural Radiance Fields (NeRF) methods excel at 3D reconstruction from multiple 2D images, even those taken with unknown camera poses. However, they still miss the fine-detailed structures that matter in industrial inspection, e.g., detecting sub-micron defects on a production line or analyzing chips with Scanning Electron Microscopy (SEM). In these scenarios, the sensor resolution is fixed and compute budgets are tight, so the only way to expose fine structure is to add zoom-in images; yet, this breaks the multi-view consistency that pose-free NeRF training relies on. We propose Multi-Zoom Enhanced NeRF (MZEN), the first NeRF framework that natively handles multi-zoom image sets. MZEN (i) augments the pin-hole camera model with an explicit, learnable zoom scalar that scales the focal length, and (ii) introduces a novel pose strategy: wide-field images are solved first to establish a global metric frame, and zoom-in images are then pose-primed to the nearest wide-field counterpart via a zoom-consistent crop-and-match procedure before joint refinement. Across eight forward-facing scenes$\unicode{x2013}$synthetic TCAD models, real SEM of micro-structures, and BLEFF objects$\unicode{x2013}$MZEN consistently outperforms pose-free baselines and even high-resolution variants, boosting PSNR by up to $28 \%$, SSIM by $10 \%$, and reducing LPIPS by up to $222 \%$. MZEN, therefore, extends NeRF to real-world factory settings, preserving global accuracy while capturing the micron-level details essential for industrial inspection.