Rebenchmarking Unsupervised Monocular 3D Occupancy Prediction
Zizhan Guo, Yi Feng, Mengtan Zhang, Haoran Zhang, Wei Ye, Rui Fan
TL;DR
This paper introduces a new benchmark and evaluation protocol for unsupervised monocular 3D occupancy prediction, improving physical consistency and occlusion handling, leading to performance comparable to supervised methods.
Contribution
It reformulates the evaluation protocol based on physically consistent occupancy representation and introduces an occlusion-aware mechanism using multi-view cues.
Findings
Outperforms existing unsupervised methods
Matches the performance of supervised approaches
Provides a new benchmark and evaluation protocol
Abstract
Inferring the 3D structure from a single image, particularly in occluded regions, remains a fundamental yet unsolved challenge in vision-centric autonomous driving. Existing unsupervised approaches typically train a neural radiance field and treat the network outputs as occupancy probabilities during evaluation, overlooking the inconsistency between training and evaluation protocols. Moreover, the prevalent use of 2D ground truth fails to reveal the inherent ambiguity in occluded regions caused by insufficient geometric constraints. To address these issues, this paper presents a reformulated benchmark for unsupervised monocular 3D occupancy prediction. We first interpret the variables involved in the volume rendering process and identify the most physically consistent representation of the occupancy probability. Building on these analyses, we improve existing evaluation protocols by…
Peer Reviews
Decision·Submitted to ICLR 2026
1. Clarity of Presentation: The paper is well-structured with clear explanations of concepts, methodologies, and experimental results. 2. Significance of Benchmark Construction: Constructing a benchmark for monocular 3D occupancy prediction addresses a critical gap in the field. As unsupervised monocular 3D occupancy prediction is essential for vision-centric autonomous driving, a dedicated benchmark contributes to standardized evaluation and fair comparison of subsequent methods, which is of gr
1. **Limited Dataset Evaluation:** The paper only conducts experiments on the KITTI-360 dataset and lacks evaluation on mainstream autonomous driving datasets such as nuScenes. Mainstream datasets like nuScenes cover more complex scenarios, and evaluating only on KITTI-360 fails to demonstrate the generalizability of the proposed benchmark and method. This limits the reliability of the work’s conclusions regarding real-world applicability. 2. **Insignificant Performance Advantages:** As shown in
1. This method identifies the inconsistency between point-wise rendering weight outputs and voxel-wise ground truth in existing NeRF-based methods, and uses opacity to resolve this, improving evaluation reliability. 2. The coordinate-transformed sampling effectively bridges the spatial gap between radial opacity and uniform voxel grids, enabling direct comparison between unsupervised and supervised methods. 3. The occlusion-aware polarization mechanism leverages color differences between adjacen
1. The benchmark relies solely on the KITTI-360 dataset. No experiments are conducted on other datasets, such as nuScenes, to verify the method’s generalizability to different driving scenarios. 2. Qualitative results for occluded regions lack dedicated quantitative metrics, making it hard to objectively assess the polarization mechanism’s improvement on occlusion reasoning. Additionally, there is no clear explanation as to whether the improved ability of occlusion reasoning contributes to the s
1. The paper thoughtfully revisits evaluation protocols and provides a clear, equation-rich justification for why opacity ($\alpha$) is preferable to network density ($\sigma$) for occupancy probability. 2. The methodology is generally well explained, algorithms are clearly laid out, supplemental details/figures are available, and the quantification of evaluation regions enhances reproducibility and interpretability. 3. Robust Experimental Suite: Quantitative and ablation results on KITTI-360 pr
1. All results are on a single dataset (KITTI-360), and there is no exploration of other driving, indoor, or multi-modal datasets. This raises questions about generalization and the broader applicability of the proposed evaluation protocol and methods. Given the benchmark’s ambition for field-wide adoption, a demonstration on at least one additional, differently-distributed dataset would have been highly appropriate.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Computer Graphics and Visualization Techniques · 3D Shape Modeling and Analysis