A Differentiable Wave Optics Model for End-to-End Computational Imaging System Optimization

TL;DR

This paper introduces a differentiable wave optics simulator that enables end-to-end optimization of imaging systems, improving robustness and performance by accurately modeling aberration and diffraction effects.

Contribution

It presents an efficient differentiable optics simulator that models both aberration and diffraction, facilitating more accurate end-to-end optimization of computational imaging systems.

Findings

Optimized lenses and algorithms vary with wave optics modeling.

Systems without wave optics modeling perform worse when wave effects are considered.

Accurate wave optics modeling enhances robustness and performance.

Abstract

End-to-end optimization, which simultaneously optimizes optics and algorithms, has emerged as a powerful data-driven method for computational imaging system design. This method achieves joint optimization through backpropagation by incorporating differentiable optics simulators to generate measurements and algorithms to extract information from measurements. However, due to high computational costs, it is challenging to model both aberration and diffraction in light transport for end-to-end optimization of compound optics. Therefore, most existing methods compromise physical accuracy by neglecting wave optics effects or off-axis aberrations, which raises concerns about the robustness of the resulting designs. In this paper, we propose a differentiable optics simulator that efficiently models both aberration and diffraction for compound optics. Using the simulator, we conduct end-to-end optimization on scene reconstruction and classification. Experimental results demonstrate that both lenses and algorithms adopt different configurations depending on whether wave optics is modeled. We also show that systems optimized without wave optics suffer from performance degradation when wave optics effects are introduced during testing. These findings underscore the importance of accurate wave optics modeling in optimizing imaging systems for robust, high-performance applications.