The Differentiable Lens: Compound Lens Search over Glass Surfaces and Materials for Object Detection

TL;DR

This paper introduces a differentiable lens simulation model and an optimization strategy that jointly designs lens geometries and glass materials, improving object detection performance in automotive applications.

Contribution

It develops a differentiable spherical lens model and a novel optimization method that includes quantized glass variables for end-to-end lens and material design.

Findings

Enhanced object detection accuracy in automotive scenarios.

Effective lens designs with fewer elements despite lower image quality.

Successful joint optimization of lens geometry and materials.

Abstract

Most camera lens systems are designed in isolation, separately from downstream computer vision methods. Recently, joint optimization approaches that design lenses alongside other components of the image acquisition and processing pipeline -- notably, downstream neural networks -- have achieved improved imaging quality or better performance on vision tasks. However, these existing methods optimize only a subset of lens parameters and cannot optimize glass materials given their categorical nature. In this work, we develop a differentiable spherical lens simulation model that accurately captures geometrical aberrations. We propose an optimization strategy to address the challenges of lens design -- notorious for non-convex loss function landscapes and many manufacturing constraints -- that are exacerbated in joint optimization tasks. Specifically, we introduce quantized continuous glass variables to facilitate the optimization and selection of glass materials in an end-to-end design context, and couple this with carefully designed constraints to support manufacturability. In automotive object detection, we report improved detection performance over existing designs even when simplifying designs to two- or three-element lenses, despite significantly degrading the image quality.