Realistic Image Degradation with Measured PSF

TL;DR

This paper introduces a neural network-based numerical model for the PSF of optical systems, enabling realistic image degradation simulation for autonomous vehicle training and validation.

Contribution

A portable, parameterized PSF model using neural networks that can incorporate measured lens data into optical simulations for autonomous vehicle applications.

Findings

Developed a neural network-based PSF model from lens measurements.

Enabled realistic image degradation by convolving images with the modeled PSF.

Validated the model's applicability in autonomous driving scenarios.

Abstract

Training autonomous vehicles requires lots of driving sequences in all situations\cite{zhao2016}. Typically a simulation environment (software-in-the-loop, SiL) accompanies real-world test drives to systematically vary environmental parameters. A missing piece in the optical model of those SiL simulations is the sharpness, given in linear system theory by the point-spread function (PSF) of the optical system. We present a novel numerical model for the PSF of an optical system that can efficiently model both experimental measurements and lens design simulations of the PSF. The numerical basis for this model is a non-linear regression of the PSF with an artificial neural network (ANN). The novelty lies in the portability and the parameterization of this model, which allows to apply this model in basically any conceivable optical simulation scenario, e.g. inserting a measured lens into a computer game to train autonomous vehicles. We present a lens measurement series, yielding a numerical function for the PSF that depends only on the parameters defocus, field and azimuth. By convolving existing images and videos with this PSF model we apply the measured lens as a transfer function, therefore generating an image as if it were seen with the measured lens itself. Applications of this method are in any optical scenario, but we focus on the context of autonomous driving, where quality of the detection algorithms depends directly on the optical quality of the used camera system. With the parameterization of the optical model we present a method to validate the functional and safety limits of camera-based ADAS based on the real, measured lens actually used in the product.