Neural Camera Simulators

TL;DR

This paper introduces a deep neural network-based controllable camera simulator that synthesizes raw images under various settings, outperforming baselines and enabling applications like HDR and data augmentation.

Contribution

It is the first to combine traditional raw sensor features with deep learning for realistic camera sensor simulation.

Findings

Outperforms relevant baselines in raw data synthesis.

Enables applications like HDR, auto exposure, and data augmentation.

Uses a dataset of 10,000 raw images across 450 scenes.

Abstract

We present a controllable camera simulator based on deep neural networks to synthesize raw image data under different camera settings, including exposure time, ISO, and aperture. The proposed simulator includes an exposure module that utilizes the principle of modern lens designs for correcting the luminance level. It also contains a noise module using the noise level function and an aperture module with adaptive attention to simulate the side effects on noise and defocus blur. To facilitate the learning of a simulator model, we collect a dataset of the 10,000 raw images of 450 scenes with different exposure settings. Quantitative experiments and qualitative comparisons show that our approach outperforms relevant baselines in raw data synthesize on multiple cameras. Furthermore, the camera simulator enables various applications, including large-aperture enhancement, HDR, auto exposure, and data augmentation for training local feature detectors. Our work represents the first attempt to simulate a camera sensor's behavior leveraging both the advantage of traditional raw sensor features and the power of data-driven deep learning.