ISETHDR: A Physics-based Synthetic Radiance Dataset for High Dynamic Range Driving Scenes

TL;DR

This paper introduces ISETHDR, a physics-based synthetic HDR radiance dataset for driving scenes, along with a simulation framework for sensor analysis, enabling improved HDR sensor design and testing.

Contribution

It provides a novel synthetic HDR dataset and a validated simulation framework for analyzing and comparing HDR sensors in driving environments.

Findings

Created a labeled HDR driving scene dataset

Developed and validated an end-to-end simulation framework

Compared two HDR sensors using the dataset and simulation

Abstract

This paper describes a physics-based end-to-end software simulation for image systems. We use the software to explore sensors designed to enhance performance in high dynamic range (HDR) environments, such as driving through daytime tunnels and under nighttime conditions. We synthesize physically realistic HDR spectral radiance images and use them as the input to digital twins that model the optics and sensors of different systems. This paper makes three main contributions: (a) We create a labeled (instance segmentation and depth), synthetic radiance dataset of HDR driving scenes. (b) We describe the development and validation of the end-to-end simulation framework. (c) We present a comparative analysis of two single-shot sensors designed for HDR. We open-source both the dataset and the software.