DRL-ISP: Multi-Objective Camera ISP with Deep Reinforcement Learning

TL;DR

This paper introduces a multi-objective camera ISP framework using deep reinforcement learning that adaptively selects ISP tools to optimize image quality for specific vision tasks.

Contribution

It presents a novel DRL-based ISP framework with a large toolbox and an efficient network architecture for task-specific image enhancement.

Findings

Improves image quality for RAW-to-RGB restoration

Enhances 2D object detection accuracy

Boosts monocular depth estimation performance

Abstract

In this paper, we propose a multi-objective camera ISP framework that utilizes Deep Reinforcement Learning (DRL) and camera ISP toolbox that consist of network-based and conventional ISP tools. The proposed DRL-based camera ISP framework iteratively selects a proper tool from the toolbox and applies it to the image to maximize a given vision task-specific reward function. For this purpose, we implement total 51 ISP tools that include exposure correction, color-and-tone correction, white balance, sharpening, denoising, and the others. We also propose an efficient DRL network architecture that can extract the various aspects of an image and make a rigid mapping relationship between images and a large number of actions. Our proposed DRL-based ISP framework effectively improves the image quality according to each vision task such as RAW-to-RGB image restoration, 2D object detection, and monocular depth estimation.