DynamicISP: Dynamically Controlled Image Signal Processor for Image Recognition

TL;DR

DynamicISP adaptively adjusts image signal processing parameters in real-time based on recognition feedback, achieving high accuracy with low computational cost for image recognition tasks.

Contribution

We introduce DynamicISP, a novel method that dynamically controls ISP parameters per frame using recognition results, balancing expressive power and efficiency.

Findings

Achieves state-of-the-art accuracy in object detection

Maintains low computational cost on edge devices

Effectively controls multiple ISP functions dynamically

Abstract

Image Signal Processors (ISPs) play important roles in image recognition tasks as well as in the perceptual quality of captured images. In most cases, experts make a lot of effort to manually tune many parameters of ISPs, but the parameters are sub-optimal. In the literature, two types of techniques have been actively studied: a machine learning-based parameter tuning technique and a DNN-based ISP technique. The former is lightweight but lacks expressive power. The latter has expressive power, but the computational cost is too heavy on edge devices. To solve these problems, we propose "DynamicISP," which consists of multiple classical ISP functions and dynamically controls the parameters of each frame according to the recognition result of the previous frame. We show our method successfully controls the parameters of multiple ISP functions and achieves state-of-the-art accuracy with low computational cost in single and multi-category object detection tasks.