UniISP: A Unified ISP Framework for Both Human and Machine Vision

TL;DR

UniISP is a unified ISP framework that enhances raw sensor data processing to satisfy both human visual quality and machine vision needs, using attention mechanisms and feature adaptation.

Contribution

The paper introduces UniISP, a novel ISP framework with a Hybrid Attention Module and Feature Adapter for improved visual quality and downstream task performance.

Findings

Achieves state-of-the-art results across multiple datasets.

Ensures generated images are visually appealing for humans.

Effectively propagates informative features to downstream networks.

Abstract

Compared to RGB images, raw sensor data provides a richer representation of information, which is crucial for accurate recognition, particularly under challenging conditions such as low-light environments. The traditional Image Signal Processing (ISP) pipeline generates visually pleasing RGB images for human perception through a series of steps, but some of these operations may adversely impact the information integrity by introducing compression and loss. Furthermore, in computer vision tasks that directly utilize raw camera data, most existing methods integrate minimal ISP processing with downstream networks, yet the resulting images are often difficult to visualize or do not align with human aesthetic preferences. This paper proposes UniISP, a novel ISP framework designed to simultaneously meet the requirements of both human visual perception and computer vision applications. By incorporating a carefully designed Hybrid Attention Module (HAM) and employing supervised learning, the proposed method ensures that the generated images are visually appealing. Additionally, a Feature Adapter module is introduced to effectively propagate informative features from the ISP stage to subsequent downstream networks. Extensive experiments demonstrate that our approach achieves state-of-the-art performance across various scenarios and multiple datasets, proving its generalizability and effectiveness.