Learning the image processing pipeline

TL;DR

This paper introduces a machine learning-based method to automate the design of image processing pipelines by modeling them as collections of local linear filters, streamlining the development for novel sensor architectures.

Contribution

The paper presents a novel approach combining machine learning and simulation to automatically design image processing pipelines as collections of local linear filters.

Findings

Successfully designed pipelines for new sensor architectures

Reduced time and cost in pipeline development

Demonstrated effectiveness in consumer photography applications

Abstract

Many creative ideas are being proposed for image sensor designs, and these may be useful in applications ranging from consumer photography to computer vision. To understand and evaluate each new design, we must create a corresponding image processing pipeline that transforms the sensor data into a form that is appropriate for the application. The need to design and optimize these pipelines is time-consuming and costly. We explain a method that combines machine learning and image systems simulation that automates the pipeline design. The approach is based on a new way of thinking of the image processing pipeline as a large collection of local linear filters. We illustrate how the method has been used to design pipelines for novel sensor architectures in consumer photography applications.