Procedural Kernel Networks

TL;DR

Procedural Kernel Networks (PKNs) are lightweight models that generate filter parameters for image processing, enabling faster and more efficient low-level image restoration on mobile devices by combining traditional algorithms with machine learning.

Contribution

Introduction of PKNs, a novel framework that generates parameters for traditional image filters using lightweight CNNs, improving efficiency and broadening applications in mobile image processing.

Findings

PKNs significantly speed up kernel-based image processing.

PKNs improve traditional algorithms' performance in low-level tasks.

PKNs unify previous machine learning approaches for image restoration.

Abstract

In the last decade Convolutional Neural Networks (CNNs) have defined the state of the art for many low level image processing and restoration tasks such as denoising, demosaicking, upscaling, or inpainting. However, on-device mobile photography is still dominated by traditional image processing techniques, and uses mostly simple machine learning techniques or limits the neural network processing to producing low resolution masks. High computational and memory requirements of CNNs, limited processing power and thermal constraints of mobile devices, combined with large output image resolutions (typically 8--12 MPix) prevent their wider application. In this work, we introduce Procedural Kernel Networks (PKNs), a family of machine learning models which generate parameters of image filter kernels or other traditional algorithms. A lightweight CNN processes the input image at a lower resolution, which yields a significant speedup compared to other kernel-based machine learning methods and allows for new applications. The architecture is learned end-to-end and is especially well suited for a wide range of low-level image processing tasks, where it improves the performance of many traditional algorithms. We also describe how this framework unifies some previous work applying machine learning for common image restoration tasks.