Deep Nonparametric Convexified Filtering for Computational Photography, Image Synthesis and Adversarial Defense

TL;DR

This paper introduces Deep Nonparametric Convexified Filtering (DNCF), a general framework for computational photography that enhances image recovery, robustness, and adversarial defense without relying on training data.

Contribution

It proposes a nonparametric deep network framework that models physical image formation equations, offering strong generalization and robustness, with accelerated inference and adversarial defense capabilities.

Findings

DNCF achieves 10X faster inference than Deep Image Prior.

It effectively defends image classifiers against adversarial attacks in real-time.

The method generalizes well across various image restoration tasks.

Abstract

We aim to provide a general framework of for computational photography that recovers the real scene from imperfect images, via the Deep Nonparametric Convexified Filtering (DNCF). It is consists of a nonparametric deep network to resemble the physical equations behind the image formation, such as denoising, super-resolution, inpainting, and flash. DNCF has no parameterization dependent on training data, therefore has a strong generalization and robustness to adversarial image manipulation. During inference, we also encourage the network parameters to be nonnegative and create a bi-convex function on the input and parameters, and this adapts to second-order optimization algorithms with insufficient running time, having 10X acceleration over Deep Image Prior. With these tools, we empirically verify its capability to defend image classification deep networks against adversary attack algorithms in real-time.