Intermediate Layer Optimization for Inverse Problems using Deep Generative Models

TL;DR

This paper introduces Intermediate Layer Optimization (ILO), a new method that progressively optimizes deeper layers of deep generative models to improve inverse problem solutions, outperforming existing techniques.

Contribution

The paper presents ILO, a novel optimization algorithm that enhances inverse problem solving by optimizing over intermediate layers of deep generative models, with theoretical error bounds and empirical superiority.

Findings

Outperforms state-of-the-art methods like StyleGAN-2 and PULSE.

Provides theoretical error bounds for compressed sensing.

Effective across various inverse problems such as inpainting, denoising, and super-resolution.

Abstract

We propose Intermediate Layer Optimization (ILO), a novel optimization algorithm for solving inverse problems with deep generative models. Instead of optimizing only over the initial latent code, we progressively change the input layer obtaining successively more expressive generators. To explore the higher dimensional spaces, our method searches for latent codes that lie within a small $l_1$ ball around the manifold induced by the previous layer. Our theoretical analysis shows that by keeping the radius of the ball relatively small, we can improve the established error bound for compressed sensing with deep generative models. We empirically show that our approach outperforms state-of-the-art methods introduced in StyleGAN-2 and PULSE for a wide range of inverse problems including inpainting, denoising, super-resolution and compressed sensing.