Primal-dual residual networks

TL;DR

This paper introduces a neural network architecture inspired by primal-dual splitting methods, connecting deep learning with convex optimization techniques, and demonstrates its effectiveness in speech dequantization tasks.

Contribution

The work presents a novel neural network design based on primal-dual splitting, establishing a theoretical link to residual networks and applying it to unroll optimization algorithms.

Findings

The architecture closely relates to residual networks.

It outperforms classical splitting methods in speech dequantization.

The approach effectively unrolls optimization algorithms for constrained problems.

Abstract

In this work, we propose a deep neural network architecture motivated by primal-dual splitting methods from convex optimization. We show theoretically that there exists a close relation between the derived architecture and residual networks, and further investigate this connection in numerical experiments. Moreover, we demonstrate how our approach can be used to unroll optimization algorithms for certain problems with hard constraints. Using the example of speech dequantization, we show that our method can outperform classical splitting methods when both are applied to the same task.