# Plug-and-Play Unplugged: Optimization Free Reconstruction using Consensus Equilibrium

## TL;DR

This paper introduces Consensus Equilibrium, a flexible framework for image reconstruction that unifies diverse models and denoisers without relying on traditional optimization, enabling improved results through model fusion.

## Contribution

It presents Consensus Equilibrium, a novel approach that generalizes regularized inversion to include heterogeneous models without requiring cost functions, expanding the scope of image reconstruction methods.

## Key findings

- Consensus Equilibrium generalizes MAP estimation.
- Algorithms like ADMM and Newton's method solve CE equations.
- Neural network denoisers in consensus outperform individual denoisers.

## Abstract

Regularized inversion methods for image reconstruction are used widely due to their tractability and ability to combine complex physical sensor models with useful regularity criteria. Such methods motivated the recently developed Plug-and-Play prior method, which provides a framework to use advanced denoising algorithms as regularizers in inversion. However, the need to formulate regularized inversion as the solution to an optimization problem limits the possible regularity conditions and physical sensor models. In this paper, we introduce Consensus Equilibrium (CE), which generalizes regularized inversion to include a much wider variety of both forward components and prior components without the need for either to be expressed with a cost function. CE is based on the solution of a set of equilibrium equations that balance data fit and regularity. In this framework, the problem of MAP estimation in regularized inversion is replaced by the problem of solving these equilibrium equations, which can be approached in multiple ways. The key contribution of CE is to provide a novel framework for fusing multiple heterogeneous models of physical sensors or models learned from data. We describe the derivation of the CE equations and prove that the solution of the CE equations generalizes the standard MAP estimate under appropriate circumstances. We also discuss algorithms for solving the CE equations, including ADMM with a novel form of preconditioning and Newton's method. We give examples to illustrate consensus equilibrium and the convergence properties of these algorithms and demonstrate this method on some toy problems and on a denoising example in which we use an array of convolutional neural network denoisers, none of which is tuned to match the noise level in a noisy image but which in consensus can achieve a better result than any of them individually.