Learning Single Index Models with Diffusion Priors

TL;DR

This paper introduces a new method using diffusion models for accurate signal recovery in semi-parametric single index models, handling complex nonlinearities with fewer neural evaluations.

Contribution

It proposes an efficient reconstruction technique for single index models with diffusion priors, capable of managing discontinuous and unknown link functions.

Findings

More accurate reconstructions compared to existing methods

Requires only one round of sampling and partial inversion

Uses fewer neural function evaluations

Abstract

Diffusion models (DMs) have demonstrated remarkable ability to generate diverse and high-quality images by efficiently modeling complex data distributions. They have also been explored as powerful generative priors for signal recovery, resulting in a substantial improvement in the quality of reconstructed signals. However, existing research on signal recovery with diffusion models either focuses on specific reconstruction problems or is unable to handle nonlinear measurement models with discontinuous or unknown link functions. In this work, we focus on using DMs to achieve accurate recovery from semi-parametric single index models, which encompass a variety of popular nonlinear models that may have {\em discontinuous} and {\em unknown} link functions. We propose an efficient reconstruction method that only requires one round of unconditional sampling and (partial) inversion of DMs. Theoretical analysis on the effectiveness of the proposed methods has been established under appropriate conditions. We perform numerical experiments on image datasets for different nonlinear measurement models. We observe that compared to competing methods, our approach can yield more accurate reconstructions while utilizing significantly fewer neural function evaluations.