Joint Enhancement and Classification using Coupled Diffusion Models of Signals and Logits

TL;DR

This paper introduces a novel coupled diffusion model framework that jointly enhances signals and classifies in noisy environments, leveraging mutual guidance between signal and logits without retraining classifiers.

Contribution

It presents a domain-agnostic, coupled diffusion approach that integrates signal enhancement and classification, improving robustness without classifier retraining.

Findings

Outperforms traditional sequential enhancement methods in noisy conditions.

Improves classification accuracy in image and speech tasks under noise.

Enables mutual guidance between signal and logits without retraining classifiers.

Abstract

Robust classification in noisy environments remains a fundamental challenge in machine learning. Standard approaches typically treat signal enhancement and classification as separate, sequential stages: first enhancing the signal and then applying a classifier. This approach fails to leverage the semantic information in the classifier's output during denoising. In this work, we propose a general, domain-agnostic framework that integrates two interacting diffusion models: one operating on the input signal and the other on the classifier's output logits, without requiring any retraining or fine-tuning of the classifier. This coupled formulation enables mutual guidance, where the enhancing signal refines the class estimation and, conversely, the evolving class logits guide the signal reconstruction towards discriminative regions of the manifold. We introduce three strategies to effectively model the joint distribution of the input and the logit. We evaluated our joint enhancement method for image classification and automatic speech recognition. The proposed framework surpasses traditional sequential enhancement baselines, delivering robust and flexible improvements in classification accuracy under diverse noise conditions.