Classification-Denoising Networks
Louis Thiry, Florentin Guth
TL;DR
This paper introduces a unified model for image classification and denoising that improves robustness and efficiency by jointly modeling noisy images and labels, leveraging score matching and the Tweedie-Miyasawa formula.
Contribution
It proposes a novel joint probability framework combining classification and denoising, with a training method that integrates cross-entropy and score matching losses.
Findings
Achieves competitive classification and denoising results on CIFAR-10 and ImageNet.
Significantly enhances robustness to adversarial attacks.
Provides a new interpretation of adversarial gradients as denoiser differences.
Abstract
Image classification and denoising suffer from complementary issues of lack of robustness or partially ignoring conditioning information. We argue that they can be alleviated by unifying both tasks through a model of the joint probability of (noisy) images and class labels. Classification is performed with a forward pass followed by conditioning. Using the Tweedie-Miyasawa formula, we evaluate the denoising function with the score, which can be computed by marginalization and back-propagation. The training objective is then a combination of cross-entropy loss and denoising score matching loss integrated over noise levels. Numerical experiments on CIFAR-10 and ImageNet show competitive classification and denoising performance compared to reference deep convolutional classifiers/denoisers, and significantly improves efficiency compared to previous joint approaches. Our model shows an…
Peer Reviews
Decision·Submitted to ICLR 2025
- The attempt to jointly classify and denoise looks interesting. One network can be effectively used for both classification and denoising.
- It is less convincing why one needs to combine the tasks of classification and denoising into a single network. Moreover, it is unclear what are the novel contributions of this work over prior works such as JEM. - The evaluation look simplified by using too small networks for the problem like ImageNet classification, by denoising too small images with simple synthetic noises, and by using too limited benchmarks. - Diffusion models were mentioned in this work, but there is no experimental resu
The authors propose a nice joint formulation for denoising and classification. The mathematical derivation is clear and the formulation is sound. If there were not many prior works that did very similar things I would have recommended accepting the paper. But a proper work should compare to prior works...
The paper elegantly (or may be not) ignores all the prior works that use diffusion models to perform classification. In fact, they cite one work (your diffusion model is secretly one shot classifier) but don't compare to it. Indeed, the work they already cite, deals less with robustness but there are many other works that perform joint classification and denoising and study robustness. For example: (CERTIFIED!!) ADVERSARIAL ROBUSTNESS FOR FREE!, ICLR 2023 Robust Classification via a Single Diffu
+ Research on the joint learning of denoising and classification is limited; this paper highlights the potential of such integrated approaches. + The numerical results for classification surpass those of previous joint methods. + The relationship between the proposed method and the adversarial noise and energy-based models is analyzed in depth and discussed in detail.
- While this paper demonstrates superior performance in joint methods compared to others, it does not show significant advantages in standalone classification or denoising tasks. The denoising task under Gaussian noise can be viewed as a sampling process within a score-based diffusion model that uses Gaussian noise as a prior. In other words, the method presented in this paper sacrifices generative capabilities but learns a fixed noise-level denoiser in favor of enhanced classification performan
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNeural Networks and Applications
MethodsDenoising Score Matching