Trained Model in Supervised Deep Learning is a Conditional Risk Minimizer

TL;DR

This paper proves that trained models in supervised deep learning minimize conditional risk, providing new insights into model behavior, connecting supervised and unsupervised learning, and enabling uncertainty estimation in image super-resolution.

Contribution

It establishes that trained models minimize conditional risk, links supervised and unsupervised learning, and offers methods for uncertainty estimation in image tasks.

Findings

Many existing methods are explained by the theorem

Validated property of noisy label classification on MNIST

Demonstrated uncertainty estimation in super-resolution on ImageNet

Abstract

We proved that a trained model in supervised deep learning minimizes the conditional risk for each input (Theorem 2.1). This property provided insights into the behavior of trained models and established a connection between supervised and unsupervised learning in some cases. In addition, when the labels are intractable but can be written as a conditional risk minimizer, we proved an equivalent form of the original supervised learning problem with accessible labels (Theorem 2.2). We demonstrated that many existing works, such as Noise2Score, Noise2Noise and score function estimation can be explained by our theorem. Moreover, we derived a property of classification problem with noisy labels using Theorem 2.1 and validated it using MNIST dataset. Furthermore, We proposed a method to estimate uncertainty in image super-resolution based on Theorem 2.2 and validated it using ImageNet dataset. Our code is available on github.