BGDB: Bernoulli-Gaussian Decision Block with Improved Denoising Diffusion Probabilistic Models

TL;DR

This paper introduces BGDB, a novel module that uses probabilistic modeling inspired by the Central Limit Theorem to improve denoising diffusion models by reconstructing logits as if from multiple training sessions, enhancing generative classifier performance.

Contribution

The paper proposes BGDB, a new module leveraging IDDPM and the CLT to synthesize multiple training logits from a single model, providing a theoretical foundation and experimental validation.

Findings

BGDB improves denoising diffusion probabilistic models' performance.

Theoretical analysis supports the CLT-based approach.

Experimental results show enhanced classification and segmentation accuracy.

Abstract

Generative models can enhance discriminative classifiers by constructing complex feature spaces, thereby improving performance on intricate datasets. Conventional methods typically augment datasets with more detailed feature representations or increase dimensionality to make nonlinear data linearly separable. Utilizing a generative model solely for feature space processing falls short of unlocking its full potential within a classifier and typically lacks a solid theoretical foundation. We base our approach on a novel hypothesis: the probability information (logit) derived from a single model training can be used to generate the equivalent of multiple training sessions. Leveraging the central limit theorem, this synthesized probability information is anticipated to converge toward the true probability more accurately. To achieve this goal, we propose the Bernoulli-Gaussian Decision Block (BGDB), a novel module inspired by the Central Limit Theorem and the concept that the mean of multiple Bernoulli trials approximates the probability of success in a single trial. Specifically, we utilize Improved Denoising Diffusion Probabilistic Models (IDDPM) to model the probability of Bernoulli Trials. Our approach shifts the focus from reconstructing features to reconstructing logits, transforming the logit from a single iteration into logits analogous to those from multiple experiments. We provide the theoretical foundations of our approach through mathematical analysis and validate its effectiveness through experimental evaluation using various datasets for multiple imaging tasks, including both classification and segmentation.