# Class-Conditional Compression and Disentanglement: Bridging the Gap between Neural Networks and Naive Bayes Classifiers

## TL;DR

This paper proposes a novel class-conditional information bottleneck approach for neural networks, enabling learned representations to be effectively used with naive Bayes classifiers, bridging neural and probabilistic models.

## Contribution

It introduces a class-conditional compression functional, relaxes it with a variational bound, and demonstrates its application for training neural networks with naive Bayes decoders.

## Key findings

- Latent representations are suitable for naive Bayes classification.
- The method bridges neural networks and probabilistic classifiers.
- Experimental framework inspired by recent information bottleneck research.

## Abstract

In this draft, which reports on work in progress, we 1) adapt the information bottleneck functional by replacing the compression term by class-conditional compression, 2) relax this functional using a variational bound related to class-conditional disentanglement, 3) consider this functional as a training objective for stochastic neural networks, and 4) show that the latent representations are learned such that they can be used in a naive Bayes classifier. We continue by suggesting a series of experiments along the lines of Nonlinear In-formation Bottleneck [Kolchinsky et al., 2018], Deep Variational Information Bottleneck [Alemi et al., 2017], and Information Dropout [Achille and Soatto, 2018]. We furthermore suggest a neural network where the decoder architecture is a parameterized naive Bayes decoder.