On Resource-Efficient Bayesian Network Classifiers and Deep Neural Networks

TL;DR

This paper introduces methods for reducing the complexity of Bayesian network classifiers through quantization-aware training and structure learning, enabling effective trade-offs between model size and accuracy, and compares them with quantized deep neural networks.

Contribution

It presents novel quantization and structure learning techniques for Bayesian networks, extending recent methods and analyzing their performance relative to deep neural networks.

Findings

Quantized BN classifiers achieve competitive accuracy with reduced model size.

Proposed methods enable Pareto optimal trade-offs between size, operations, and error.

Both Bayesian networks and DNNs are viable for resource-constrained scenarios.

Abstract

We present two methods to reduce the complexity of Bayesian network (BN) classifiers. First, we introduce quantization-aware training using the straight-through gradient estimator to quantize the parameters of BNs to few bits. Second, we extend a recently proposed differentiable tree-augmented naive Bayes (TAN) structure learning approach by also considering the model size. Both methods are motivated by recent developments in the deep learning community, and they provide effective means to trade off between model size and prediction accuracy, which is demonstrated in extensive experiments. Furthermore, we contrast quantized BN classifiers with quantized deep neural networks (DNNs) for small-scale scenarios which have hardly been investigated in the literature. We show Pareto optimal models with respect to model size, number of operations, and test error and find that both model classes are viable options.