Efficient Parameter Estimation for Bayesian Network Classifiers using Hierarchical Linear Smoothing

TL;DR

This paper introduces a fast, flexible linear smoothing method for Bayesian network classifiers that improves parameter estimation, making them more competitive with random forests while maintaining interpretability.

Contribution

A novel linear regression-based smoothing technique for BNCs that outperforms hierarchical Dirichlet process smoothing in speed and maintains competitive accuracy.

Findings

Outperforms HDP smoothing in speed

Remains competitive with random forests

Offers a simple, interpretable model

Abstract

Bayesian network classifiers (BNCs) possess a number of properties desirable for a modern classifier: They are easily interpretable, highly scalable, and offer adaptable complexity. However, traditional methods for learning BNCs have historically underperformed when compared to leading classification methods such as random forests. Recent parameter smoothing techniques using hierarchical Dirichlet processes (HDPs) have enabled BNCs to achieve performance competitive with random forests on categorical data, but these techniques are relatively inflexible, and require a complicated, specialized sampling process. In this paper, we introduce a novel method for parameter estimation that uses a log-linear regression to approximate the behaviour of HDPs. As a linear model, our method is remarkably flexible and simple to interpret, and can leverage the vast literature on learning linear models. Our experiments show that our method can outperform HDP smoothing while being orders of magnitude faster, remaining competitive with random forests on categorical data.