Geometric Implications of the Naive Bayes Assumption

TL;DR

This paper explores the geometric implications of the Naive Bayes assumption, extending known results about decision surfaces to m-ary observations and examining how dependencies affect these surfaces.

Contribution

It generalizes the hyperplane separability result to m-ary observations and discusses the impact of observation dependencies on decision boundaries.

Findings

Decision surfaces are hyperplanes for binary observations.

Extension of hyperplane separability to m-ary observations.

Observation dependencies influence decision surface geometry.

Abstract

A naive (or Idiot) Bayes network is a network with a single hypothesis node and several observations that are conditionally independent given the hypothesis. We recently surveyed a number of members of the UAI community and discovered a general lack of understanding of the implications of the Naive Bayes assumption on the kinds of problems that can be solved by these networks. It has long been recognized [Minsky 61] that if observations are binary, the decision surfaces in these networks are hyperplanes. We extend this result (hyperplane separability) to Naive Bayes networks with m-ary observations. In addition, we illustrate the effect of observation-observation dependencies on decision surfaces. Finally, we discuss the implications of these results on knowledge acquisition and research in learning.