On the Product Rule for Classification Problems

TL;DR

This paper explores the theoretical foundations of the product rule in supervised classification, linking it to MAP classifiers, distance minimization, and feature concatenation, thereby clarifying its assumptions and implications.

Contribution

It provides a comprehensive theoretical analysis of the product rule, establishing its connections to MAP classification, distance metrics, and feature concatenation under various conditions.

Findings

Product rule arises from MAP classifier with equal priors and independent features

Under certain conditions, product rule minimizes weighted squared distances to class centers

Product rule is equivalent to feature vector concatenation in some hypotheses

Abstract

We discuss theoretical aspects of the product rule for classification problems in supervised machine learning for the case of combining classifiers. We show that (1) the product rule arises from the MAP classifier supposing equivalent priors and conditional independence given a class; (2) under some conditions, the product rule is equivalent to minimizing the sum of the squared distances to the respective centers of the classes related with different features, such distances being weighted by the spread of the classes; (3) observing some hypothesis, the product rule is equivalent to concatenating the vectors of features.