The CM Algorithm for the Maximum Mutual Information Classifications of Unseen Instances

TL;DR

This paper introduces the Channels Matching (CM) algorithm for Maximum Mutual Information classification of unseen instances, emphasizing simplicity, speed, and potential integration with neural networks for high-dimensional data.

Contribution

The paper presents a novel CM algorithm that simplifies MMI classification by avoiding complex boundary searches and introduces semantic information methods for improved classification.

Findings

CM algorithm achieves over 99% mutual information with few iterations.

The algorithm is simple and fast for low-dimensional feature spaces.

Potential integration with neural networks for high-dimensional spaces.

Abstract

The Maximum Mutual Information (MMI) criterion is different from the Least Error Rate (LER) criterion. It can reduce failing to report small probability events. This paper introduces the Channels Matching (CM) algorithm for the MMI classifications of unseen instances. It also introduces some semantic information methods, which base the CM algorithm. In the CM algorithm, label learning is to let the semantic channel match the Shannon channel (Matching I) whereas classifying is to let the Shannon channel match the semantic channel (Matching II). We can achieve the MMI classifications by repeating Matching I and II. For low-dimensional feature spaces, we only use parameters to construct n likelihood functions for n different classes (rather than to construct partitioning boundaries as gradient descent) and expresses the boundaries by numerical values. Without searching in parameter spaces, the computation of the CM algorithm for low-dimensional feature spaces is very simple and fast. Using a two-dimensional example, we test the speed and reliability of the CM algorithm by different initial partitions. For most initial partitions, two iterations can make the mutual information surpass 99% of the convergent MMI. The analysis indicates that for high-dimensional feature spaces, we may combine the CM algorithm with neural networks to improve the MMI classifications for faster and more reliable convergence.