On the Convergence of Tsetlin Machines for the IDENTITY- and NOT Operators

TL;DR

This paper provides a mathematical analysis of the convergence properties of the Tsetlin Machine when learning the IDENTITY and NOT operators, explaining its ability to learn logical functions and capture rare patterns.

Contribution

It offers the first convergence analysis of the Tsetlin Machine for basic logical operators, enhancing understanding of its theoretical foundations.

Findings

TM with one clause converges correctly to logical operators

It can learn rare patterns and resolve pattern conflicts

Provides insights into TM's state-of-the-art performance

Abstract

The Tsetlin Machine (TM) is a recent machine learning algorithm with several distinct properties, such as interpretability, simplicity, and hardware-friendliness. Although numerous empirical evaluations report on its performance, the mathematical analysis of its convergence is still open. In this article, we analyze the convergence of the TM with only one clause involved for classification. More specifically, we examine two basic logical operators, namely, the "IDENTITY"- and "NOT" operators. Our analysis reveals that the TM, with just one clause, can converge correctly to the intended logical operator, learning from training data over an infinite time horizon. Besides, it can capture arbitrarily rare patterns and select the most accurate one when two candidate patterns are incompatible, by configuring a granularity parameter. The analysis of the convergence of the two basic operators lays the foundation for analyzing other logical operators. These analyses altogether, from a mathematical perspective, provide new insights on why TMs have obtained state-of-the-art performance on several pattern recognition problems.