Exploring State Space and Reasoning by Elimination in Tsetlin Machines

TL;DR

This paper enhances Tsetlin Machines by introducing Reasoning by Elimination to improve clause descriptiveness and representation, employing TM-AE for dense word vectors, and demonstrating robustness across datasets with high accuracy.

Contribution

It proposes a novel Reasoning by Elimination approach within Tsetlin Machines and explores state space regulation to improve pattern representation and classification performance.

Findings

Achieved 90.62% accuracy on IMDB dataset.

Demonstrated improved clause descriptiveness with RbE.

Validated robustness across multiple datasets.

Abstract

The Tsetlin Machine (TM) has gained significant attention in Machine Learning (ML). By employing logical fundamentals, it facilitates pattern learning and representation, offering an alternative approach for developing comprehensible Artificial Intelligence (AI) with a specific focus on pattern classification in the form of conjunctive clauses. In the domain of Natural Language Processing (NLP), TM is utilised to construct word embedding and describe target words using clauses. To enhance the descriptive capacity of these clauses, we study the concept of Reasoning by Elimination (RbE) in clauses' formulation, which involves incorporating feature negations to provide a more comprehensive representation. In more detail, this paper employs the Tsetlin Machine Auto-Encoder (TM-AE) architecture to generate dense word vectors, aiming at capturing contextual information by extracting feature-dense vectors for a given vocabulary. Thereafter, the principle of RbE is explored to improve descriptivity and optimise the performance of the TM. Specifically, the specificity parameter s and the voting margin parameter T are leveraged to regulate feature distribution in the state space, resulting in a dense representation of information for each clause. In addition, we investigate the state spaces of TM-AE, especially for the forgotten/excluded features. Empirical investigations on artificially generated data, the IMDB dataset, and the 20 Newsgroups dataset showcase the robustness of the TM, with accuracy reaching 90.62\% for the IMDB.