A Relational Tsetlin Machine with Applications to Natural Language Understanding

TL;DR

This paper extends Tsetlin Machines to a first-order logic framework, enabling relational reasoning and improved natural language understanding with more compact knowledge bases and higher accuracy.

Contribution

It introduces a first-order logic-based relational Tsetlin Machine that leverages logical structures in language for better rule learning and reasoning.

Findings

Achieves 10x more compact knowledge bases in question-answering.

Increases answering accuracy from 94.83% to 99.48%.

Demonstrates robustness to noisy and incomplete data.

Abstract

TMs are a pattern recognition approach that uses finite state machines for learning and propositional logic to represent patterns. In addition to being natively interpretable, they have provided competitive accuracy for various tasks. In this paper, we increase the computing power of TMs by proposing a first-order logic-based framework with Herbrand semantics. The resulting TM is relational and can take advantage of logical structures appearing in natural language, to learn rules that represent how actions and consequences are related in the real world. The outcome is a logic program of Horn clauses, bringing in a structured view of unstructured data. In closed-domain question-answering, the first-order representation produces 10x more compact KBs, along with an increase in answering accuracy from 94.83% to 99.48%. The approach is further robust towards erroneous, missing, and superfluous information, distilling the aspects of a text that are important for real-world understanding.