LLM-Guided Semantic Bootstrapping for Interpretable Text Classification with Tsetlin Machines

TL;DR

This paper introduces a semantic bootstrapping method that transfers knowledge from large language models into Tsetlin Machines, enhancing interpretability and accuracy in text classification without needing embeddings or runtime LLM calls.

Contribution

It presents a novel framework that combines LLM-generated sub-intents with symbolic Tsetlin Machines, enabling semantic understanding and improved performance in interpretable text classification.

Findings

Achieves BERT-level accuracy with fully symbolic models.

Enhances interpretability by extracting high-confidence semantic literals.

Requires no embeddings or real-time LLM calls during inference.

Abstract

Pretrained language models (PLMs) like BERT provide strong semantic representations but are costly and opaque, while symbolic models such as the Tsetlin Machine (TM) offer transparency but lack semantic generalization. We propose a semantic bootstrapping framework that transfers LLM knowledge into symbolic form, combining interpretability with semantic capacity. Given a class label, an LLM generates sub-intents that guide synthetic data creation through a three-stage curriculum (seed, core, enriched), expanding semantic diversity. A Non-Negated TM (NTM) learns from these examples to extract high-confidence literals as interpretable semantic cues. Injecting these cues into real data enables a TM to align clause logic with LLM-inferred semantics. Our method requires no embeddings or runtime LLM calls, yet equips symbolic models with pretrained semantic priors. Across multiple text classification tasks, it improves interpretability and accuracy over vanilla TM, achieving performance comparable to BERT while remaining fully symbolic and efficient.