TMComposites: Plug-and-Play Collaboration Between Specialized Tsetlin Machines

TL;DR

This paper introduces TM Composites, a plug-and-play collaboration framework for specialized Tsetlin Machines that enhances their performance on complex image classification tasks by combining diverse expert models.

Contribution

The paper presents a novel collaborative approach allowing specialized Tsetlin Machines to work together without fine-tuning, significantly improving accuracy on challenging datasets.

Findings

Achieved state-of-the-art results for TMs on CIFAR-10 and CIFAR-100.

Improved accuracy on Fashion-MNIST by 2 percentage points.

TM Composites outperform individual TMs by leveraging specialization and confidence-based decision-making.

Abstract

Tsetlin Machines (TMs) provide a fundamental shift from arithmetic-based to logic-based machine learning. Supporting convolution, they deal successfully with image classification datasets like MNIST, Fashion-MNIST, and CIFAR-2. However, the TM struggles with getting state-of-the-art performance on CIFAR-10 and CIFAR-100, representing more complex tasks. This paper introduces plug-and-play collaboration between specialized TMs, referred to as TM Composites. The collaboration relies on a TM's ability to specialize during learning and to assess its competence during inference. When teaming up, the most confident TMs make the decisions, relieving the uncertain ones. In this manner, a TM Composite becomes more competent than its members, benefiting from their specializations. The collaboration is plug-and-play in that members can be combined in any way, at any time, without fine-tuning. We implement three TM specializations in our empirical evaluation: Histogram of Gradients, Adaptive Gaussian Thresholding, and Color Thermometers. The resulting TM Composite increases accuracy on Fashion-MNIST by two percentage points, CIFAR-10 by twelve points, and CIFAR-100 by nine points, yielding new state-of-the-art results for TMs. Overall, we envision that TM Composites will enable an ultra-low energy and transparent alternative to state-of-the-art deep learning on more tasks and datasets.