An FPGA Architecture for Online Learning using the Tsetlin Machine

TL;DR

This paper introduces a specialized FPGA architecture for efficient online learning with the Tsetlin Machine, enabling on-chip training and inference, suitable for power-critical edge applications with evolving data.

Contribution

It presents a novel FPGA-based infrastructure that supports low-complexity, real-time online learning and inference with the Tsetlin Machine, including on-demand and interleaved training capabilities.

Findings

Supports on-chip offline and online learning

Demonstrates energy and performance trade-offs

Provides modular and reliable testing infrastructure

Abstract

There is a need for machine learning models to evolve in unsupervised circumstances. New classifications may be introduced, unexpected faults may occur, or the initial dataset may be small compared to the data-points presented to the system during normal operation. Implementing such a system using neural networks involves significant mathematical complexity, which is a major issue in power-critical edge applications. This paper proposes a novel field-programmable gate-array infrastructure for online learning, implementing a low-complexity machine learning algorithm called the Tsetlin Machine. This infrastructure features a custom-designed architecture for run-time learning management, providing on-chip offline and online learning. Using this architecture, training can be carried out on-demand on the \ac{FPGA} with pre-classified data before inference takes place. Additionally, our architecture provisions online learning, where training can be interleaved with inference during operation. Tsetlin Machine (TM) training naturally descends to an optimum, with training also linked to a threshold hyper-parameter which is used to reduce the probability of issuing feedback as the TM becomes trained further. The proposed architecture is modular, allowing the data input source to be easily changed, whilst inbuilt cross-validation infrastructure allows for reliable and representative results during system testing. We present use cases for online learning using the proposed infrastructure and demonstrate the energy/performance/accuracy trade-offs.