Developing a Self-Explanatory Transformer

TL;DR

This paper introduces a novel, training-free transformer based on non-deterministic finite automata for real-time, cost-effective data processing in IoT applications, specifically optimized for FPGA implementation.

Contribution

It presents a self-explanatory, training-free mapping transformer leveraging finite automata, addressing training time issues in NLP tasks on FPGAs.

Findings

Enables real-time processing with reduced training requirements

Offers cost-effective and dataset-loading advantages

Addresses design challenges for future improvements

Abstract

While IoT devices provide significant benefits, their rapid growth results in larger data volumes, increased complexity, and higher security risks. To manage these issues, techniques like encryption, compression, and mapping are used to process data efficiently and securely. General-purpose and AI platforms handle these tasks well, but mapping in natural language processing is often slowed by training times. This work explores a self-explanatory, training-free mapping transformer based on non-deterministic finite automata, designed for Field-Programmable Gate Arrays (FPGAs). Besides highlighting the advantages of this proposed approach in providing real-time, cost-effective processing and dataset-loading, we also address the challenges and considerations for enhancing the design in future iterations.