SMART SLM: Structured Memory and Reasoning Transformer, A Small Language Model for Accurate Document Assistance

TL;DR

SMART SLM introduces a hierarchical, structured approach with a compact transformer and memory system to improve accuracy and efficiency in document assistance tasks involving complex engineering manuals.

Contribution

It presents a novel hierarchical model combining syntax-aware fact extraction, indexed memory, and a small transformer, achieving higher accuracy with fewer parameters than larger models.

Findings

Achieves 21.3% higher accuracy than GPT-2.

Uses 64% fewer parameters than GPT-2 and BERT.

Provides fast inference for known documents and effective retrieval for new uploads.

Abstract

The user of Engineering Manuals (EM) finds it difficult to read EM s because they are long, have a dense format which includes written documents, step by step procedures, and standard parameter lists for engineering equipment. Off the shelf transformers, especially compact ones, treat this material as a flat stream of tokens. This approach leads to confident but incorrect numeric answers and forces the models to memorize separate facts inefficiently. SMART (Structured Memory and Reasoning Transformer) offers a different and practical solution to the above problem. SMART structures its processing by using a hierarchical approach, and is based upon three main job categories (1) A syntax-aware Fact Extractor (Grammarian) Tree LSTM which extracts facts as subject relation object relations from EM sentences (2) A compact indexed memory MANN (Memory Augmented Neural Network) that indexes these Rational Subject Relation Objects as 384 dimensional vectors that are associated with the source of the information, and (3) A 6 layer Transformer that learns to fuse the previously retrieved facts into its generated response. The entire SMART model utilizes 45.51M parameters, which is 64% less than GPT-2 (124M) and 69% less than BERT (133M), and it achieves a 21.3% higher accuracy than GPT-2, indicating that SMART fits the data better with the least amount of processing requirements. SMART employs dual modes of inference an indexed fast path for known documents (sub-second answer times) and an indexed dynamic path assisted by RAGs for new uploads (FAISS Top 20 results with memory severed at 64 slots). In real world deployment, this framework leads to more well supported results with reduced hallucinations than comparable small transformer models.