Dissecting Multiplication in Transformers: Insights into LLMs

TL;DR

This paper analyzes how transformers perform integer multiplication, identifies their limitations in handling carryovers, and proposes improvements that significantly boost accuracy, enhancing interpretability and trust in large language models.

Contribution

The paper provides a detailed analysis of transformers' shortcomings in multiplication and introduces targeted enhancements that improve performance and interpretability.

Findings

Transformers decompose multiplication into parallel subtasks.

Difficulty in calculating carryovers limits performance.

Proposed improvements achieve over 99.9% accuracy on 5-digit multiplication.

Abstract

Transformer-based large language models have achieved remarkable performance across various natural language processing tasks. However, they often struggle with seemingly easy tasks like arithmetic despite their vast capabilities. This stark disparity raise human's concerns about their safe and ethical use, hinder their widespread adoption.In this paper, we focus on a typical arithmetic task, integer multiplication, to explore and explain the imperfection of transformers in this domain. We provide comprehensive analysis of a vanilla transformer trained to perform n-digit integer multiplication. Our observations indicate that the model decomposes multiplication task into multiple parallel subtasks, sequentially optimizing each subtask for each digit to complete the final multiplication. Based on observation and analysis, we infer the reasons of transformers deficiencies in multiplication tasks lies in their difficulty in calculating successive carryovers and caching intermediate results, and confirmed this inference through experiments. Guided by these findings, we propose improvements to enhance transformers performance on multiplication tasks. These enhancements are validated through rigorous testing and mathematical modeling, not only enhance transformer's interpretability, but also improve its performance, e.g., we achieve over 99.9% accuracy on 5-digit integer multiplication with a tiny transformer, outperform LLMs GPT-4. Our method contributes to the broader fields of model understanding and interpretability, paving the way for analyzing more complex tasks and Transformer models. This work underscores the importance of explainable AI, helping to build trust in large language models and promoting their adoption in critical applications.