Mechanistic Interpretability of Binary and Ternary Transformers

TL;DR

This paper investigates whether binary and ternary transformer networks learn different algorithms from full-precision models, finding they learn similar algorithms in a toy problem, which questions their interpretability advantage.

Contribution

It applies mechanistic interpretability techniques to binary and ternary transformers, revealing they learn similar algorithms as full-precision models in a specific task.

Findings

Binary and ternary networks learn similar algorithms to full-precision networks.

Results challenge the idea that lower-precision models are more interpretable.

Study conducted on the modular addition toy problem.

Abstract

Recent research (arXiv:2310.11453, arXiv:2402.17764) has proposed binary and ternary transformer networks as a way to significantly reduce memory and improve inference speed in Large Language Models (LLMs) while maintaining accuracy. In this work, we apply techniques from mechanistic interpretability to investigate whether such networks learn distinctly different or similar algorithms when compared to full-precision transformer networks. In particular, we reverse engineer the algorithms learned for the toy problem of modular addition where we find that binary and ternary networks learn similar algorithms as full precision networks. This provides evidence against the possibility of using binary and ternary networks as a more interpretable alternative in the LLM setting.