# Sparsity Emerges Naturally in Neural Language Models

## TL;DR

This paper investigates whether well-trained neural language models naturally develop sparsity, finding that frequent words tend to produce sparse activations and gradients, which has implications for interpretability and efficiency.

## Contribution

The study reveals that neural language models inherently exhibit sparsity patterns related to word frequency and function, providing insights into their internal representations.

## Key findings

- Frequent input words have concentrated activations.
- Frequent target words have dispersed activations but concentrated gradients.
- Gradients of function words are more concentrated than those of content words.

## Abstract

Concerns about interpretability, computational resources, and principled inductive priors have motivated efforts to engineer sparse neural models for NLP tasks. If sparsity is important for NLP, might well-trained neural models naturally become roughly sparse? Using the Taxi-Euclidean norm to measure sparsity, we find that frequent input words are associated with concentrated or sparse activations, while frequent target words are associated with dispersed activations but concentrated gradients. We find that gradients associated with function words are more concentrated than the gradients of content words, even controlling for word frequency.