What Matters to an LLM? Behavioral and Computational Evidences from Summarization

TL;DR

This paper investigates what large language models prioritize during summarization by combining behavioral experiments and computational analysis, revealing consistent importance patterns and internal representations.

Contribution

It introduces a novel combined behavioral and computational approach to understand LLM importance, highlighting attention heads and layers linked to importance.

Findings

LLMs show consistent importance patterns across documents.

Attention heads in middle-to-late layers align with importance.

LLMs cluster more by family than by size in importance patterns.

Abstract

Large Language Models (LLMs) are now state-of-the-art at summarization, yet the internal notion of importance that drives their information selections remains hidden. We propose to investigate this by combining behavioral and computational analyses. Behaviorally, we generate a series of length-controlled summaries for each document and derive empirical importance distributions based on how often each information unit is selected. These reveal that LLMs converge on consistent importance patterns, sharply different from pre-LLM baselines, and that LLMs cluster more by family than by size. Computationally, we identify that certain attention heads align well with empirical importance distributions, and that middle-to-late layers are strongly predictive of importance. Together, these results provide initial insights into what LLMs prioritize in summarization and how this priority is internally represented, opening a path toward interpreting and ultimately controlling information selection in these models.