Behavioral Analysis of Information Salience in Large Language Models

TL;DR

This paper investigates how large language models determine what information is most important during summarization, revealing a hierarchical but internally inaccessible notion of salience that only weakly aligns with human judgments.

Contribution

It introduces an explainable framework to analyze information salience in LLMs through their summarization behavior, providing new insights into their content prioritization mechanisms.

Findings

LLMs exhibit a hierarchical and consistent notion of salience.

Salience patterns are similar across different models and sizes.

Internal salience does not strongly correlate with human perceptions.

Abstract

Large Language Models (LLMs) excel at text summarization, a task that requires models to select content based on its importance. However, the exact notion of salience that LLMs have internalized remains unclear. To bridge this gap, we introduce an explainable framework to systematically derive and investigate information salience in LLMs through their summarization behavior. Using length-controlled summarization as a behavioral probe into the content selection process, and tracing the answerability of Questions Under Discussion throughout, we derive a proxy for how models prioritize information. Our experiments on 13 models across four datasets reveal that LLMs have a nuanced, hierarchical notion of salience, generally consistent across model families and sizes. While models show highly consistent behavior and hence salience patterns, this notion of salience cannot be accessed through introspection, and only weakly correlates with human perceptions of information salience.