MiA-Signature: Approximating Global Activation for Long-Context Understanding

TL;DR

This paper introduces MiA-Signature, a compact representation of global activation patterns in language models, improving long-context understanding by approximating full activation states efficiently.

Contribution

It proposes MiA-Signature, a submodular-based compressed activation representation that enhances long-context processing in LLMs with minimal computational overhead.

Findings

MiA-Signature improves performance on long-context tasks.

Integration of MiA-Signature yields consistent gains in RAG and agentic systems.

The method efficiently approximates global activation effects.

Abstract

A growing body of work in cognitive science suggests that reportable conscious access is associated with \emph{global ignition} over distributed memory systems, while such activation is only partially accessible as individuals cannot directly access or enumerate all activated contents. This tension suggests a plausible mechanism that cognition may rely on a compact representation that approximates the global influence of activation on downstream processing. Inspired by this idea, we introduce the concept of \textbf{Mindscape Activation Signature (MiA-Signature)}, a compressed representation of the global activation pattern induced by a query. In LLM systems, this is instantiated via submodular-based selection of high-level concepts that cover the activated context space, optionally refined through lightweight iterative updates using working memory. The resulting MiA-Signature serves as a conditioning signal that approximates the effect of the full activation state while remaining computationally tractable. Integrating MiA-Signatures into both RAG and agentic systems yields consistent performance gains across multiple long-context understanding tasks.