From Global to Local: Learning Context-Aware Graph Representations for Document Classification and Summarization

TL;DR

This paper introduces a data-driven, context-aware graph construction method for document classification and summarization, leveraging dynamic attention to capture semantic dependencies efficiently.

Contribution

It presents a novel graph construction approach using dynamic sliding-window attention, improving classification performance and exploring summarization applications.

Findings

Achieves competitive classification results with lower computational costs

Demonstrates potential of learned graphs for extractive summarization

Highlights limitations and future directions for graph-based summarization

Abstract

This paper proposes a data-driven method to automatically construct graph-based document representations. Building upon the recent work of Bugue\~no and de Melo (2025), we leverage the dynamic sliding-window attention module to effectively capture local and mid-range semantic dependencies between sentences, as well as structural relations within documents. Graph Attention Networks (GATs) trained on our learned graphs achieve competitive results on document classification while requiring lower computational resources than previous approaches. We further present an exploratory evaluation of the proposed graph construction method for extractive document summarization, highlighting both its potential and current limitations. The implementation of this project can be found on GitHub.