Rethinking Graph-Based Document Classification: Learning Data-Driven Structures Beyond Heuristic Approaches

TL;DR

This paper introduces a data-driven method for constructing graph structures in document classification, replacing heuristic approaches with learned dependencies, leading to improved accuracy and robustness.

Contribution

It proposes a novel approach to learn graph structures using self-attention, reducing reliance on heuristics and domain-specific rules in document classification.

Findings

Learned graphs outperform heuristic-based graphs in accuracy and F1 score.

Statistical filtering enhances classification robustness.

The approach generalizes well across multiple datasets.

Abstract

In document classification, graph-based models effectively capture document structure, overcoming sequence length limitations and enhancing contextual understanding. However, most existing graph document representations rely on heuristics, domain-specific rules, or expert knowledge. Unlike previous approaches, we propose a method to learn data-driven graph structures, eliminating the need for manual design and reducing domain dependence. Our approach constructs homogeneous weighted graphs with sentences as nodes, while edges are learned via a self-attention model that identifies dependencies between sentence pairs. A statistical filtering strategy aims to retain only strongly correlated sentences, improving graph quality while reducing the graph size. Experiments on three document classification datasets demonstrate that learned graphs consistently outperform heuristic-based graphs, achieving higher accuracy and $F_1$ score. Furthermore, our study demonstrates the effectiveness of the statistical filtering in improving classification robustness. These results highlight the potential of automatic graph generation over traditional heuristic approaches and open new directions for broader applications in NLP.