Abstract:Recent NLP systems commonly represent documents as linear token sequences. Although this captures sequential order, it can hinder modeling long-range dependencies and global document structure, especially for long texts. This paper proposes a data-driven method to automatically construct graph-based document representations. Building upon the recent work of Bugueño and de Melo (2025), we leverage the dynamic sliding-window attention module to effectively capture local and mid-range semantic dependencies between sentences, as well as structural relations within documents. Graph Attention Networks (GATs) trained on our learned graphs achieve competitive results on document classification while requiring lower computational resources than previous approaches. We further present an exploratory evaluation of the proposed graph construction method for extractive document summarization, highlighting both its potential and current limitations. The implementation of this project can be found on GitHub.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2603.00021 [cs.CL] |
| (or arXiv:2603.00021v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2603.00021 arXiv-issued DOI via DataCite |
Submission history
From: Ruangrin Ldallitsakool [view email]
[v1]
Tue, 3 Feb 2026 14:48:47 UTC (3,244 KB)
[v2]
Sat, 30 May 2026 11:05:28 UTC (3,554 KB)