Abstract:Graph neural networks (GNNs) aim to learn well-trained representations in a lower-dimension space for downstream tasks while preserving the topological structures. In recent years, attention mechanism, which is brilliant in the fields of natural language processing and computer vision, is introduced to GNNs to adaptively select the discriminative features and automatically filter the noisy information. To the best of our knowledge, due to the fast-paced advances in this domain, a systematic overview of attention-based GNNs is still missing. To fill this gap, this paper aims to provide a comprehensive survey on recent advances in attention-based GNNs. Firstly, we propose a novel two-level taxonomy for attention-based GNNs from the perspective of development history and architectural perspectives. Specifically, the upper level reveals the three developmental stages of attention-based GNNs, including graph recurrent attention networks, graph attention networks, and graph transformers. The lower level focuses on various typical architectures of each stage. Secondly, we review these attention-based methods following the proposed taxonomy in detail and summarize the advantages and disadvantages of various models. A model characteristics table is also provided for a more comprehensive comparison. Thirdly, we share our thoughts on some open issues and future directions of attention-based GNNs. We hope this survey will provide researchers with an up-to-date reference regarding applications of attention-based GNNs. In addition, to cope with the rapid development in this field, we intend to share the relevant latest papers as an open resource at this https URL.
| Comments: | This is the accepted manuscript of an article published in Artificial Intelligence Review. The final version is available online at: [https://doi.org/10.1007/s10462-023-10577-2](this https URL) |
| Subjects: | Social and Information Networks (cs.SI); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| MSC classes: | 68T07, 68T05, 68-02 |
| ACM classes: | A.1; G.2.2; I.2.6 |
| Cite as: | arXiv:2605.08679 [cs.SI] |
| (or arXiv:2605.08679v1 [cs.SI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.08679 arXiv-issued DOI via DataCite (pending registration) |
|
| Journal reference: | Artif Intell Rev 56 (Suppl 2), 2263 2310 (2023) |
| Related DOI: | https://doi.org/10.1007/s10462-023-10577-2
DOI(s) linking to related resources |
Submission history
From: Chengcheng Sun [view email]
[v1]
Sat, 9 May 2026 04:33:14 UTC (1,387 KB)