Graph-Agnostic Linear Transformers
文献类型：期刊论文
| 作者 | Guo, Zhiyu1,2; Liu, Yang2; Ao, Xiang1,2,3; Tang, Yateng; Chen, Xinhuan; Zheng, Xuehao; He, Qing1,2 |
| 刊名 | NEURAL NETWORKS
 |
| 出版日期 | 2026-06-01 |
| 卷号 | 198页码:12 |
| 关键词 | Graph neural network Graph transformer Linear attention Graph-agnostic model |
| ISSN号 | 0893-6080 |
| DOI | 10.1016/j.neunet.2026.108595 |
| 英文摘要 | Graph Transformers (GTs), as emerging foundational encoders for graph-structured data, have shown promising performance due to the integration of local graph structures with global attention mechanisms. However, the complex attention functions and their coupling with graph structures incur significant computational overhead, particularly in large-scale graphs. In this paper, we decouple graph structures from Transformers and propose the Graph-Agnostic Linear Transformer (GALiT). In GALiT, graph structures are solely utilized to denoise raw node features before training, as our findings reveal that these denoised features have integrated the main information of the graph structure and can replace it to guide Transformers. By excluding graph structures from the training and inference stages, GALiT serves as a graph-agnostic model which significantly reduces computational complexity. Additionally, we simplify the linear attention functions inherited from traditional Transformers, which further reduces computational overhead while still capturing the relationships between nodes. Through weighted combination, we integrate the denoised features into the attention mechanism, as our theoretical analysis reveals the key role of the synergy between linear attention and denoised features in enhancing representation diversity. Despite decoupling graph structures and simplifying attention mechanisms, our model surprisingly outperforms most GNNs and GTs on benchmark graphs. Experimental results indicate that GALiT achieves high efficiency while maintaining or even enhancing performance. |
| 资助项目 | National Natural Science Foundation of China[62406307] ; National Natural Science Foundation of China[62576333] ; National Natural Science Foundation of China[62476263] ; Strategic Priority Research Program of the Chinese Academy of Sciences[XDB0680201] ; Beijing Natural Science Foundation[JQ25015] ; Innovation Funding of ICT, CAS[E461060] ; Postdoctoral Fellowship Program of CPSF[GZB20240761] |
| WOS研究方向 | Computer Science ; Neurosciences & Neurology |
| 语种 | 英语 |
| WOS记录号 | WOS:001676295000001 |
| 出版者 | PERGAMON-ELSEVIER SCIENCE LTD |
| 源URL | [http://119.78.100.204/handle/2XEOYT63/42851]  |
| 专题 | 中国科学院计算技术研究所 |
| 通讯作者 | Liu, Yang |
| 作者单位 | 1.Univ Chinese Acad Sci, Beijing 100190, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, State Key Lab AI Safety, Beijing 100190, Peoples R China 3.Chinese Acad Sci, Inst Intelligent Comp Technol, Suzhou 215100, Peoples R China |
| 推荐引用方式 GB/T 7714 | Guo, Zhiyu,Liu, Yang,Ao, Xiang,et al. Graph-Agnostic Linear Transformers[J]. NEURAL NETWORKS,2026,198:12. |
| APA | Guo, Zhiyu.,Liu, Yang.,Ao, Xiang.,Tang, Yateng.,Chen, Xinhuan.,...&He, Qing.(2026).Graph-Agnostic Linear Transformers.NEURAL NETWORKS,198,12. |
| MLA | Guo, Zhiyu,et al."Graph-Agnostic Linear Transformers".NEURAL NETWORKS 198(2026):12. |
入库方式： OAI收割
来源：计算技术研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。