中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
Towards generalizable Graph Contrastive Learning: An information theory perspective

文献类型：期刊论文

作者Yuan, Yige; Xu, Bingbing; Shen, Huawei; Cao, Qi; Cen, Keting; Zheng, Wen; Cheng, Xueqi
刊名NEURAL NETWORKS
出版日期2024-04-01
卷号172页码:17
关键词Graph Contrastive Learning Generalization Information theory
ISSN号0893-6080
DOI10.1016/j.neunet.2024.106125
英文摘要Graph Contrastive Learning (GCL) is increasingly employed in graph representation learning with the primary aim of learning node/graph representations from a predefined pretext task that can generalize to various downstream tasks. Meanwhile, the transition from a specific pretext task to diverse and unpredictable downstream tasks poses a significant challenge for GCL's generalization ability. Most existing GCL approaches maximize mutual information between two views derived from the original graph, either randomly or heuristically. However, the generalization ability of GCL and its theoretical principles are still less studied. In this paper, we introduce a novel metric GCL-GE, to quantify the generalization gap between predefined pretext and agnostic downstream tasks. Given the inherent intractability of GCL-GE, we leverage concepts from information theory to derive a mutual information upper bound that is independent of the downstream tasks, thus enabling the metric's optimization despite the variability in downstream tasks. Based on the theoretical insight, we propose InfoAdv, a GCL framework to directly enhance generalization by jointly optimizing GCL-GE and InfoMax. Extensive experiments validate the capability of InfoAdv to enhance performance across a wide variety of downstream tasks, demonstrating its effectiveness in improving the generalizability of GCL.
资助项目National Natural Science Foun-dation of China[U21B2046] ; National Natural Science Foun-dation of China[62202448] ; Na-tional Key R&D Program of China[2022YFB3103700] ; Na-tional Key R&D Program of China[2022YFB3103704]
WOS研究方向Computer Science ; Neurosciences & Neurology
语种英语
WOS记录号WOS:001179930100001
出版者PERGAMON-ELSEVIER SCIENCE LTD
源URL[http://119.78.100.204/handle/2XEOYT63/38768]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Yuan, Yige; Xu, Bingbing
作者单位Chinese Acad Sci, Data Intelligence Syst Res Ctr, Inst Comp Technol, Beijing 100190, Peoples R China
推荐引用方式
GB/T 7714		 Yuan, Yige,Xu, Bingbing,Shen, Huawei,et al. Towards generalizable Graph Contrastive Learning: An information theory perspective[J]. NEURAL NETWORKS,2024,172:17.
APA Yuan, Yige.,Xu, Bingbing.,Shen, Huawei.,Cao, Qi.,Cen, Keting.,...&Cheng, Xueqi.(2024).Towards generalizable Graph Contrastive Learning: An information theory perspective.NEURAL NETWORKS,172,17.
MLA Yuan, Yige,et al."Towards generalizable Graph Contrastive Learning: An information theory perspective".NEURAL NETWORKS 172(2024):17.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。