中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
Algorithm-Dependent Generalization of AUPRC Optimization: Theory and Algorithm

文献类型：期刊论文

作者Wen, Peisong1; Xu, Qianqian1; Yang, Zhiyong2; He, Yuan3; Huang, Qingming1,2,4
刊名IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
出版日期2024-07-01
卷号46期号:7页码:5062-5079
关键词Optimization Stability analysis Stochastic processes Measurement Standards Approximation algorithms Machine learning algorithms Machine learning AUPRC learning to rank algorithm-dependent generalization stability
ISSN号0162-8828
DOI10.1109/TPAMI.2024.3361861
英文摘要Stochastic optimization of the Area Under the Precision-Recall Curve (AUPRC) is a crucial problem for machine learning. Despite extensive studies on AUPRC optimization, generalization is still an open problem. In this work, we present the first trial in the algorithm-dependent generalization of stochastic AUPRC optimization. The obstacles to our destination are three-fold. First, according to the consistency analysis, the majority of existing stochastic estimators are biased with biased sampling strategies. To address this issue, we propose a stochastic estimator with sampling-rate-invariant consistency and reduce the consistency error by estimating the full-batch scores with score memory. Second, standard techniques for algorithm-dependent generalization analysis cannot be directly applied to listwise losses. To fill this gap, we extend the model stability from instance-wise losses to listwise losses. Third, AUPRC optimization involves a compositional optimization problem, which brings complicated computations. In this work, we propose to reduce the computational complexity by matrix spectral decomposition. Based on these techniques, we derive the first algorithm-dependent generalization bound for AUPRC optimization. Motivated by theoretical results, we propose a generalization-induced learning framework, which improves the AUPRC generalization by equivalently increasing the batch size and the number of valid training examples. Practically, experiments on image retrieval and long-tailed classification speak to the effectiveness and soundness of our framework.
资助项目National Key R#x0026;D Program of China
WOS研究方向Computer Science ; Engineering
语种英语
WOS记录号WOS:001240147800017
出版者IEEE COMPUTER SOC
源URL[http://119.78.100.204/handle/2XEOYT63/39888]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Xu, Qianqian; Huang, Qingming
作者单位1.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China
2.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing 101408, Peoples R China
3.Alibaba Grp, Hangzhou, Peoples R China
4.Univ Chinese Acad Sci, Key Lab Big Data Min & Knowledge Management BDKM, Beijing 101408, Peoples R China
推荐引用方式
GB/T 7714		 Wen, Peisong,Xu, Qianqian,Yang, Zhiyong,et al. Algorithm-Dependent Generalization of AUPRC Optimization: Theory and Algorithm[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2024,46(7):5062-5079.
APA Wen, Peisong,Xu, Qianqian,Yang, Zhiyong,He, Yuan,&Huang, Qingming.(2024).Algorithm-Dependent Generalization of AUPRC Optimization: Theory and Algorithm.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,46(7),5062-5079.
MLA Wen, Peisong,et al."Algorithm-Dependent Generalization of AUPRC Optimization: Theory and Algorithm".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 46.7(2024):5062-5079.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。