中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 自动化研究所
  3. 中国科学院自动化研究所
  4. 多模态人工智能系统全国重点实验室
  5. 先进控制与自动化团队
Stability-Based Generalization Analysis of Distributed Learning Algorithms for Big Data

文献类型：期刊论文

作者Wu, Xinxing1; Zhang, Junping2; Wang, Fei-Yue3,4,5
刊名IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
出版日期2020-03-01
卷号31期号:3页码:801-812
ISSN号2162-237X
关键词Big data distributed learning algorithms distributed simulations generalization
DOI10.1109/TNNLS.2019.2910188
通讯作者Zhang, Junping(jpzhang@fudan.edu.cn) ; Wang, Fei-Yue(feiyue@ieee.org)
英文摘要As one of the efficient approaches to deal with big data, divide-and-conquer distributed algorithms, such as the distributed kernel regression, bootstrap, structured perception training algorithms, and so on, are proposed and broadly used in learning systems. Some learning theories have been built to analyze the feasibility, approximation, and convergence bounds of these distributed learning algorithms. However, less work has been studied on the stability of these distributed learning algorithms. In this paper, we discuss the generalization bounds of distributed learning algorithms from the view of algorithmic stability. First, we introduce a definition of uniform distributed stability for distributed algorithms and study the distributed algorithms' generalization risk bounds. Then, we analyze the stability properties and generalization risk bounds of a kind of regularization-based distributed algorithms. Two generalization distributed risks obtained show that the generalization distributed risk bounds for the difference between their generalization distributed and empirical distributed/leave-one-computer-out risks are closely related to the size of samples n and the amount of working computers m mathcal O(m/n(1/2)). Furthermore, the results in this paper indicate that, for a good generalization regularized distributed kernel algorithm, the regularization parameter lambda should be adjusted with the change of the term m/n(1/2). These theoretic discoveries provide the useful guidance when deploying the distributed algorithms on practical big data platforms. We explore our theoretic analyses through two simulation experiments. Finally, we discuss some problems about the sufficient amount of working computers, nonequivalence, and generalization for distributed learning. We show that the rules for the computation on one single computer may not always hold for distributed learning.
WOS关键词DATA ANALYTICS ; CONQUER ; REGRESSION
资助项目National Key Research and Development Program of China[2018YFB1305104] ; National Natural Science Foundation of China[61673118] ; National Natural Science Foundation of China[61533019] ; Beijing Municipal Science and Technology Commission[Z181100008918007] ; Intel Collaborative Research Institute for Intelligent and Automated Connected Vehicles (ICRI-IACV) ; Shanghai Talents Development Funds[201629]
WOS研究方向Computer Science ; Engineering
语种英语
出版者IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号WOS:000521961300008
资助机构National Key Research and Development Program of China ; National Natural Science Foundation of China ; Beijing Municipal Science and Technology Commission ; Intel Collaborative Research Institute for Intelligent and Automated Connected Vehicles (ICRI-IACV) ; Shanghai Talents Development Funds
源URL[http://ir.ia.ac.cn/handle/173211/38734]  
专题自动化研究所_复杂系统管理与控制国家重点实验室_先进控制与自动化团队
通讯作者Zhang, Junping; Wang, Fei-Yue
作者单位1.Shanghai Tech Inst Elect & Informat, Dept Commun & Informat Engn, Shanghai 201411, Peoples R China
2.Fudan Univ, Sch Comp Sci, Shanghai Key Lab Intelligent Informat Proc, Shanghai 200433, Peoples R China
3.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China
4.Macau Univ Sci & Technol, Inst Syst Engn, Macau 999078, Peoples R China
5.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
推荐引用方式
GB/T 7714		 Wu, Xinxing,Zhang, Junping,Wang, Fei-Yue. Stability-Based Generalization Analysis of Distributed Learning Algorithms for Big Data[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2020,31(3):801-812.
APA Wu, Xinxing,Zhang, Junping,&Wang, Fei-Yue.(2020).Stability-Based Generalization Analysis of Distributed Learning Algorithms for Big Data.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,31(3),801-812.
MLA Wu, Xinxing,et al."Stability-Based Generalization Analysis of Distributed Learning Algorithms for Big Data".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 31.3(2020):801-812.

入库方式： OAI收割

来源：自动化研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。