中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 数学与系统科学研究院
  3. 中国科学院数学与系统科学研究院
On the Asymptotic Convergence and Acceleration of Gradient Methods

文献类型：期刊论文

作者Huang, Yakui1; Dai, Yu-Hong2,3; Liu, Xin-Wei1; Zhang, Hongchao4
刊名JOURNAL OF SCIENTIFIC COMPUTING
出版日期2022
卷号90期号:1页码:29
关键词Gradient methods Asymptotic convergence Spectral property Acceleration of gradient methods Barzilai-Borwein method Unconstrained optimization Quadratic optimization
ISSN号0885-7474
DOI10.1007/s10915-021-01685-8
英文摘要We consider the asymptotic behavior of a family of gradient methods, which include the steepest descent and minimal gradient methods as special instances. It is proved that each method in the family will asymptotically zigzag between two directions. Asymptotic convergence results of the objective value, gradient norm, and stepsize are presented as well. To accelerate the family of gradient methods, we further exploit spectral properties of stepsizes to break the zigzagging pattern. In particular, a new stepsize is derived by imposing finite termination on minimizing two-dimensional strictly convex quadratic function. It is shown that, for the general quadratic function, the proposed stepsize asymptotically converges to the reciprocal of the largest eigenvalue of the Hessian. Furthermore, based on this spectral property, we propose a periodic gradient method by incorporating the Barzilai-Borwein method. Numerical comparisons with some recent successful gradient methods show that our new method is very promising.
资助项目National Natural Science Foundation of China[11701137] ; National Natural Science Foundation of China[11631013] ; National Natural Science Foundation of China[12071108] ; National Natural Science Foundation of China[11671116] ; National Natural Science Foundation of China[11991021] ; National Natural Science Foundation of China[12021001] ; Strategic Priority Research Program of Chinese Academy of Sciences[XDA27000000] ; Beijing Academy of Artificial Intelligence (BAAI) ; Natural Science Foundation of Hebei Province[A2021202010] ; China Scholarship Council[201806705007] ; USA National Science Foundation[DMS-1819161] ; USA National Science Foundation[DMS-2110722]
WOS研究方向Mathematics
语种英语
WOS记录号WOS:000720653400002
出版者SPRINGER/PLENUM PUBLISHERS
源URL[http://ir.amss.ac.cn/handle/2S8OKBNM/59571]  
专题中国科学院数学与系统科学研究院
通讯作者Zhang, Hongchao
作者单位1.Hebei Univ Technol, Inst Math, Tianjin 300401, Peoples R China
2.Chinese Acad Sci, Acad Math & Syst Sci, ICMSEC, LSEC, Beijing 100190, Peoples R China
3.Univ Chinese Acad Sci, Sch Math Sci, Beijing 100049, Peoples R China
4.Louisiana State Univ, Dept Math, Baton Rouge, LA 70803 USA
推荐引用方式
GB/T 7714		 Huang, Yakui,Dai, Yu-Hong,Liu, Xin-Wei,et al. On the Asymptotic Convergence and Acceleration of Gradient Methods[J]. JOURNAL OF SCIENTIFIC COMPUTING,2022,90(1):29.
APA Huang, Yakui,Dai, Yu-Hong,Liu, Xin-Wei,&Zhang, Hongchao.(2022).On the Asymptotic Convergence and Acceleration of Gradient Methods.JOURNAL OF SCIENTIFIC COMPUTING,90(1),29.
MLA Huang, Yakui,et al."On the Asymptotic Convergence and Acceleration of Gradient Methods".JOURNAL OF SCIENTIFIC COMPUTING 90.1(2022):29.

入库方式： OAI收割

来源：数学与系统科学研究院

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。