中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 数学与系统科学研究院
  3. 中国科学院数学与系统科学研究院
Improve Robustness and Accuracy of Deep Neural Network with L-2,L-infinity Normalization

文献类型：期刊论文

作者Yu Lijia; Gao Xiao-Shan1
刊名JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY
出版日期2022-09-15
页码26
关键词Deep neural network global robustness measure L-2,L-infinity normalization over-fitting Rademacher complexity smooth DNN
ISSN号1009-6124
DOI10.1007/s11424-022-1326-y
英文摘要In this paper, the L-2,L-infinity normalization of the weight matrices is used to enhance the robustness and accuracy of the deep neural network (DNN) with Relu as activation functions. It is shown that the L-2,L-infinity normalization leads to large dihedral angles between two adjacent faces of the DNN function graph and hence smoother DNN functions, which reduces over-fitting of the DNN. A global measure is proposed for the robustness of a classification DNN, which is the average radius of the maximal robust spheres with the training samples as centers. A lower bound for the robustness measure in terms of the L-2,L-infinity norm is given. Finally, an upper bound for the Rademacher complexity of DNNs with L-2,L-infinity normalization is given. An algorithm is given to train DNNs with the L-2,L-infinity normalization and numerical experimental results are used to show that the L-2,L-infinity normalization is effective in terms of improving the robustness and accuracy.
资助项目NKRDP[2018YFA0704705] ; National Natural Science Foundation of China[12288201]
WOS研究方向Mathematics
语种英语
WOS记录号WOS:000854858300001
出版者SPRINGER HEIDELBERG
源URL[http://ir.amss.ac.cn/handle/2S8OKBNM/61004]  
专题中国科学院数学与系统科学研究院
通讯作者Gao Xiao-Shan
作者单位1.Chinese Acad Sci, Acad Math & Syst Sci, Beijing 100190, Peoples R China
2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
推荐引用方式
GB/T 7714		 Yu Lijia,Gao Xiao-Shan. Improve Robustness and Accuracy of Deep Neural Network with L-2,L-infinity Normalization[J]. JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY,2022:26.
APA Yu Lijia,&Gao Xiao-Shan.(2022).Improve Robustness and Accuracy of Deep Neural Network with L-2,L-infinity Normalization.JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY,26.
MLA Yu Lijia,et al."Improve Robustness and Accuracy of Deep Neural Network with L-2,L-infinity Normalization".JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY (2022):26.

入库方式： OAI收割

来源：数学与系统科学研究院

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。