中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking

文献类型：期刊论文

作者Liu, Chang2; Dong, Yinpeng1,5; Xiang, Wenzhao3,7; Yang, Xiao1; Su, Hang1,6; Zhu, Jun1,5; Chen, Yuefeng4; He, Yuan4; Xue, Hui4; Zheng, Shibao2
刊名INTERNATIONAL JOURNAL OF COMPUTER VISION
出版日期2024-08-09
页码23
关键词Robustness benchmark Distribution shift Pre-training Adversarial training Image classification
ISSN号0920-5691
DOI10.1007/s11263-024-02196-3
英文摘要The robustness of deep neural networks is frequently compromised when faced with adversarial examples, common corruptions, and distribution shifts, posing a significant research challenge in the advancement of deep learning. Although new deep learning methods and robustness improvement techniques have been constantly proposed, the robustness evaluations of existing methods are often inadequate due to their rapid development, diverse noise patterns, and simple evaluation metrics. Without thorough robustness evaluations, it is hard to understand the advances in the field and identify the effective methods. In this paper, we establish a comprehensive robustness benchmark called ARES-Bench on the image classification task. In our benchmark, we evaluate the robustness of 61 typical deep learning models on ImageNet with diverse architectures (e.g., CNNs, Transformers) and learning algorithms (e.g., normal supervised training, pre-training, adversarial training) under numerous adversarial attacks and out-of-distribution (OOD) datasets. Using robustness curves as the major evaluation criteria, we conduct large-scale experiments and draw several important findings, including: (1) there exists an intrinsic trade-off between the adversarial and natural robustness of specific noise types for the same model architecture; (2) adversarial training effectively improves adversarial robustness, especially when performed on Transformer architectures; (3) pre-training significantly enhances natural robustness by leveraging larger training datasets, incorporating multi-modal data, or employing self-supervised learning techniques. Based on ARES-Bench, we further analyze the training tricks in large-scale adversarial training on ImageNet. Through tailored training settings, we achieve a new state-of-the-art in adversarial robustness. We have made the benchmarking results and code platform publicly available.
资助项目National Natural Science Foundation of China[62076147] ; National Natural Science Foundation of China[92370124] ; National Natural Science Foundation of China[62276149] ; National Natural Science Foundation of China[92248303] ; National Natural Science Foundation of China[62071292] ; National Natural Science Foundation of China[U21B2013] ; National Natural Science Foundation of China[U2341228] ; Science and Technology Commission of Shanghai Municipality[22DZ2229005] ; Tsinghua-Alibaba Joint Research Program ; China National Postdoctoral Program for Innovative Talents ; CCF-BaiChuan-Ebtech Foundation Model Fund
WOS研究方向Computer Science
语种英语
WOS记录号WOS:001287481400002
出版者SPRINGER
源URL[http://119.78.100.204/handle/2XEOYT63/39661]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Su, Hang; Zheng, Shibao
作者单位1.Chinese Acad Sci, Key Lab Intelligent Informat Proc, Inst Comp Technol ICT, Beijing 100190, Peoples R China
2.Shanghai Jiao Tong Univ, Inst Image Commun & Networks Engn, Dept Elect Engn EE, Shanghai 200240, Peoples R China
3.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China
4.Alibaba Grp, Hangzhou 310023, Zhejiang, Peoples R China
5.RealAI, Beijing 100085, Peoples R China
6.Zhongguancun Lab, Beijing 100080, Peoples R China
7.Peng Cheng Lab, Shenzhen 518000, Peoples R China
推荐引用方式
GB/T 7714		 Liu, Chang,Dong, Yinpeng,Xiang, Wenzhao,et al. A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking[J]. INTERNATIONAL JOURNAL OF COMPUTER VISION,2024:23.
APA Liu, Chang.,Dong, Yinpeng.,Xiang, Wenzhao.,Yang, Xiao.,Su, Hang.,...&Zheng, Shibao.(2024).A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking.INTERNATIONAL JOURNAL OF COMPUTER VISION,23.
MLA Liu, Chang,et al."A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking".INTERNATIONAL JOURNAL OF COMPUTER VISION (2024):23.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。