中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
FullLoRA: Efficiently Boosting the Robustness of Pretrained Vision Transformers

文献类型：期刊论文

作者Yuan, Zheng1,2; Zhang, Jie1,2; Shan, Shiguang1,2; Chen, Xilin1,2
刊名IEEE TRANSACTIONS ON IMAGE PROCESSING
出版日期2025
卷号34页码:4580-4590
关键词Training Computational modeling Robustness Adaptation models Computer vision Transformers Visualization Natural language processing Image classification Head Adversarial training parameter-efficient pretrained model
ISSN号1057-7149
DOI10.1109/TIP.2025.3587598
英文摘要In recent years, the Vision Transformer (ViT) model has gradually become mainstream in various computer vision tasks, and the robustness of the model has received increasing attention. However, existing large models tend to prioritize performance during training, potentially neglecting the robustness, which may lead to serious security concerns. In this paper, we establish a new challenge: exploring how to use a small number of additional parameters for adversarial finetuning to quickly and effectively enhance the adversarial robustness of a standardly trained model. To address this challenge, we develop novel LNLoRA module, incorporating a learnable layer normalization before the conventional LoRA module, which helps mitigate magnitude differences in parameters between the adversarial and standard training paradigms. Furthermore, we propose the FullLoRA framework by integrating the learnable LNLoRA modules into all key components of ViT-based models while keeping the pretrained model frozen, which can significantly improve the model robustness via adversarial finetuning in a parameter-efficient manner. Extensive experiments on several datasets demonstrate the superiority of our proposed FullLoRA framework. It achieves comparable robustness with full finetuning while only requiring about 5% of the learnable parameters. This also effectively addresses concerns regarding extra model storage space and enormous training time caused by adversarial finetuning.
资助项目Strategic Priority Research Program of Chinese Academy of Sciences[XDB0680202] ; Beijing Nova Program[20230484368] ; Suzhou Frontier Technology Research Project[SYG202325] ; Youth Innovation Promotion Association of Chinese Academy of Sciences (CAS)
WOS研究方向Computer Science ; Engineering
语种英语
WOS记录号WOS:001534512500004
出版者IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
源URL[http://119.78.100.204/handle/2XEOYT63/42044]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Zhang, Jie
作者单位1.Chinese Acad Sci, Inst Comp Technol, State Key Lab AI Safety, Beijing 100190, Peoples R China
2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
推荐引用方式
GB/T 7714		 Yuan, Zheng,Zhang, Jie,Shan, Shiguang,et al. FullLoRA: Efficiently Boosting the Robustness of Pretrained Vision Transformers[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2025,34:4580-4590.
APA Yuan, Zheng,Zhang, Jie,Shan, Shiguang,&Chen, Xilin.(2025).FullLoRA: Efficiently Boosting the Robustness of Pretrained Vision Transformers.IEEE TRANSACTIONS ON IMAGE PROCESSING,34,4580-4590.
MLA Yuan, Zheng,et al."FullLoRA: Efficiently Boosting the Robustness of Pretrained Vision Transformers".IEEE TRANSACTIONS ON IMAGE PROCESSING 34(2025):4580-4590.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。