中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
Fpar: filter pruning via attention and rank enhancement for deep convolutional neural networks acceleration

文献类型：期刊论文

作者Chen, Yanming2; Wu, Gang2; Shuai, Mingrui2; Lou, Shubin2; Zhang, Yiwen2; An, Zhulin1
刊名INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS
出版日期2024-01-29
页码13
关键词Neural network Model compression Filter pruning Attention Rank enhancement CNNs
ISSN号1868-8071
DOI10.1007/s13042-023-02076-1
英文摘要Pruning deep neural networks is crucial for enabling their deployment on resource-constrained edge devices, where the vast number of parameters and computational requirements pose significant challenges. However, many of these methods consider only the importance of a single filter to the network and neglect the correlation between filters. To solve this problem, we propose a novel filter pruning method, called Filter Pruning via Attention and Rank Enhancement for Deep Convolutional Neural Networks Acceleration (FPAR), based on the attention mechanism and rank of feature maps. Moreover, the inspiration for it comes from a discovery: for a network with attention modules, irrespective of the batch of input images, the mean of channel-wise weights of the attention module is almost constant. Thus, we can use a few batches of input data to obtain this indicator to guide pruning. A large number of experiments have proved that our method outperforms the most advanced methods with similar accuracy. For example, using VGG-16, our method removes 62.8% of floating-point operations (FLOPs) even with a 0.24% of the accuracy increase on CIFAR-10. With ResNet-110, our FPAR method can reduce FLOPs by 61.7% by removing 62.7% of the parameters, with slight improvement of 0.05% in the top 1 accuracy on CIFAR-10.
资助项目National Science Foundation of China (NSFC)[62262067] ; Key Natural Science Foundation of Education Department of Anhui[KJ2021A0046]
WOS研究方向Computer Science
语种英语
WOS记录号WOS:001150668700003
出版者SPRINGER HEIDELBERG
源URL[http://119.78.100.204/handle/2XEOYT63/38375]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者An, Zhulin
作者单位1.Chinese Acad Sci, Inst Comp Technol, Beijing 100000, Peoples R China
2.Anhui Univ, Sch Compute Sci & Technol, Hefei 230000, Anhui, Peoples R China
推荐引用方式
GB/T 7714		 Chen, Yanming,Wu, Gang,Shuai, Mingrui,et al. Fpar: filter pruning via attention and rank enhancement for deep convolutional neural networks acceleration[J]. INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS,2024:13.
APA Chen, Yanming,Wu, Gang,Shuai, Mingrui,Lou, Shubin,Zhang, Yiwen,&An, Zhulin.(2024).Fpar: filter pruning via attention and rank enhancement for deep convolutional neural networks acceleration.INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS,13.
MLA Chen, Yanming,et al."Fpar: filter pruning via attention and rank enhancement for deep convolutional neural networks acceleration".INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS (2024):13.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。