中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
热门
A deep neural network compression algorithm based on knowledge transfer for edge devices

文献类型：期刊论文

作者Chen, Yanming1; Li, Chao2; Gong, Luqi2; Wen, Xiang1; Zhang, Yiwen1; Shi, Weisong3
刊名COMPUTER COMMUNICATIONS
出版日期2020-11-01
卷号163页码:186-194
ISSN号0140-3664
关键词Edge device Deep learning Neural network compression Knowledge transfer
DOI10.1016/j.comcom.2020.09.016
英文摘要The computation and storage capacity of the edge device are limited, which seriously restrict the application of deep neural network in the device. Toward to the intelligent application of the edge device, we introduce the deep neural network compression algorithm based on knowledge transfer, a three-stage pipeline: lightweight, multi-level knowledge transfer and pruning that reduce the network depth, parameter and operation complexity of the deep learning neural networks. We lighten the neural networks by using a global average pooling layer instead of a fully connected layer and replacing a standard convolution with separable convolutions. Next, the multi-level knowledge transfer minimizes the difference between the output of the "student network" and the "teacher network" in the middle and logits layer, increasing the supervised information when training the "student network". Lastly, we prune the network by cutting off the unimportant convolution kernels with a global iterative pruning strategy. The experiment results show that the proposed method improve the efficiency up to 30% than the knowledge distillation method in reducing the loss of classification performance. Benchmarked on GPU (Graphics Processing Unit) server, Raspberry Pi 3 and Cambricon-1A, the parameters of the compressed network after using our knowledge transfer and pruning method have achieved more than 49.5 times compression and the time efficiency of a single feedforward operation has been improved more than 3.2 times.
资助项目National Natural Science Foundation of China[61702487] ; National Natural Science Foundation of China[61802001]
WOS研究方向Computer Science ; Engineering ; Telecommunications
语种英语
出版者ELSEVIER
WOS记录号WOS:000579768300013
源URL[http://119.78.100.204/handle/2XEOYT63/15692]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Li, Chao
作者单位1.Anhui Univ, Sch Comp Sci, Hefei, Peoples R China
2.Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China
3.Wayne State Univ, Dept Comp Sci, Detroit, MI 48202 USA
推荐引用方式
GB/T 7714		 Chen, Yanming,Li, Chao,Gong, Luqi,et al. A deep neural network compression algorithm based on knowledge transfer for edge devices[J]. COMPUTER COMMUNICATIONS,2020,163:186-194.
APA Chen, Yanming,Li, Chao,Gong, Luqi,Wen, Xiang,Zhang, Yiwen,&Shi, Weisong.(2020).A deep neural network compression algorithm based on knowledge transfer for edge devices.COMPUTER COMMUNICATIONS,163,186-194.
MLA Chen, Yanming,et al."A deep neural network compression algorithm based on knowledge transfer for edge devices".COMPUTER COMMUNICATIONS 163(2020):186-194.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。