中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
DPQ: dynamic pseudo-mean mixed-precision quantization for pruned neural network

文献类型：期刊论文

作者Pei, Songwen1,2,3; Wang, Jiyao3; Zhang, Bingxue3; Qin, Wei3; Xue, Hai3; Ye, Xiaochun2; Chen, Mingsong1
刊名MACHINE LEARNING
出版日期2024-01-31
页码14
关键词Big data Compression Deep learning Pseudo-mean mixed-precision quantization Pruned neural network
ISSN号0885-6125
DOI10.1007/s10994-023-06453-3
英文摘要The ever-increasing layers and hyper-parameters of deep neural network are continuously growing to generate large-scale network by training huge masses of data. However, it is difficult to deploy deep neural network on resource-constrained edge devices. Network mixed-precision quantization is a challenging way to prune and compress deep neural network models while discovering the optimal bit width for each layer. To solve the big challenge, we therefore propose the dynamic pseudo-mean mixed-precision quantization (DPQ) by introducing two-bit scaling factors to compensate errors of quantization. Furthermore, the activation quantization named random parameters clipping (RPC) is proposed. RPC adopts partial activation quantization to reduce loss of accuracy. Therefore, DPQ can dynamically adjust the bit precision of weight quantization according to the distribution of weights. It results in a quantification scheme with strong robustness compared to previous methods. Extensive experiments demonstrate that DPQ achieves 15.43x\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\times$$\end{document} compression rate of ResNet20 on CIFAR-10 dataset with 0.22% increase in accuracy, and 35.25x\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\times$$\end{document} compression rate of Resnet56 on SVHN dataset with 0.12% increase in accuracy.
资助项目National Natural Science Foundation of China[61975124] ; National Natural Science Foundation of China[20ZR1438500] ; Shanghai Natural Science Foundation[CARCHA202111] ; State Key Laboratory of Computer Architecture (ICT, CAS)[OP202202] ; Engineering Research Center of Software/Hardware Co-design Technology and Application, Ministry of Education, East China Normal University
WOS研究方向Computer Science
语种英语
WOS记录号WOS:001151730300001
出版者SPRINGER
源URL[http://119.78.100.204/handle/2XEOYT63/38401]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Pei, Songwen
作者单位1.East China Normal Univ, Software Hardware Codesign Technol & Applicat, Minist Educ, Engn Res Ctr, Shanghai 200062, Peoples R China
2.Chinese Acad Sci, Inst Comp Technol, State Key Lab Comp Architecture, Beijing 100190, Peoples R China
3.Univ Shanghai Sci & Technol, Sch Opt Elect & Comp Engn, Shanghai 200093, Peoples R China
推荐引用方式
GB/T 7714		 Pei, Songwen,Wang, Jiyao,Zhang, Bingxue,et al. DPQ: dynamic pseudo-mean mixed-precision quantization for pruned neural network[J]. MACHINE LEARNING,2024:14.
APA Pei, Songwen.,Wang, Jiyao.,Zhang, Bingxue.,Qin, Wei.,Xue, Hai.,...&Chen, Mingsong.(2024).DPQ: dynamic pseudo-mean mixed-precision quantization for pruned neural network.MACHINE LEARNING,14.
MLA Pei, Songwen,et al."DPQ: dynamic pseudo-mean mixed-precision quantization for pruned neural network".MACHINE LEARNING (2024):14.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。