中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
热门
Addressing Sparsity in Deep Neural Networks

文献类型：期刊论文

作者Zhou, Xuda1,6,7; Du, Zidong6,7; Zhang, Shijin6,7; Zhang, Lei4,5,6,7; Lan, Huiying4,5,6,7; Liu, Shaoli6,7; Li, Ling3; Guo, Qi6,7; Chen, Tianshi6,7; Chen, Yunji2,7
刊名IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS
出版日期2019-10-01
卷号38期号:10页码:1858-1871
ISSN号0278-0070
关键词Accelerator architecture deep neural networks (DNNs) sparsity
DOI10.1109/TCAD.2018.2864289
英文摘要Neural networks (NNs) have been demonstrated to be useful in a broad range of applications, such as image recognition, automatic translation, and advertisement recommendation. State-of-the-art NNs are known to be both computationally and memory intensive, due to the ever-increasing deep structure, i.e., multiple layers with massive neurons and connections (i.e., synapses). Sparse NNs have emerged as an effective solution to reduce the amount of computation and memory required. Though existing NN accelerators are able to efficiently process dense and regular networks, they cannot benefit from the reduction of synaptic weights. In this paper, we propose a novel accelerator, Cambricon-X, to exploit the sparsity and irregularity of NN models for increased efficiency. The proposed accelerator features a processing element (PE)-based architecture consisting of multiple PEs. An indexing module efficiently selects and transfers needed neurons to connected PEs with reduced bandwidth requirement, while each PE stores irregular and compressed synapses for local computation in an asynchronous fashion. With 16 PEs, our accelerator is able to achieve at most 544 GOP/s in a small form factor (6.38 mm(2) and 954 mW at 65 nm). Experimental results over a number of representative sparse networks show that our accelerator achieves, on average, 7.23x speedup and 6.43x energy saving against the state-of-the-art NN accelerator. We further investigate possibilities of leveraging activation sparsity and multi-issue controller, which improve the efficiency of Cambricon-X. To ease the burden of programmers, we also propose a high efficient library-based programming environment for our accelerator.
资助项目National Key Research and Development Program of China[2017YFA0700902] ; National Key Research and Development Program of China[2017YFA0700900] ; National Key Research and Development Program of China[2017YFB1003101] ; NSF of China[61472396] ; NSF of China[61432016] ; NSF of China[61473275] ; NSF of China[61522211] ; NSF of China[61532016] ; NSF of China[61521092] ; NSF of China[61502446] ; NSF of China[61672491] ; NSF of China[61602441] ; NSF of China[61602446] ; NSF of China[61732002] ; NSF of China[61702478] ; 973 Program of China[2015CB358800] ; National Science and Technology Major Project[2018ZX01031102] ; Strategic Priority Research Program of Chinese Academy of Sciences[XDBS01050200]
WOS研究方向Computer Science ; Engineering
语种英语
出版者IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号WOS:000487193400007
源URL[http://119.78.100.204/handle/2XEOYT63/4675]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Chen, Yunji
作者单位1.Univ Sci & Technol China, Sch Comp Sci & Technol, Hefei 230000, Anhui, Peoples R China
2.Chinese Acad Sci, CAS Ctr Excellence Brain Sci, Beijing 100190, Peoples R China
3.Chinese Acad Sci, Inst Software, Beijing 100190, Peoples R China
4.Univ Chinese Acad Sci, Intelligence Processor Res Ctr, Beijing 100190, Peoples R China
5.Univ Chinese Acad Sci, State Key Lab Comp Architecture, Beijing 100190, Peoples R China
6.Cambricon Tech Ltd, Beijing 100191, Peoples R China
7.Chinese Acad Sci, Intelligence Processor Res Ctr, Inst Comp Technol, Beijing 100190, Peoples R China
推荐引用方式
GB/T 7714		 Zhou, Xuda,Du, Zidong,Zhang, Shijin,et al. Addressing Sparsity in Deep Neural Networks[J]. IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS,2019,38(10):1858-1871.
APA Zhou, Xuda.,Du, Zidong.,Zhang, Shijin.,Zhang, Lei.,Lan, Huiying.,...&Chen, Yunji.(2019).Addressing Sparsity in Deep Neural Networks.IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS,38(10),1858-1871.
MLA Zhou, Xuda,et al."Addressing Sparsity in Deep Neural Networks".IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS 38.10(2019):1858-1871.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。