Evaluating and analyzing the energy efficiency of CNN inference on high-performance GPU
文献类型：期刊论文
| 作者 | Yao, Chunrong1; Liu, Wantao2; Tang, Weiqing1,3; Guo, Jinrong2; Hu, Songlin2; Lu, Yijun4; Jiang, Wei5 |
| 刊名 | CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE
 |
| 出版日期 | 2020-10-21 |
| 页码 | 26 |
| 关键词 | CNNs energy efficiency high-performance GPU inference |
| ISSN号 | 1532-0626 |
| DOI | 10.1002/cpe.6064 |
| 英文摘要 | Convolutional neural network (CNN) inference usually runs on high-performance graphic processing units (GPUs). Since GPU is a high power consumption unit, that makes the energy consumption increases sharply due to the deep learning tasks. The energy efficiency of CNN inference is not only related to the software and hardware configurations, but also closely related to the application requirements of inference tasks. However, it is not clear on GPUs at present. In this paper, we conduct a comprehensive study on the model-level and layer-level energy efficiency of popular CNN models. The results point out several opportunities for further optimization. We also analyze the parameter settings (i.e., batch size, dynamic voltage and frequency scaling) and propose a revenue model to allow an optimal trade-off between energy efficiency and latency. Compared with the default settings, the optimal settings can improve revenue by up to 15.31x. We obtain the following main findings: (i) GPUs do not exploit the parallelism from the model depth and small convolution kernels, resulting in low energy efficiency. (ii) Convolutional layers are the most energy-consuming CNN layers. However, due to the cache, the power consumption of all layers is relatively balanced. (iii) The energy efficiency of TensorRT is 1.53xthan that of TensorFlow. |
| 资助项目 | National Key Research and Development Program of China[2017YFB1010000] |
| WOS研究方向 | Computer Science |
| 语种 | 英语 |
| WOS记录号 | WOS:000580529000001 |
| 出版者 | WILEY |
| 源URL | [http://119.78.100.204/handle/2XEOYT63/15725]  |
| 专题 | 中国科学院计算技术研究所期刊论文_英文 |
| 通讯作者 | Liu, Wantao |
| 作者单位 | 1.Nanjing Univ Sci & Technol, Sch Comp Sci & Engn, Nanjing, Peoples R China 2.Chinese Acad Sci, Inst Informat Engn, Beijing 100093, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China 4.Alibaba Cloud Comp Co Ltd, Hangzhou, Peoples R China 5.State Grid Corp China, Dept Energy Internet, Beijing, Peoples R China |
| 推荐引用方式 GB/T 7714 | Yao, Chunrong,Liu, Wantao,Tang, Weiqing,et al. Evaluating and analyzing the energy efficiency of CNN inference on high-performance GPU[J]. CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE,2020:26. |
| APA | Yao, Chunrong.,Liu, Wantao.,Tang, Weiqing.,Guo, Jinrong.,Hu, Songlin.,...&Jiang, Wei.(2020).Evaluating and analyzing the energy efficiency of CNN inference on high-performance GPU.CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE,26. |
| MLA | Yao, Chunrong,et al."Evaluating and analyzing the energy efficiency of CNN inference on high-performance GPU".CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE (2020):26. |
入库方式： OAI收割
来源：计算技术研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。