中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
EALI: Energy-aware layer-level scheduling for convolutional neural network inference services on GPUs

文献类型：期刊论文

作者Yao, Chunrong3; Liu, Wantao4; Liu, Zhibing4; Yan, Longchuan1; Hu, Songlin4; Tang, Weiqing2,3
刊名NEUROCOMPUTING
出版日期2022-10-01
卷号507页码:265-281
ISSN号0925-2312
关键词Scheduling Convolutional neural networks (CNNs) GPUs Service-level-objective (SLO) Energy minimization Inference services
DOI10.1016/j.neucom.2022.08.025
英文摘要The success of convolutional neural networks (CNNs) has made low-latency inference services on Graphic Processing Units (GPUs) a hot research topic. However, GPUs are hardware processors with high power consumption. To have the least energy consumption while meeting latency Service-Level-Objective (SLO), batching strategy and dynamic voltage frequency scaling (DVFS) are two important solutions. However, existing studies do not coordinate them and regard CNN as a black box, which makes inference services less energy-efficient. In this paper, we propose EALI, an energy-aware layer-level adaptive scheduling framework that is comprised of a power prediction model, a layer combination strategy, and an energy-aware layer-level scheduler. The power prediction model uses classic machine learning techniques to predict fine-grained layer-level power consumption. The layer combination strategy com-bines multiple layers into optimization units to lower scheduling overhead and complexity. The energy -aware layer-level scheduler adaptively coordinates batching strategy and layer-level DVFS according to workloads to minimize the energy consumption while meeting SLO. Our experimental results on NVIDIA Tesla M40 and V100 GPUs show that, compared to the state-of-the-art approaches, EALI decreases energy consumption by up to 36.24% while meeting SLO. (c) 2022 Elsevier B.V. All rights reserved.
资助项目State Grid Information and Telecommunication Branch[52993920002M]
WOS研究方向Computer Science
语种英语
出版者ELSEVIER
WOS记录号WOS:000843489800007
源URL[http://119.78.100.204/handle/2XEOYT63/19462]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Liu, Wantao
作者单位1.State Grid Informat & Telecommun Branch, Beijing 100761, Peoples R China
2.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China
3.Nanjing Univ Sci & Technol, Sch Comp Sci & Engn, Nanjing 210094, Peoples R China
4.Chinese Acad Sci, Inst Informat Engn, Beijing 100093, Peoples R China
推荐引用方式
GB/T 7714		 Yao, Chunrong,Liu, Wantao,Liu, Zhibing,et al. EALI: Energy-aware layer-level scheduling for convolutional neural network inference services on GPUs[J]. NEUROCOMPUTING,2022,507:265-281.
APA Yao, Chunrong,Liu, Wantao,Liu, Zhibing,Yan, Longchuan,Hu, Songlin,&Tang, Weiqing.(2022).EALI: Energy-aware layer-level scheduling for convolutional neural network inference services on GPUs.NEUROCOMPUTING,507,265-281.
MLA Yao, Chunrong,et al."EALI: Energy-aware layer-level scheduling for convolutional neural network inference services on GPUs".NEUROCOMPUTING 507(2022):265-281.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。