中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
Comparative Analysis of Energy-Efficient Scheduling Algorithms for Big Data Applications

文献类型:期刊论文

作者Li, Hongjian1,2; Wang, Huochen1; Xiong, Anping1; Lai, Jun1; Tian, Wenhong2,3
刊名IEEE ACCESS
出版日期2018
卷号6页码:40073-40084
关键词Big Data Deadline-constrained Energy-efficient Spark Application Tasks Scheduling Algorithm
ISSN号2169-3536
DOI10.1109/ACCESS.2018.2855720
英文摘要

Nowadays, big data analytics has been widely applied in addressing the growing cybercrime threats. However, energy consumption is explosive increasing with the fast growth of big data processing in anti-cybercrime. In this paper, an energy-efficient framework for big data applications is proposed to reduce energy consumption while satisfying deadline constrains. First, the problem of energy-efficient tasks scheduling of a single Spark job is modeled as an integer program. We design an energy-efficient tasks scheduling algorithm to minimize the energy consumption for big data application in Spark. To avoid service-level agreement violations for execution time, we propose an optimal task scheduling algorithm with deadline constrains by tradingoff execution time and energy consumption. Experiments on a Spark cluster are performed to determine the energy consumption and execution time for several workloads from the HiBench benchmark suite. Our algorithms consume less energy on average than FIFO and FAIR under deadlines. The optimal algorithm is able to find near optimal tasks schedules to trade off energy consumed and response time benefit in small shuffle partitions.

资助项目National Natural Science Foundation of China[61672136] ; National Natural Science Foundation of China[6167060383] ; National Natural Science Foundation of China[61650110513] ; National Natural Science Foundation of China[61672004] ; China Postdoctoral Science[2016M600733] ; Chongqing Science and Technology Commission Project[cstc2017jcyjAX0142] ; Chongqing Science and Technology Commission Project[cstc2018jcyjAX0525] ; Foundation of CQUPT in China[A2013-21] ; Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences[R51A150Z10] ; Chongqing Engineering Technology Research Center of the Mobile Internet Data Application ; Program for Innovation Team Building at Institutions of Higher Education in Chongqing
WOS研究方向Computer Science ; Engineering ; Telecommunications
语种英语
WOS记录号WOS:000440895400001
出版者IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
源URL[http://119.78.100.138/handle/2HOD01W0/6549]  
专题大数据挖掘及应用中心
作者单位1.Chongqing Univ Posts & Telecommun, Dept Comp Sci & Technol, Chongqing 400065, Peoples R China
2.Univ Elect Sci & Technol China, Dept Informat & Software Engn, Chengdu 610054, Sichuan, Peoples R China
3.Chinese Acad Sci, Chongqing Inst Green & Intelligent Technol, Chongqing 401122, Peoples R China
推荐引用方式
GB/T 7714
Li, Hongjian,Wang, Huochen,Xiong, Anping,et al. Comparative Analysis of Energy-Efficient Scheduling Algorithms for Big Data Applications[J]. IEEE ACCESS,2018,6:40073-40084.
APA Li, Hongjian,Wang, Huochen,Xiong, Anping,Lai, Jun,&Tian, Wenhong.(2018).Comparative Analysis of Energy-Efficient Scheduling Algorithms for Big Data Applications.IEEE ACCESS,6,40073-40084.
MLA Li, Hongjian,et al."Comparative Analysis of Energy-Efficient Scheduling Algorithms for Big Data Applications".IEEE ACCESS 6(2018):40073-40084.

入库方式: OAI收割

来源:重庆绿色智能技术研究院

浏览0
下载0
收藏0
其他版本

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。