Comparative Analysis of Energy-Efficient Scheduling Algorithms for Big Data Applications
文献类型:期刊论文
作者 | Li, Hongjian1,2; Wang, Huochen1; Xiong, Anping1; Lai, Jun1; Tian, Wenhong2,3![]() |
刊名 | IEEE ACCESS
![]() |
出版日期 | 2018 |
卷号 | 6页码:40073-40084 |
关键词 | Big Data Deadline-constrained Energy-efficient Spark Application Tasks Scheduling Algorithm |
ISSN号 | 2169-3536 |
DOI | 10.1109/ACCESS.2018.2855720 |
英文摘要 | Nowadays, big data analytics has been widely applied in addressing the growing cybercrime threats. However, energy consumption is explosive increasing with the fast growth of big data processing in anti-cybercrime. In this paper, an energy-efficient framework for big data applications is proposed to reduce energy consumption while satisfying deadline constrains. First, the problem of energy-efficient tasks scheduling of a single Spark job is modeled as an integer program. We design an energy-efficient tasks scheduling algorithm to minimize the energy consumption for big data application in Spark. To avoid service-level agreement violations for execution time, we propose an optimal task scheduling algorithm with deadline constrains by tradingoff execution time and energy consumption. Experiments on a Spark cluster are performed to determine the energy consumption and execution time for several workloads from the HiBench benchmark suite. Our algorithms consume less energy on average than FIFO and FAIR under deadlines. The optimal algorithm is able to find near optimal tasks schedules to trade off energy consumed and response time benefit in small shuffle partitions. |
资助项目 | National Natural Science Foundation of China[61672136] ; National Natural Science Foundation of China[6167060383] ; National Natural Science Foundation of China[61650110513] ; National Natural Science Foundation of China[61672004] ; China Postdoctoral Science[2016M600733] ; Chongqing Science and Technology Commission Project[cstc2017jcyjAX0142] ; Chongqing Science and Technology Commission Project[cstc2018jcyjAX0525] ; Foundation of CQUPT in China[A2013-21] ; Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences[R51A150Z10] ; Chongqing Engineering Technology Research Center of the Mobile Internet Data Application ; Program for Innovation Team Building at Institutions of Higher Education in Chongqing |
WOS研究方向 | Computer Science ; Engineering ; Telecommunications |
语种 | 英语 |
WOS记录号 | WOS:000440895400001 |
出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
源URL | [http://119.78.100.138/handle/2HOD01W0/6549] ![]() |
专题 | 大数据挖掘及应用中心 |
作者单位 | 1.Chongqing Univ Posts & Telecommun, Dept Comp Sci & Technol, Chongqing 400065, Peoples R China 2.Univ Elect Sci & Technol China, Dept Informat & Software Engn, Chengdu 610054, Sichuan, Peoples R China 3.Chinese Acad Sci, Chongqing Inst Green & Intelligent Technol, Chongqing 401122, Peoples R China |
推荐引用方式 GB/T 7714 | Li, Hongjian,Wang, Huochen,Xiong, Anping,et al. Comparative Analysis of Energy-Efficient Scheduling Algorithms for Big Data Applications[J]. IEEE ACCESS,2018,6:40073-40084. |
APA | Li, Hongjian,Wang, Huochen,Xiong, Anping,Lai, Jun,&Tian, Wenhong.(2018).Comparative Analysis of Energy-Efficient Scheduling Algorithms for Big Data Applications.IEEE ACCESS,6,40073-40084. |
MLA | Li, Hongjian,et al."Comparative Analysis of Energy-Efficient Scheduling Algorithms for Big Data Applications".IEEE ACCESS 6(2018):40073-40084. |
入库方式: OAI收割
来源:重庆绿色智能技术研究院
其他版本
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。