HScheduler: an optimal approach to minimize the makespan of multiple MapReduce jobs
文献类型：期刊论文
| 作者 | Tian, Wenhong1,3,4 ; Li, Guozhong1; Yang, Wutong1; Buyya, Rajkumar2
|
| 刊名 | JOURNAL OF SUPERCOMPUTING
 |
| 出版日期 | 2016-06-01 |
| 卷号 | 72期号:6页码:2376-2393 |
| 关键词 | Hadoop MapReduce Batch workloads Optimized schedule Minimized makespan |
| ISSN号 | 0920-8542 |
| DOI | 10.1007/s11227-016-1737-4 |
| 通讯作者 | Tian, WH (reprint author), Univ Elect Sci & Technol China, Sch Informat & Software Engn, Chengdu 610054, Peoples R China. ; Tian, WH (reprint author), Univ Elect Sci & Technol China, Big Data Res Ctr, Chengdu 610054, Peoples R China. ; Tian, WH (reprint author), Chinese Acad Sci, Chongqing Inst Green & Intelligent Technol, Chongqing 400714, Peoples R China. |
| 英文摘要 | Large-scale MapReduce clusters that routinely process big data bring challenges to the cloud computing. One of the key challenges is to reduce the response time of these MapReduce clusters by minimizing their makespans. It is observed that the order in which these jobs are executed can have a significant impact on their overall makespans and resource utilization. In this work, we consider a scheduling model for multiple MapReduce jobs. The goal is to design a job scheduler that minimizes the makespan of such a set of MapReduce jobs. We exploit classical Johnson model and propose a novel framework HScheduler, which combines features of both classical Johnson's algorithm and MapReduce to minimize the makespan for both offline and online jobs. Our Offline HScheduler reaches the theoretical lower bound (optimum) and Online HScheduler is 2-competitive which is the best-known constant ratio for minimizing the makespan. Through extensive real data tests, we find that HScheduler has better performance than the best-known approach by 10.6-11.7 % on average for offline scheduling and 8-10 % on average for online scheduling. The HScheduler can be applied to improve responsive time, throughput and energy efficiency in cloud computing. |
| 资助项目 | China National Science Foundation (CNSF)[61450110440] ; Sichuan Province Technology Plan[2016GZ0322] ; Chongqing Research Program of Basic Research and Frontier Technology[cstc2015jcyjB0244] |
| WOS研究方向 | Computer Science ; Engineering |
| 语种 | 英语 |
| WOS记录号 | WOS:000376650000015 |
| 出版者 | SPRINGER |
| 源URL | [http://119.78.100.138/handle/2HOD01W0/2530]  |
| 专题 | 大数据挖掘及应用中心 |
| 通讯作者 | Tian, Wenhong |
| 作者单位 | 1.Univ Elect Sci & Technol China, Sch Informat & Software Engn, Chengdu 610054, Peoples R China 2.Univ Melbourne, Parkville, Vic 3052, Australia 3.Univ Elect Sci & Technol China, Big Data Res Ctr, Chengdu 610054, Peoples R China 4.Chinese Acad Sci, Chongqing Inst Green & Intelligent Technol, Chongqing 400714, Peoples R China |
| 推荐引用方式 GB/T 7714 | Tian, Wenhong,Li, Guozhong,Yang, Wutong,et al. HScheduler: an optimal approach to minimize the makespan of multiple MapReduce jobs[J]. JOURNAL OF SUPERCOMPUTING,2016,72(6):2376-2393. |
| APA | Tian, Wenhong,Li, Guozhong,Yang, Wutong,&Buyya, Rajkumar.(2016).HScheduler: an optimal approach to minimize the makespan of multiple MapReduce jobs.JOURNAL OF SUPERCOMPUTING,72(6),2376-2393. |
| MLA | Tian, Wenhong,et al."HScheduler: an optimal approach to minimize the makespan of multiple MapReduce jobs".JOURNAL OF SUPERCOMPUTING 72.6(2016):2376-2393. |
入库方式： OAI收割
来源：重庆绿色智能技术研究院
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。