Hybridmr: a new approach for hybrid mapreduce combining desktop grid and cloud infrastructures
文献类型：期刊论文
| 作者 | Tang, Bing1; He, Haiwu2; Fedak, Gilles3 |
| 刊名 | Concurrency and computation-practice & experience
 |
| 出版日期 | 2015-11-01 |
| 卷号 | 27期号:16页码:4140-4155 |
| 关键词 | Hybrid computing environment Volunteer computing Mapreduce Distributed file system Fault tolerance |
| ISSN号 | 1532-0626 |
| DOI | 10.1002/cpe.3515 |
| 通讯作者 | Tang, bing(btang@hnust.edu.cn) |
| 英文摘要 | This paper introduces hybridmr, a novel model for the execution of mapreduce (mr) computation on hybrid computing environment. using this model, high performance cloud resources and heterogeneous desktop personal computers (pcs) in internet or intranet can be integrated to form a hybrid computing environment. thanks to hybridmr, the computation and storage capability of large scale desktop pcs can be fully utilized to process large scale datasets. hybridmr relies on two innovative solutions to enable such large scale data-intensive computation. the first one is hybriddfs, which is a hybrid distributed file system. hybriddfs features reliable distributed storage that alleviates the volatility of desktop pcs, thanks to fault tolerance and file replication mechanism. the second innovation is a new node priority-based fair scheduling (npbfs) algorithm has been developed in hybridmr to achieve both data storage balance and job assignment balance by assigning each node a priority through quantifying cpu speed, memory size, and input and output capacity. in this paper, we describe the hybridmr, hybriddfs, and npbfs. we report on performance evaluation results, which show that the proposed hybridmr not only achieves reliable mr computation, reduces task response time, and improves the performance of mr, but also reduces the computation cost and achieves a greener computing mode. copyright (c) 2015john wiley & sons, ltd. |
| WOS关键词 | LARGE-SCALE ; SYSTEMS |
| WOS研究方向 | Computer Science |
| WOS类目 | Computer Science, Software Engineering ; Computer Science, Theory & Methods |
| 语种 | 英语 |
| WOS记录号 | WOS:000363470800007 |
| 出版者 | WILEY-BLACKWELL |
| URI标识 | http://www.irgrid.ac.cn/handle/1471x/2374163 |
| 专题 | 计算机网络信息中心 |
| 通讯作者 | Tang, Bing |
| 作者单位 | 1.Hunan Univ Sci & Technol, Sch Comp Sci & Engn, Xiangtan 411201, Peoples R China 2.Chinese Acad Sci, Comp Network Informat Ctr, Beijing 100864, Peoples R China 3.Univ Lyon, LIP Lab, F-69364 Lyon, France |
| 推荐引用方式 GB/T 7714 | Tang, Bing,He, Haiwu,Fedak, Gilles. Hybridmr: a new approach for hybrid mapreduce combining desktop grid and cloud infrastructures[J]. Concurrency and computation-practice & experience,2015,27(16):4140-4155. |
| APA | Tang, Bing,He, Haiwu,&Fedak, Gilles.(2015).Hybridmr: a new approach for hybrid mapreduce combining desktop grid and cloud infrastructures.Concurrency and computation-practice & experience,27(16),4140-4155. |
| MLA | Tang, Bing,et al."Hybridmr: a new approach for hybrid mapreduce combining desktop grid and cloud infrastructures".Concurrency and computation-practice & experience 27.16(2015):4140-4155. |
入库方式： iSwitch采集
来源：计算机网络信息中心
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。