Improving Data Locality of MapReduce by scheduling in homogeneous computing environments
文献类型：会议论文
| 作者 | Zhang, Xiaohong; Zhong, Zhiyong; Feng, Shengzhong; Tu, Bibo; Fan, Jianping |
| 出版日期 | 2011 |
| 会议名称 | 9th IEEE International Symposium on Parallel and Distributed Processing with Applications, ISPA 2011 |
| 会议地点 | Busan, Korea, Republic of |
| 英文摘要 | Data Locality is one of the critical factors to affect performance. This paper proposes a next-k-node scheduling (NKS) method to improve the data locality of map tasks. The method first calculates the probabilities of each map task, and then preferentially schedules the one with the highest probability. It generates low probabilities for the tasks which satisfy node locality with the nodes to issue requests, so it can reserve these tasks to these nodes. We have implemented the NKS method in hadoop-0.20.2. The experiment results have shown that the NKS method reduced 78% of the map tasks processed without node locality, reduced 77%of the network load caused by the tasks, and improved the performance of Hadoop MapReduce when comparing with the default task scheduling method in Hadoop. Obviously, the NKS method is very suitable for the homogeneous environment with network overload. |
| 收录类别 | EI |
| 语种 | 英语 |
| 源URL | [http://ir.siat.ac.cn:8080/handle/172644/3586]  |
| 专题 | 深圳先进技术研究院_数字所 |
| 作者单位 | 2011 |
| 推荐引用方式 GB/T 7714 | Zhang, Xiaohong,Zhong, Zhiyong,Feng, Shengzhong,et al. Improving Data Locality of MapReduce by scheduling in homogeneous computing environments[C]. 见:9th IEEE International Symposium on Parallel and Distributed Processing with Applications, ISPA 2011. Busan, Korea, Republic of. |
入库方式： OAI收割
来源：深圳先进技术研究院
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。