MEST: A Model-Driven Efficient Searching Approach for MapReduce Self-Tuning
文献类型：期刊论文
| 作者 | Zhendong Bei; Zhibin Yu; Qixiao Liu; Chengzhong Xu; Shengzhong Feng; Shuang Song |
| 刊名 | IEEE Access
 |
| 出版日期 | 2017 |
| 文献子类 | 期刊论文 |
| 英文摘要 | Hadoop is the most popular implementation framework of the MapReduce programming model, and it has a number of performance-critical configuration parameters. However, manually setting these parameters to their optimal values not only needs in-depth knowledge on Hadoop as well as the job itself, but also requires a large amount of time and efforts. Automatic approaches have therefore been proposed. Their usage, however, is still quite limited due to the intolerably long searching time. In this paper, we introduce MapreducE Self-Tuning (MEST), a framework that accelerates the searching process for the optimal configuration of a given Hadoop application. We have devised a novel mechanism by integrating the model trees algorithm with the genetic algorithm. As such, MEST significantly reduces the searching time by removing unnecessary profiling, modeling, and searching steps, which are mandatory for existing approaches. Our experiments using five benchmarks, each with two input data sets (DS1 and 2× DS1 ) show that MEST improves the searching efficiency (SE) by factors of 1.37× and 2.18× on average respectively over the state-of-the-art approach. |
| URL标识 | 查看原文 |
| 语种 | 英语 |
| 源URL | [http://ir.siat.ac.cn:8080/handle/172644/12524]  |
| 专题 | 深圳先进技术研究院_数字所 |
| 作者单位 | IEEE Access |
| 推荐引用方式 GB/T 7714 | Zhendong Bei,Zhibin Yu,Qixiao Liu,et al. MEST: A Model-Driven Efficient Searching Approach for MapReduce Self-Tuning[J]. IEEE Access,2017. |
| APA | Zhendong Bei,Zhibin Yu,Qixiao Liu,Chengzhong Xu,Shengzhong Feng,&Shuang Song.(2017).MEST: A Model-Driven Efficient Searching Approach for MapReduce Self-Tuning.IEEE Access. |
| MLA | Zhendong Bei,et al."MEST: A Model-Driven Efficient Searching Approach for MapReduce Self-Tuning".IEEE Access (2017). |
入库方式： OAI收割
来源：深圳先进技术研究院
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。