Performance Modeling for Spark Using SVM
文献类型：会议论文
| 作者 | Ni Luo; Zhibin Yu; Zhendong Bei; Chengzhong Xu; Chuntao Jiang; Lingfeng Lin |
| 出版日期 | 2016 |
| 会议名称 | CCBD 2016 |
| 会议地点 | 澳门 |
| 英文摘要 | Abstract—At present, Spark is widely used in a number of enterprises. Although Spark is much faster than Hadoop for some applications, its configuration parameters can have a great impact on its performance due to the large number of the parameters, interaction between them, and various characteristics of applications as well. Unfortunately, there is not yet any research conducted to predict the performance of Spark based on itsconfiguration sets. In this paper, we employ a machine learning method- Support Vector Machine(SVM) to build performance models for Spark. The input of configuration sets is collected by running Spark application previously with randomly modified and combined parameter values. In this way, we also determine the range of each property and gain a deeper understanding about how theseproperties work in Spark. We also use Artificial Neural Network to model the performance of Spark and find that the error rate of ANN is on average 1.98 times that of SVM for three workloads from HiBench. |
| 收录类别 | EI |
| 语种 | 英语 |
| 源URL | [http://ir.siat.ac.cn:8080/handle/172644/10338]  |
| 专题 | 深圳先进技术研究院_数字所 |
| 作者单位 | 2016 |
| 推荐引用方式 GB/T 7714 | Ni Luo,Zhibin Yu,Zhendong Bei,et al. Performance Modeling for Spark Using SVM[C]. 见:CCBD 2016. 澳门. |
入库方式： OAI收割
来源：深圳先进技术研究院
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。