A distributed load balancing algorithm for climate big data processing over a multi-core CPU cluster
文献类型：期刊论文
| 作者 | Wang, Yuzhu1; Jiang, Jinrong1; Ye, Huang1; He, Juanxiong1 |
| 刊名 | CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE
 |
| 出版日期 | 2016 |
| 卷号 | 28期号:15页码:4144-4160 |
| 关键词 | TERRESTRIAL ECOSYSTEMS TROPOSPHERIC OZONE RANDOM FORESTS CLIMATE MODEL PRECIPITATION BIOSPHERE PATTERNS BIOMASS AFRICA |
| 通讯作者 | Wang, YZ (reprint author), Chinese Acad Sci, Inst Remote Sensing & Digital Earth, Beijing 100094, Peoples R China. |
| 英文摘要 | Load imbalance is a common problem to be tackled urgently in large scale data-driven simulation systems or data intensive computing. According to the coupler, the Chinese Academy of Sciences-Earth System Model (CAS-ESM) implements one-way nesting of the Institute of Atmospheric Physics of Chinese Academy of Sciences Atmospheric General Circulation Model version 4.0 (IAP AGCM4.0) and Weather Research and Forecasting model (WRF). The METGRID (meteorological grid) and REAL program modules in the WRF are used to process meteorological data. In the CAS-ESM, the load of the METGRID module is seriously unbalanced on many CPU cores. The load imbalance has a serious impact on the processing speed of meteorological data, so this study designs an optimization algorithm to solve the problem. Numerical experiments show that compared to before optimization, the optimization algorithm can solve the load imbalance of the METGRID, and the computation speed of the METGRID and REAL modules after optimization on 64 CPU cores is about 7.2 times faster than before. Meanwhile, the whole computation speed of the CAS-ESM can improve by 217.53%. In addition, results indicate that they also can reach to a similar speedup on different numbers of CPU cores. Copyright (C) 2016 John Wiley & Sons, Ltd. |
| 学科主题 | Computer Science |
| 类目[WOS] | Computer Science, Software Engineering ; Computer Science, Theory & Methods |
| 收录类别 | SCI |
| 语种 | 英语 |
| WOS记录号 | WOS:000385858000014 |
| 源URL | [http://ir.radi.ac.cn/handle/183411/39373]  |
| 专题 | 遥感与数字地球研究所_SCI/EI期刊论文_期刊论文 |
| 作者单位 | 1.Chinese Acad Sci, Inst Remote Sensing & Digital Earth, Beijing 100094, Peoples R China 2.Chinese Acad Sci, Comp Network Informat Ctr, Beijing, Peoples R China 3.Chinese Acad Sci, Inst Atmospher Phys, Beijing, Peoples R China |
| 推荐引用方式 GB/T 7714 | Wang, Yuzhu,Jiang, Jinrong,Ye, Huang,et al. A distributed load balancing algorithm for climate big data processing over a multi-core CPU cluster[J]. CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE,2016,28(15):4144-4160. |
| APA | Wang, Yuzhu,Jiang, Jinrong,Ye, Huang,&He, Juanxiong.(2016).A distributed load balancing algorithm for climate big data processing over a multi-core CPU cluster.CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE,28(15),4144-4160. |
| MLA | Wang, Yuzhu,et al."A distributed load balancing algorithm for climate big data processing over a multi-core CPU cluster".CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE 28.15(2016):4144-4160. |
入库方式： OAI收割
来源：遥感与数字地球研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。