Parallelization and optimization of spatial analysis for large scale environmental model data assembly
文献类型：期刊论文
| 作者 | Zhao Gang(赵刚) ; Bryan Brett A. ; King Darran ; Song Xiaodong ; Yu Qiang |
| 刊名 | Computers and Electronics in Agriculture
 |
| 出版日期 | 2012 |
| 卷号 | 89页码:94-99 |
| 关键词 | Algorithms Computer simulation Computer software selection and evaluation Efficiency Forestry Geographic information systems Water resources |
| ISSN号 | 01681699 |
| 通讯作者 | Zhao Gang(赵刚) |
| 英文摘要 | Spatial-temporal modelling of environmental systems such as agriculture, forestry, and water resources requires high resolution input data. Assembling and summarizing this data in the appropriate format for model input often requires a series of spatial analyses which can be extremely time-consuming, especially when many large data sets are involved. In this paper we investigated the ability of high-performance computing techniques to improve the efficiency of spatial analysis for model data assembly. We implemented an array-based algorithm to calculate summary statistics for long time-series daily grid climate data sets for 11,575 climate-soil zones across the Australian wheat-growing regions for input into a crop simulation model. We developed a zonal statistics algorithm using Python's Numpy module then parallelized it and processed it using a shared memory, multi-processor system. We assessed algorithm performance with a varying number of CPU cores, and assessed the influence of load balancing on the efficiency of parallel processing. Compared with traditional desktop GIS software, the serial and parallel (32 cores) implementation achieved about 180 and 1440 times speed-up, respectively. We also found that the most efficient computation occurred when not all of the available CPU cores were used, and the chunk size of jobs also had an important influence on computing efficiency. The algorithm and the parallel processing scheme provides a useful approach to address computing challenges posed by spatial analysis of numerous large data sets for large scale environmental modelling. 2012 Elsevier B.V. |
| 收录类别 | SCI |
| 资助信息 | CSIRO's Sustainable Agriculture Flagship；National Basic Research Program of China 2012CB955304；WRON Data Centre；Chinese Scholarship Council |
| 语种 | 英语 |
| 公开日期 | 2012-11-23 |
| 源URL | [http://ir.igsnrr.ac.cn/handle/311030/27726]  |
| 专题 | 地理科学与资源研究所_研究生部 |
| 推荐引用方式 GB/T 7714 | Zhao Gang,Bryan Brett A.,King Darran,et al. Parallelization and optimization of spatial analysis for large scale environmental model data assembly[J]. Computers and Electronics in Agriculture,2012,89:94-99. |
| APA | Zhao Gang,Bryan Brett A.,King Darran,Song Xiaodong,&Yu Qiang.(2012).Parallelization and optimization of spatial analysis for large scale environmental model data assembly.Computers and Electronics in Agriculture,89,94-99. |
| MLA | Zhao Gang,et al."Parallelization and optimization of spatial analysis for large scale environmental model data assembly".Computers and Electronics in Agriculture 89(2012):94-99. |
入库方式： OAI收割
来源：地理科学与资源研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。