A Parallel File System with Application-Aware Data Layout Policies for Massive Remote Sensing Image Processing in Digital Earth
文献类型：期刊论文
| 作者 | Wang, Lizhe1; Ma, Yan1; Zomaya, Albert Y.1; Ranjan, Rajiv1; Chen, Dan1 |
| 刊名 | IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS
 |
| 出版日期 | 2015 |
| 卷号 | 26期号:6页码:6489-6509 |
| 关键词 | Parallel file system parallel I/O data-intensive computing digital earth big data remote sensing image processing |
| 通讯作者 | Wang, LZ (reprint author), Chinese Acad Sci, Inst Remote Sensing & Digital Earth, Beijing 100094, Peoples R China. |
| 英文摘要 | Remote sensing applications in Digital Earth are overwhelmed with vast quantities of remote sensing (RS) image data. The intolerable I/O burden introduced by the massive amounts of RS data and the irregular RS data access patterns has made the traditional cluster based parallel I/O systems no longer applicable. We propose a RS data object-based parallel file system for remote sensing applications and implement it with the OrangeFS file system. It provides application-aware data layout policies, together with RS data object based data I/O interfaces, for efficient support of various data access patterns of RS applications from the server side. With the prior knowledge of the desired RS data access patterns, HPGFS could offer relevant space-filling curves to organize the sliced 3-D data bricks and distribute them over I/O servers. In this way, data layouts consistent with expected data access patterns could be created to explore data locality and achieve performance improvement. Moreover, the multi-band RS data with complex structured geographical metadata could be accessed and managed as a single data object. Through experiments on remote sensing applications with different access patterns, we have achieved performance improvement of about 30 percent for I/O and 20 percent overall. |
| 研究领域[WOS] | Computer Science, Theory & Methods ; Engineering, Electrical & Electronic |
| 收录类别 | SCI ; EI |
| 语种 | 英语 |
| WOS记录号 | WOS:000354526000001 |
| 源URL | [http://ir.ceode.ac.cn/handle/183411/38200]  |
| 专题 | 遥感与数字地球研究所_SCI/EI期刊论文_期刊论文 |
| 作者单位 | 1.[Wang, Lizhe 2.Ma, Yan] Chinese Acad Sci, Inst Remote Sensing & Digital Earth, Beijing 100094, Peoples R China 3.[Zomaya, Albert Y.] Univ Sydney, Sch Informat Technol, Sydney, NSW 2006, Australia 4.[Ranjan, Rajiv] ICT CSIRO, Marsfield, NSW, Australia 5.[Chen, Dan] Wuhan Univ, Sch Comp, Wuhan 430072, Hubei, Peoples R China |
| 推荐引用方式 GB/T 7714 | Wang, Lizhe,Ma, Yan,Zomaya, Albert Y.,et al. A Parallel File System with Application-Aware Data Layout Policies for Massive Remote Sensing Image Processing in Digital Earth[J]. IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS,2015,26(6):6489-6509. |
| APA | Wang, Lizhe,Ma, Yan,Zomaya, Albert Y.,Ranjan, Rajiv,&Chen, Dan.(2015).A Parallel File System with Application-Aware Data Layout Policies for Massive Remote Sensing Image Processing in Digital Earth.IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS,26(6),6489-6509. |
| MLA | Wang, Lizhe,et al."A Parallel File System with Application-Aware Data Layout Policies for Massive Remote Sensing Image Processing in Digital Earth".IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 26.6(2015):6489-6509. |
入库方式： OAI收割
来源：遥感与数字地球研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。