OpenCluster: A Flexible Distributed Computing Framework for Astronomical Data Processing
文献类型：期刊论文
| 作者 | Wei SL(卫守林)1,2,3 ; Wang F(王锋)1,2,3; Deng, Hui2; Liu, Cuiyin2; Dai W(戴伟)1,2,3; Liang, Bo2; Mei Y(梅盈)1,2,3; Shi, Congming2; Liu, Yingbo4; Wu, Jingping2
|
| 刊名 | PUBLICATIONS OF THE ASTRONOMICAL SOCIETY OF THE PACIFIC
 |
| 出版日期 | 2017-02-01 |
| 卷号 | 129期号:972 |
| 关键词 | instrumentation: interferometers methods: data analysis methods: numerical techniques: image processing techniques: interferometric |
| ISSN号 | 0004-6280 |
| 产权排序 | 第一完成单位 |
| 通讯作者 | Wang, F |
| 英文摘要 | The volume of data generated by modern astronomical telescopes is extremely large and rapidly growing. However, current high-performance data processing architectures/frameworks are not well suited for astronomers because of their limitations and programming difficulties. In this paper, we therefore present OpenCluster, an open-source distributed computing framework to support rapidly developing high-performance processing pipelines of astronomical big data. We first detail the OpenCluster design principles and implementations and present the APIs facilitated by the framework. We then demonstrate a case in which OpenCluster is used to resolve complex data processing problems for developing a pipeline for the Mingantu Ultrawide Spectral Radioheliograph. Finally, we present our OpenCluster performance evaluation. Overall, OpenCluster provides not only high fault tolerance and simple programming interfaces, but also a flexible means of scaling up the number of interacting entities. OpenCluster thereby provides an easily integrated distributed computing framework for quickly developing a high-performance data processing system of astronomical telescopes and for significantly reducing software development expenses. |
| WOS标题词 | Science & Technology ; Physical Sciences |
| 学科主题 | 天文学 |
| 类目[WOS] | Astronomy & Astrophysics |
| 研究领域[WOS] | Astronomy & Astrophysics |
| 关键词[WOS] | MAPREDUCE ; COMPUTATIONS |
| 收录类别 | SCI |
| 原文出处 | http://iopscience.iop.org/article/10.1088/1538-3873/129/972/024001/meta |
| 语种 | 英语 |
| WOS记录号 | WOS:000391578300001 |
| 源URL | [http://ir.ynao.ac.cn/handle/114a53/9874]  |
| 专题 | 云南天文台_其他 |
| 作者单位 | 1.Yunnan Observatories, Chinese Academy of Sciences, China 2.Computer Technology Application Key Lab of Yunnan Province, Kunming University of Science and Technology, China 3.University of Chinese Academy of Sciences, China 4.Yunnan academy of scientific & technical information, China |
| 推荐引用方式 GB/T 7714 | Wei SL,Wang F,Deng, Hui,et al. OpenCluster: A Flexible Distributed Computing Framework for Astronomical Data Processing[J]. PUBLICATIONS OF THE ASTRONOMICAL SOCIETY OF THE PACIFIC,2017,129(972). |
| APA | Wei SL.,Wang F.,Deng, Hui.,Liu, Cuiyin.,Dai W.,...&Wu, Jingping.(2017).OpenCluster: A Flexible Distributed Computing Framework for Astronomical Data Processing.PUBLICATIONS OF THE ASTRONOMICAL SOCIETY OF THE PACIFIC,129(972). |
| MLA | Wei SL,et al."OpenCluster: A Flexible Distributed Computing Framework for Astronomical Data Processing".PUBLICATIONS OF THE ASTRONOMICAL SOCIETY OF THE PACIFIC 129.972(2017). |
入库方式： OAI收割
来源：云南天文台
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。