A Data Pipeline for Optical and Near-infrared Solar Eruption Tracer
文献类型:期刊论文
作者 | Wang XH(王新华)1,2; Chen D(陈东)1; Deng, Tao3; Dai, Hongbing3; Xiang YY(向永源)1; Dai HB(代红兵) |
刊名 | 天文研究与技术(Astronomical Research & Technology) |
出版日期 | 2021 |
ISSN号 | 1672-7673 |
关键词 | ONSET data pipeline container GPU |
DOI | 10.14005/j.cnki.issn1672-7673.20210315.001 |
其他题名 | ONSET数据流水线 |
产权排序 | 第1完成单位 |
英文摘要 | With the advent of large astronomical equipments, the traditional development model for data reduction faces problems such as redundancy of programs and conflicting environmental dependencies; Besides as a cluster is a highly coupled computing resource, serious environmental conflicts can lead to the unavailability of the entire cluster. To address this problem, we have developed a new pipeline framework using the concept of microservices. This paper presents the ONSET data pipeline developed through this framework. To achieve near real-time data processing, we optimize the core program using MPI and GPU technologies and evaluate the final performance. The results show that this development model can be built in a short time to meet the requirements of the pipeline, and we believe that this development model has implications for future multi-band and multi-terminal astronomical data processing. |
学科主题 | 计算机科学技术 ; 计算机应用 |
分类号 | P182.2+1 ; TP751.2 |
资助项目 | National Natural Science Foundation of China[U1831210] ; National Natural Science Foundation of China[11973088] |
语种 | 英语 |
资助机构 | National Natural Science Foundation of China[U1831210, 11973088] |
源URL | [http://ir.ynao.ac.cn/handle/114a53/24163] |
专题 | 云南天文台_抚仙湖太阳观测站 |
通讯作者 | Chen D(陈东) |
作者单位 | 1.Yunnan Astronomical Observatories, Chinese Academy of Sciences, Kunming, Yunnan, 650011 2.University of Chinese Academy of Sciences, Beijing, 100049 3.School of Information, Yunnan University, Kunming, Yunnan, 650504 |
推荐引用方式 GB/T 7714 | Wang XH,Chen D,Deng, Tao,et al. A Data Pipeline for Optical and Near-infrared Solar Eruption Tracer[J]. 天文研究与技术(Astronomical Research & Technology),2021. |
APA | Wang XH,Chen D,Deng, Tao,Dai, Hongbing,Xiang YY,&Dai HB.(2021).A Data Pipeline for Optical and Near-infrared Solar Eruption Tracer.天文研究与技术(Astronomical Research & Technology). |
MLA | Wang XH,et al."A Data Pipeline for Optical and Near-infrared Solar Eruption Tracer".天文研究与技术(Astronomical Research & Technology) (2021). |
入库方式: OAI收割
来源:云南天文台
其他版本
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。