中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
A Data Pipeline for Optical and Near-infrared Solar Eruption Tracer

文献类型:期刊论文

作者Wang XH(王新华)1,2; Chen D(陈东)1; Deng, Tao3; Dai, Hongbing3; Xiang YY(向永源)1; Dai HB(代红兵)
刊名天文研究与技术(Astronomical Research & Technology)
出版日期2021
ISSN号1672-7673
关键词ONSET data pipeline container GPU
DOI10.14005/j.cnki.issn1672-7673.20210315.001
其他题名ONSET数据流水线
产权排序第1完成单位
英文摘要

With the advent of large astronomical equipments, the traditional development model for data reduction faces problems such as redundancy of programs and conflicting environmental dependencies; Besides as a cluster is a highly coupled computing resource, serious environmental conflicts can lead to the unavailability of the entire cluster. To address this problem, we have developed a new pipeline framework using the concept of microservices. This paper presents the ONSET data pipeline developed through this framework. To achieve near real-time data processing, we optimize the core program using MPI and GPU technologies and evaluate the final performance. The results show that this development model can be built in a short time to meet the requirements of the pipeline, and we believe that this development model has implications for future multi-band and multi-terminal astronomical data processing.

学科主题计算机科学技术 ; 计算机应用
分类号P182.2+1 ; TP751.2
资助项目National Natural Science Foundation of China[U1831210] ; National Natural Science Foundation of China[11973088]
语种英语
资助机构National Natural Science Foundation of China[U1831210, 11973088]
源URL[http://ir.ynao.ac.cn/handle/114a53/24163]  
专题云南天文台_抚仙湖太阳观测站
通讯作者Chen D(陈东)
作者单位1.Yunnan Astronomical Observatories, Chinese Academy of Sciences, Kunming, Yunnan, 650011
2.University of Chinese Academy of Sciences, Beijing, 100049
3.School of Information, Yunnan University, Kunming, Yunnan, 650504
推荐引用方式
GB/T 7714
Wang XH,Chen D,Deng, Tao,et al. A Data Pipeline for Optical and Near-infrared Solar Eruption Tracer[J]. 天文研究与技术(Astronomical Research & Technology),2021.
APA Wang XH,Chen D,Deng, Tao,Dai, Hongbing,Xiang YY,&Dai HB.(2021).A Data Pipeline for Optical and Near-infrared Solar Eruption Tracer.天文研究与技术(Astronomical Research & Technology).
MLA Wang XH,et al."A Data Pipeline for Optical and Near-infrared Solar Eruption Tracer".天文研究与技术(Astronomical Research & Technology) (2021).

入库方式: OAI收割

来源:云南天文台

浏览0
下载0
收藏0
其他版本

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。