中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 新疆天文台
  3. 研究单元未命名
  4. 计算机技术室
Study on tiered storage algorithm based on heat correlation of astronomical data

文献类型：期刊论文

作者Ye, Xin-Chen1,2,4; Zhang, Hai-Long1,2,3,4; Wang, Jie2,4; Zhang, Ya-Zhou1,2; Du, Xu1,2; Wu, Han1,2
刊名FRONTIERS IN ASTRONOMY AND SPACE SCIENCES
出版日期2024-03-14
卷号11页码:1371249
关键词tiered strorage astronomical data processing load prediction decision tree high performance computing
ISSN号2296-987X
DOI10.3389/fspas.2024.1371249
产权排序1
英文摘要With the surge in astronomical data volume, modern astronomical research faces significant challenges in data storage, processing, and access. The I/O bottleneck issue in astronomical data processing is particularly prominent, limiting the efficiency of data processing. To address this issue, this paper proposes a tiered storage algorithm based on the access characteristics of astronomical data. The C4.5 decision tree algorithm is employed as the foundation to implement an astronomical data access correlation algorithm. Additionally, a data copy migration strategy is designed based on tiered storage technology to achieve efficient data access. Preprocessing tests were conducted on 418GB NSRT (Nanshan Radio Telescope) formaldehyde spectral line data, showcasing that tiered storage can potentially reduce data processing time by up to 38.15%. Similarly, utilizing 802.2 GB data from FAST (Five-hundred-meter Aperture Spherical radio Telescope) observations for pulsar search data processing tests, the tiered storage approach demonstrated a maximum reduction of 29.00% in data processing time. In concurrent testing of data processing workflows, the proposed astronomical data heat correlation algorithm in this paper achieved an average reduction of 17.78% in data processing time compared to centralized storage. Furthermore, in comparison to traditional heat algorithms, it reduced data processing time by 5.15%. The effectiveness of the proposed algorithm is positively correlated with the associativity between the algorithm and the processed data. The tiered storage algorithm based on the characteristics of astronomical data proposed in this paper is poised to provide algorithmic references for large-scale data processing in the field of astronomy in the future.
资助项目National Key R&D Program of China[2021YFC2203502] ; National Key R&D Program of China[2022YFF0711502] ; National Natural Science Foundation of China (NSFC)[12173077] ; National Natural Science Foundation of China (NSFC)[12003062] ; Tianshan Innovation Team Plan of Xinjiang Uygur Autonomous Region[2022D14020] ; Tianshan Talent Project of Xinjiang Uygur Autonomous Region[2022TSYCCX0095] ; Scientific Instrument Developing Project of the Chinese Academy of Sciences[PTYQ2022YZZD01] ; China National Astronomical Data Center (NADC) ; Operation, Maintenance and Upgrading Fund for Astronomical Telescopes and Facility Instruments ; Ministry of Finance of China ; Natural Science Foundation of Xinjiang Uygur Autonomous Region[2022D01A360]
WOS研究方向Astronomy & Astrophysics
语种英语
WOS记录号WOS:001191944200001
出版者FRONTIERS MEDIA SA
资助机构National Key R&D Program of China ; National Natural Science Foundation of China (NSFC) ; Tianshan Innovation Team Plan of Xinjiang Uygur Autonomous Region ; Tianshan Talent Project of Xinjiang Uygur Autonomous Region ; Scientific Instrument Developing Project of the Chinese Academy of Sciences ; China National Astronomical Data Center (NADC) ; Operation, Maintenance and Upgrading Fund for Astronomical Telescopes and Facility Instruments ; Ministry of Finance of China ; Natural Science Foundation of Xinjiang Uygur Autonomous Region
源URL[http://ir.xao.ac.cn/handle/45760611-7/5958]  
专题新疆天文台_计算机技术室
射电天文研究室_利用南山26米射电望远镜观测数据的文章
科研仪器设备产出_我台利用FAST观测数据文章
通讯作者Zhang, Hai-Long
作者单位1.Univ Chinese Acad Sci, Beijing, Peoples R China
2.Chinese Acad Sci, Xinjiang Astron Observ, Urumqi, Peoples R China
3.Chinese Acad Sci, Key Lab Radio Astron, Nanjing, Peoples R China
4.Natl Astron Data Ctr, Beijing, Peoples R China
推荐引用方式
GB/T 7714		 Ye, Xin-Chen,Zhang, Hai-Long,Wang, Jie,et al. Study on tiered storage algorithm based on heat correlation of astronomical data[J]. FRONTIERS IN ASTRONOMY AND SPACE SCIENCES,2024,11:1371249.
APA Ye, Xin-Chen,Zhang, Hai-Long,Wang, Jie,Zhang, Ya-Zhou,Du, Xu,&Wu, Han.(2024).Study on tiered storage algorithm based on heat correlation of astronomical data.FRONTIERS IN ASTRONOMY AND SPACE SCIENCES,11,1371249.
MLA Ye, Xin-Chen,et al."Study on tiered storage algorithm based on heat correlation of astronomical data".FRONTIERS IN ASTRONOMY AND SPACE SCIENCES 11(2024):1371249.

入库方式： OAI收割

来源：新疆天文台

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。