热门
Data Management Challenges and Event Index Technologies in High Energy Physics
文献类型：期刊论文
| 作者 | Zhang, Xiao ; Cheng YD(程耀东); Ma, Can; Qi, Yong; Hou, Di; Zha, Li; Wang, Peijian; Cheng, Yaodong
|
| 刊名 | Computer Research and Development计算机研究与发展
 |
| 出版日期 | 2017 |
| 卷号 | 54期号:2页码:258-266 |
| ISSN号 | 1000-1239 |
| DOI | 10.7544/issn1000-1239.2017.20160939 |
| 其他题名 | 高能物理大数据挑战与海量事例特征索引技术研究 |
| 文献子类 | Article |
| 英文摘要 | Nowadays, more and more scientific data has been produced by new generation high energy physics facilities. The scale of the data can be achieved to PB or EB level even by one experiment, which brings big challenges to data management technologies such as data acquisition, storage, transmission, sharing, analyzing and processing. Event is the basic data unit of high energy physics, and one large high energy physics experiment can produce trillions of events. The traditional high energy physical data processing technology adopts file as a basic data management unit, and each file contains thousands of events. The benefit of file-based method is to simplify the complexity of data management system. However, one physical analysis task is only interested in very few events, which leads to some problems including transferring too much redundant data, I/O bottleneck and low efficiency of data processing. To solve these problems, this paper proposes an event-oriented high energy physical data management method, which focuses on high efficiency indexing technology of massive events. In this method, event data is still stored in ROOT file while a large amount of events are indexed by some specified properties and stored in NoSQL database. Finally, experimental test results show the feasibility of the method, and optimized HBase system can meet the requirements of event index. © 2017, Science Press. All right reserved. |
| 语种 | 中文 |
| CSCD记录号 | CSCD:5928217 |
| 源URL | [http://ir.ihep.ac.cn/handle/311005/284235]  |
| 专题 | 中国科学院高能物理研究所 |
| 作者单位 | 1.Institute of Information Engineering, Chinese Academy of Sciences, Beijing; 100193, China 2.Institute of Computing Technology, Chinese Academy of Sciences, Beijing; 100190, China; 3.Department of Computer Science and Technology, Xi'an Jiaotong University, Xi'an; 710049, China; 4.Institute of High Energy Physics, Chinese Academy of Sciences, Beijing; 100049, China; |
| 推荐引用方式 GB/T 7714 | Zhang, Xiao,Cheng YD,Ma, Can,et al. Data Management Challenges and Event Index Technologies in High Energy Physics[J]. Computer Research and Development计算机研究与发展,2017,54(2):258-266. |
| APA | Zhang, Xiao.,程耀东.,Ma, Can.,Qi, Yong.,Hou, Di.,...&Cheng, Yaodong.(2017).Data Management Challenges and Event Index Technologies in High Energy Physics.Computer Research and Development计算机研究与发展,54(2),258-266. |
| MLA | Zhang, Xiao,et al."Data Management Challenges and Event Index Technologies in High Energy Physics".Computer Research and Development计算机研究与发展 54.2(2017):258-266. |
入库方式： OAI收割
来源：高能物理研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。