Distributed Metadata Management of Mass Storage System in High Energy Physics
文献类型：期刊论文
| 作者 | Du, Ran ; Cheng, Yaodong; Shi, Jingyan; Kan WX(阚文枭); Chen G(陈刚); Shi JY(石京燕); Cheng YD(程耀东); Du R(杜然); Huang QL(黄秋兰); Kan, Wenxiao
|
| 刊名 | Journal of Physics: Conference Series
 |
| 出版日期 | 2017 |
| 卷号 | 898期号:6页码:062003 |
| ISSN号 | 1742-6588 |
| DOI | 10.1088/1742-6596/898/6/062003 |
| 文献子类 | Proceedings Paper |
| 英文摘要 | In this contribution, we design and implement a dynamic and scalable distributed metadata management system(StarFS) to High Energy Physics (HEP) mass storage system. Particularly, we discuss the key technologies of the distributed metadata management system. We propose a new algorithm named Adaptive Directory Sub-tree Partition(ADSP) for metadata distribution. ADSP divides the filesystem namespace into sub-trees with directory granularity. Sub-trees will be stored on storage devices in flat structure, whose location and file attributes are recorded as extended attributes. The placement of sub-tree is adjusted adaptively according to the workload of metadata cluster so that the load balance could be improved and metadata cluster could be extended dynamically. Experiments show that ADSP achieves higher metadata performance and scalability compared to Gluster and Lustre. We also propose a new algorithm called Distributed Unified Layout(DULA) to improve dynamic scalability and efficiency of data positioning. A system with DULA could provide uniform data distribution and efficient data positioning. DULA is an improved consistent hashing algorithm which is able to locate data in O(1) without the help of routing information. Experiments prove that the better uniform data distribution and efficient data access can be achieved by DULA. This work is validated in YangBaJing Cosmic Ray(YBJ) experiment. © Published under licence by IOP Publishing Ltd. |
| 会议地点 | San Francisco, CA, United states |
| 电子版国际标准刊号 | 1742-6596 |
| 会议日期 | October 10, 2016 - October 14, 2016 |
| 语种 | 英语 |
| 源URL | [http://ir.ihep.ac.cn/handle/311005/285435]  |
| 专题 | 中国科学院高能物理研究所 |
| 作者单位 | 1.University of Chinese, Academy of Sciences, Beijing, China 2.IHEP Computing Center, P.O.Box 918-7, 19B Yuquan Road, Beijing; 100049, China; |
| 推荐引用方式 GB/T 7714 | Du, Ran,Cheng, Yaodong,Shi, Jingyan,et al. Distributed Metadata Management of Mass Storage System in High Energy Physics[J]. Journal of Physics: Conference Series,2017,898(6):062003. |
| APA | Du, Ran.,Cheng, Yaodong.,Shi, Jingyan.,阚文枭.,陈刚.,...&Chen, Gang.(2017).Distributed Metadata Management of Mass Storage System in High Energy Physics.Journal of Physics: Conference Series,898(6),062003. |
| MLA | Du, Ran,et al."Distributed Metadata Management of Mass Storage System in High Energy Physics".Journal of Physics: Conference Series 898.6(2017):062003. |
入库方式： OAI收割
来源：高能物理研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。