A multi-group and preemptable scheduling of cloud resource based on HTCondor
文献类型：期刊论文
| 作者 | Jiang XW(姜晓巍) ; Zou JH(邹佳恒); Cheng YD(程耀东); Shi JY(石京燕); Jiang, Xiaowei; Zou, Jiaheng; Cheng, Yaodong; Shi, Jingyan
|
| 刊名 | Journal of Physics: Conference Series
 |
| 出版日期 | 2017 |
| 卷号 | 898期号:9页码:092051 |
| ISSN号 | 1742-6588 |
| DOI | 10.1088/1742-6596/898/9/092051 |
| 文献子类 | Proceedings Paper |
| 英文摘要 | Due to the features of virtual machine-flexibility, easy controlling and various system environments, more and more fields utilize the virtualization technology to construct the distributed system with the virtual resources, also including high energy physics. This paper introduce a method used in high energy physics that supports multiple resource group and preemptable cloud resource scheduling, combining virtual machine with HTCondor (a batch system). It makes resource controlling more flexible and more efficient and makes resource scheduling independent of job scheduling. Firstly, the resources belong to different experiment-groups, and the type of user-groups mapping to resource-groups(same as experiment-group) is one-to-one or many-to-one. In order to make the confused group simply to be managed, we designed the permission controlling component to ensure that the different resource-groups can get the suitable jobs. Secondly, for the purpose of elastically allocating resources for suitable resource-group, it is necessary to schedule resources like scheduling jobs. So this paper designs the cloud resource scheduling to maintain a resource queue and allocate an appropriate amount of virtual resources to the request resource-group. Thirdly, in some kind of situations, because of the resource occupied for a long time, resources need to be preempted. This paper adds the preemption function for the resource scheduling that implement resource preemption based on the group priority. Additionally, the way to preempting is soft that when virtual resources are preempted, jobs will not be killed but also be held and rematched later. It is implemented with the help of HTCondor, storing the held job information in scheduler, releasing the job to idle status and doing second matcher. In IHEP (institute of high energy physics), we have built a batch system based on HTCondor with a virtual resources pool based on Openstack. And this paper will show some cases of experiment JUNO and LHAASO. The result indicates that multi-group and preemptable resource scheduling is efficient to support multi-group and soft preemption. Additionally, the permission controlling component has been used in the local computing cluster, supporting for experiment JUNO, CMS and LHAASO, and the scale will be expanded to more experiments at the first half year, including DYW, BES and so on. Its evidence that the permission controlling is efficient. © Published under licence by IOP Publishing Ltd. |
| 会议地点 | San Francisco, CA, United states |
| 电子版国际标准刊号 | 1742-6596 |
| 会议日期 | October 10, 2016 - October 14, 2016 |
| 语种 | 英语 |
| 源URL | [http://ir.ihep.ac.cn/handle/311005/285447]  |
| 专题 | 高能物理研究所_计算中心 |
| 作者单位 | Institute of High Energy Physics, Chinese Academy of Sciences, Beijing, China |
| 推荐引用方式 GB/T 7714 | Jiang XW,Zou JH,Cheng YD,et al. A multi-group and preemptable scheduling of cloud resource based on HTCondor[J]. Journal of Physics: Conference Series,2017,898(9):092051. |
| APA | 姜晓巍.,邹佳恒.,程耀东.,石京燕.,Jiang, Xiaowei.,...&Shi, Jingyan.(2017).A multi-group and preemptable scheduling of cloud resource based on HTCondor.Journal of Physics: Conference Series,898(9),092051. |
| MLA | 姜晓巍,et al."A multi-group and preemptable scheduling of cloud resource based on HTCondor".Journal of Physics: Conference Series 898.9(2017):092051. |
入库方式： OAI收割
来源：高能物理研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。