A Heuristic Big Data Processing Multi Task Efficient Deployment Method Based on QoS Aware Clustering and Bayesian Classification in Cloud Environment
文献类型：期刊论文
| 作者 | Yu, ZiYuan2; Yang, Peng2; Wu, WanJing2; Tan, Liang1,2; Ye, DanLian2; She, Kun3 |
| 刊名 | IEEE TRANSACTIONS ON SERVICES COMPUTING
 |
| 出版日期 | 2025-09-01 |
| 卷号 | 18期号:5页码:2517-2530 |
| 关键词 | Load management Cloud computing Big Data Quality of service Multitasking Heuristic algorithms Bayes methods Virtual machines Resource management Real-time systems big data Bayesian classification QoS-aware multi-task deployment |
| ISSN号 | 1939-1374 |
| DOI | 10.1109/TSC.2025.3592228 |
| 英文摘要 | The efficient deployment of Big Data processing tasks in cloud environments is the basic core function of Big Data processing, which refers to the effective deployment of tasks to the computing resources of cloud platforms, achieving high-performance and high-throughput data processing. In this process, task deployment needs to consider load balancing on the cloud platform to ensure that tasks can be evenly deployed to each computing node. However, currently in the process of providing services on cloud platforms, the available resources of all hosts will be automatically and dynamically readjusted, and it cannot be guaranteed that each task will be deployed to the host with the most remaining resources. This load imbalance in the platform will result in computational results that cannot be returned to users in a timely and effective manner. So, a heuristic multi-task efficient deployment approach for Big Data processing based on QoS awareness and Bayesian classification in cloud environments called QBC is proposed. The QBC first performs long-run QoS awareness on hosts in the cloud; Then, based on user task requirements, selects host nodes that meet QoS constraints to form a candidate set, and performs Bayesian classification to find the host node which has highest a posteriori probability to serve as the clustering center; third, designs an objective function based on euclidean spatial distance to acquire the optimum host clustering set in the candidate set; Finally, deploys the user's tasks to this optimal host cluster set. The experimental results show that this approach implements optimization of long-run load balancing in Big Data cloud platforms with minimal resource consumption, enhances the ability of the cloud platform to provide external support, and thus promotes efficient deployment of multitasking in Big Data processing under cloud computing. The proposed QBC based framework reduces the overall Energy Consumption by an average of 47.98%, MakeSpan by an average of 24.42%, Total Cost by an average of 30.17%, Average Waiting Time by an average of 36.92%, and the Throughput is increased by an average of 41.93% as compared to the existing algorithms. |
| 资助项目 | National Natural Science Foundation of China[61373126] ; National Natural Science Foundation of China[62301348] ; Sichuan Provincial Science and Technology Department Project[2022YFG0161] ; Sichuan Provincial Science and Technology Department Project[2023YFG0295] |
| WOS研究方向 | Computer Science |
| 语种 | 英语 |
| WOS记录号 | WOS:001591693600035 |
| 出版者 | IEEE COMPUTER SOC |
| 源URL | [http://119.78.100.204/handle/2XEOYT63/41608]  |
| 专题 | 中国科学院计算技术研究所期刊论文_英文 |
| 通讯作者 | Tan, Liang |
| 作者单位 | 1.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China 2.Sichuan Normal Univ, Sch Comp Sci, Chengdu 610101, Peoples R China 3.Univ Elect Sci & Technol China, Chengdu 610054, Peoples R China |
| 推荐引用方式 GB/T 7714 | Yu, ZiYuan,Yang, Peng,Wu, WanJing,et al. A Heuristic Big Data Processing Multi Task Efficient Deployment Method Based on QoS Aware Clustering and Bayesian Classification in Cloud Environment[J]. IEEE TRANSACTIONS ON SERVICES COMPUTING,2025,18(5):2517-2530. |
| APA | Yu, ZiYuan,Yang, Peng,Wu, WanJing,Tan, Liang,Ye, DanLian,&She, Kun.(2025).A Heuristic Big Data Processing Multi Task Efficient Deployment Method Based on QoS Aware Clustering and Bayesian Classification in Cloud Environment.IEEE TRANSACTIONS ON SERVICES COMPUTING,18(5),2517-2530. |
| MLA | Yu, ZiYuan,et al."A Heuristic Big Data Processing Multi Task Efficient Deployment Method Based on QoS Aware Clustering and Bayesian Classification in Cloud Environment".IEEE TRANSACTIONS ON SERVICES COMPUTING 18.5(2025):2517-2530. |
入库方式： OAI收割
来源：计算技术研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。