A resource-aware workload scheduling method for unbalanced GEMMs on GPUs
文献类型：期刊论文
| 作者 | Liu, Hangda1,2; Diao, Boyu1; Chen, Wenxin1,2; Xu, Yongjun1 |
| 刊名 | COMPUTER JOURNAL
 |
| 出版日期 | 2024-10-25 |
| 页码 | 10 |
| ISSN号 | 0010-4620 |
| DOI | 10.1093/comjnl/bxae110 |
| 英文摘要 | GEMM (General Matrix Multiplication) serves as a fundamental operator for deep learning computations. Especially in attention-based deep learning models, such as Bert, GPT, and SAM, the sizes of matrices involved in GEMMs exhibit an unbalanced distribution due to the variable input, resulting in the low utilization of hardware resources. To address the issue, this paper proposes inserting a novel GEMM processing layer into the deep learning inference stack and using an adaptive load balancing method to partition and schedule GEMM computation tasks. The method is implemented with hardware runtime resource information, such as the occupancy of computing units, etc. Experiment results show the remarkable performance of our method in unbalanced input GEMM scenarios, achieving an average performance improvement of 2.3x. The method also performs well in attention-based models (GPT-2 and SAM), achieving an average inference speed improvement of 1.1x. These findings highlight the effectiveness of resource-aware algorithm optimization, especially for computation task scheduling. |
| WOS研究方向 | Computer Science |
| 语种 | 英语 |
| WOS记录号 | WOS:001342476600001 |
| 出版者 | OXFORD UNIV PRESS |
| 源URL | [http://119.78.100.204/handle/2XEOYT63/39507]  |
| 专题 | 中国科学院计算技术研究所期刊论文_英文 |
| 通讯作者 | Diao, Boyu |
| 作者单位 | 1.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100190, Peoples R China |
| 推荐引用方式 GB/T 7714 | Liu, Hangda,Diao, Boyu,Chen, Wenxin,et al. A resource-aware workload scheduling method for unbalanced GEMMs on GPUs[J]. COMPUTER JOURNAL,2024:10. |
| APA | Liu, Hangda,Diao, Boyu,Chen, Wenxin,&Xu, Yongjun.(2024).A resource-aware workload scheduling method for unbalanced GEMMs on GPUs.COMPUTER JOURNAL,10. |
| MLA | Liu, Hangda,et al."A resource-aware workload scheduling method for unbalanced GEMMs on GPUs".COMPUTER JOURNAL (2024):10. |
入库方式： OAI收割
来源：计算技术研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。