中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
CUTE: A scalable CPU-centric and Ultra-utilized Tensor Engine for convolutions

文献类型：期刊论文

作者Li, Wenqing3,4; Ye, Jinpeng3,4; Zhang, Fuxin3,4; Liu, Tianyi2; Zhang, Tingting1,4; Wang, Jian3,4
刊名JOURNAL OF SYSTEMS ARCHITECTURE
出版日期2024-04-01
卷号149页码:15
关键词Tensor engine Convolution Scalable architecture CPU-centric Utilization
ISSN号1383-7621
DOI10.1016/j.sysarc.2024.103106
英文摘要Convolution is a fundamental and computationally expensive primitive and finds ubiquitous in deep neural networks (DNNs). The evolving DNNs have spurred the emergence of numerous accelerators and they successfully achieve high throughput. However, for DNN inference with small batch sizes, the computational resources of the accelerators are often under-utilized, and the overhead of offloading is significant. Compared to accelerators, the CPU can better meet fast response requirements of inference, flexibly handle various models, and is suitable for various scenarios (from edge to data center). Therefore, CPU remains an attractive platform for DNN inference, despite the sub-optimal performance, and resource efficiency. In this paper, we propose CUTE, a scalable CPU-centric and ultra-utilized tensor engine for convolutions. It co-designs data flow and hardware architecture to leverage the data reuse and parallelism of convolutions. CUTE is composed of several small tensor elements (TEs) and two-level buffers. It employs a decoupled accessexecution architecture and greedy strategy to feed data to TEs, enabling it to achieve ultra utilization and great scalability. CUTE is tightly coupled with the CPU to minimize offloading latency, thereby providing efficient convolution computing capabilities for the system. Experimental results show that under the same bandwidth, CUTE achieves an average performance improvement of 3.8x compared with the CPU AVX512 unit and 1.6x compared with the CPU AMX unit. Besides, CUTE achieves a speedup of 7.0x and 3.9x over Nvidia V100 GPU and Eyeriss accelerator respectively, due to higher utilization of computing units.
资助项目Strategic Priority Research Program of the Chinese Academy of Sciences[XDC05020100]
WOS研究方向Computer Science
语种英语
WOS记录号WOS:001207560600001
出版者ELSEVIER
源URL[http://119.78.100.204/handle/2XEOYT63/38702]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Wang, Jian
作者单位1.Loongson Technol Corp Ltd, Beijing, Peoples R China
2.Univ Texas San Antonio, San Antonio, TX USA
3.Univ Chinese Acad Sci, Beijing, Peoples R China
4.Chinese Acad Sci, Inst Comp Technol, State Key Lab Processors, Beijing, Peoples R China
推荐引用方式
GB/T 7714		 Li, Wenqing,Ye, Jinpeng,Zhang, Fuxin,et al. CUTE: A scalable CPU-centric and Ultra-utilized Tensor Engine for convolutions[J]. JOURNAL OF SYSTEMS ARCHITECTURE,2024,149:15.
APA Li, Wenqing,Ye, Jinpeng,Zhang, Fuxin,Liu, Tianyi,Zhang, Tingting,&Wang, Jian.(2024).CUTE: A scalable CPU-centric and Ultra-utilized Tensor Engine for convolutions.JOURNAL OF SYSTEMS ARCHITECTURE,149,15.
MLA Li, Wenqing,et al."CUTE: A scalable CPU-centric and Ultra-utilized Tensor Engine for convolutions".JOURNAL OF SYSTEMS ARCHITECTURE 149(2024):15.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。