中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
Fast and scalable all-optical network architecture for distributed deep learning

文献类型：期刊论文

作者Li, Wenzhe3; Yuan, Guojun3; Wang, Zhan3; Tan, Guangming3; Zhang, Peiheng1,3; Rouskas, George N.2
刊名JOURNAL OF OPTICAL COMMUNICATIONS AND NETWORKING
出版日期2024-03-01
卷号16期号:3页码:342-357
ISSN号1943-0620
DOI10.1364/JOCN.511696
英文摘要With the ever-increasing size of training models and datasets, network communication has emerged as a major bottleneck in distributed deep learning training. To address this challenge, we propose an optical distributed deep learning (ODDL) architecture. ODDL utilizes a fast yet scalable all-optical network architecture to accelerate distributed training. One of the key features of the architecture is its flow-based transmit scheduling with fast reconfiguration. This allows ODDL to allocate dedicated optical paths for each traffic stream dynamically, resulting in low network latency and high network utilization. Additionally, ODDL provides physically isolated and tailored network resources for training tasks by reconfiguring the optical switch using LCoS-WSS technology. The ODDL topology also uses tunable transceivers to adapt to time-varying traffic patterns. To achieve accurate and fine-grained scheduling of optical circuits, we propose an efficient distributed control scheme that incurs minimal delay overhead. Our evaluation on real-world traces showcases ODDL's remarkable performance. When implemented with 1024 nodes and 100 Gbps bandwidth, ODDL accelerates VGG19 training by 1.6x and 1.7x compared to conventional fat-tree electrical networks and photonic SiP-Ring architectures, respectively. We further build a four-node testbed, and our experiments show that ODDL can achieve comparable training time compared to that of an ideal electrical switching network. (c) 2024 Optica Publishing Group
资助项目National Key Research and Development Program of China[2021YFB0300700] ; National Natural Science Foundation of China[61972380] ; Jiangsu Science and Technology Project[BE2022051-2] ; National Science Foundation[CNS-1907142]
WOS研究方向Computer Science ; Optics ; Telecommunications
语种英语
WOS记录号WOS:001177075100001
出版者Optica Publishing Group
源URL[http://119.78.100.204/handle/2XEOYT63/38808]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Yuan, Guojun
作者单位1.Chinese Acad Sci, Inst Intelligent Comp Technol, 88 Jinji Lake Ave,Ind Pk, Suzhou, Peoples R China
2.North Carolina State Univ, Dept Comp Sci, 890 Oval Dr, Raleigh, NC 27695 USA
3.Chinese Acad Sci, Inst Comp Technol, 6 Kexueyuan South Rd Zhongguancun, Beijing, Peoples R China
推荐引用方式
GB/T 7714		 Li, Wenzhe,Yuan, Guojun,Wang, Zhan,et al. Fast and scalable all-optical network architecture for distributed deep learning[J]. JOURNAL OF OPTICAL COMMUNICATIONS AND NETWORKING,2024,16(3):342-357.
APA Li, Wenzhe,Yuan, Guojun,Wang, Zhan,Tan, Guangming,Zhang, Peiheng,&Rouskas, George N..(2024).Fast and scalable all-optical network architecture for distributed deep learning.JOURNAL OF OPTICAL COMMUNICATIONS AND NETWORKING,16(3),342-357.
MLA Li, Wenzhe,et al."Fast and scalable all-optical network architecture for distributed deep learning".JOURNAL OF OPTICAL COMMUNICATIONS AND NETWORKING 16.3(2024):342-357.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。