Directly training temporal Spiking Neural Network with sparse surrogate gradient
文献类型：期刊论文
| 作者 | Li, Yang1,2 ; Zhao, Feifei1,2; Zhao, Dongcheng1; Zeng, Yi1,2,3,4
|
| 刊名 | NEURAL NETWORKS
 |
| 出版日期 | 2024-11-01 |
| 卷号 | 179页码:9 |
| 关键词 | Spiking Neural Network Sparse Surrogate Gradient Direct Training Temporally Weighted Output |
| ISSN号 | 0893-6080 |
| DOI | 10.1016/j.neunet.2024.106499 |
| 通讯作者 | Zeng, Yi(yi.zeng@ia.ac.cn) |
| 英文摘要 | Brain-inspired Spiking Neural Networks (SNNs) have attracted much attention due to their event-based computing and energy-efficient features. However, the spiking all-or-none nature has prevented direct training of SNNs for various applications. The surrogate gradient (SG) algorithm has recently enabled spiking neural networks to shine in neuromorphic hardware. However, introducing surrogate gradients has caused SNNs to lose their original sparsity, thus leading to the potential performance loss. In this paper, we first analyze the current problem of direct training using SGs and then propose Masked Surrogate Gradients (MSGs) to balance the effectiveness of training and the sparseness of the gradient, thereby improving the generalization ability of SNNs. Moreover, we introduce a temporally weighted output (TWO) method to decode the network output, reinforcing the importance of correct timesteps. Extensive experiments on diverse network structures and datasets show that training with MSG and TWO surpasses the SOTA technique. |
| WOS关键词 | INTELLIGENCE ; DEEPER |
| 资助项目 | National Key Research and Development Program, China[2020AAA0107800] |
| WOS研究方向 | Computer Science ; Neurosciences & Neurology |
| 语种 | 英语 |
| WOS记录号 | WOS:001271937500001 |
| 出版者 | PERGAMON-ELSEVIER SCIENCE LTD |
| 资助机构 | National Key Research and Development Program, China |
| 源URL | [http://ir.ia.ac.cn/handle/173211/59273]  |
| 专题 | 类脑智能研究中心_类脑认知计算 |
| 通讯作者 | Zeng, Yi |
| 作者单位 | 1.Chinese Acad Sci, Inst Automat, Brain Inspired Cognit Intelligence Lab, Beijing, Peoples R China 2.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing, Peoples R China 3.Univ Chinese Acad Sci, Sch Future Technol, Beijing, Peoples R China 4.Chinese Acad Sci, Ctr Excellence Brain Sci & Intelligence Technol, Shanghai, Peoples R China |
| 推荐引用方式 GB/T 7714 | Li, Yang,Zhao, Feifei,Zhao, Dongcheng,et al. Directly training temporal Spiking Neural Network with sparse surrogate gradient[J]. NEURAL NETWORKS,2024,179:9. |
| APA | Li, Yang,Zhao, Feifei,Zhao, Dongcheng,&Zeng, Yi.(2024).Directly training temporal Spiking Neural Network with sparse surrogate gradient.NEURAL NETWORKS,179,9. |
| MLA | Li, Yang,et al."Directly training temporal Spiking Neural Network with sparse surrogate gradient".NEURAL NETWORKS 179(2024):9. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。