Multi-scale full spike pattern for semantic segmentation
文献类型：期刊论文
| 作者 | Su, Qiaoyi1,3 ; He, Weihua2; Wei, Xiaobao3; Xu, Bo1,3; Li, Guoqi1,3,4
|
| 刊名 | NEURAL NETWORKS
 |
| 出版日期 | 2024-08-01 |
| 卷号 | 176页码:12 |
| 关键词 | Spiking neural network Semantic segmentation Neuromorphic computing Deep neural network Energy efficiency Brain-inspired computing |
| ISSN号 | 0893-6080 |
| DOI | 10.1016/j.neunet.2024.106330 |
| 通讯作者 | Li, Guoqi(guoqi.li@ia.ac.cn) |
| 英文摘要 | Spiking neural networks (SNNs), as the brain-inspired neural networks, encode information in spatio-temporal dynamics. They have the potential to serve as low-power alternatives to artificial neural networks (ANNs) due to their sparse and event-driven nature. However, existing SNN-based models for pixel-level semantic segmentation tasks suffer from poor performance and high memory overhead, failing to fully exploit the computational effectiveness and efficiency of SNNs. To address these challenges, we propose the multi-scale and full spike segmentation network (MFS-Seg), which is based on the deep direct trained SNN and represents the first attempt to train a deep SNN with surrogate gradients for semantic segmentation. Specifically, we design an efficient fully-spike residual block (EFS-Res) to alleviate representation issues caused by spiking noise on different channels. EFS-Res utilizes depthwise separable convolution to improve the distributions of spiking feature maps. The visualization shows that our model can effectively extract the edge features of segmented objects. Furthermore, it can significantly reduce the memory overhead and energy consumption of the network. In addition, we theoretically analyze and prove that EFS-Res can avoid the degradation problem based on block dynamical isometry theory. Experimental results on the Camvid dataset, the DDD17 dataset, and the DSEC-Semantic dataset show that our model achieves comparable performance to the mainstream UNet network with up to 31 x fewer parameters, while significantly reducing power consumption by over 13 x . Overall, our MFS-Seg model demonstrates promising results in terms of performance, memory efficiency, and energy consumption, showcasing the potential of deep SNNs for semantic segmentation tasks. Our code is available in https://github.com/BICLab/MFS-Seg. |
| WOS关键词 | NEURAL-NETWORK ; VIDEO ; MODEL |
| 资助项目 | National Science Foundation for National Science and Technology Major Project[2020AAA0105802] ; Distinguished Young Scholars[62325603] ; National Natural Science Foundation of China[62236009] ; National Natural Science Foundation of China[U22A20103] ; National Natural Science Foundation of China[62441606] ; Beijing Natural Science Foundation for Distinguished Young Scholars[JQ21015] |
| WOS研究方向 | Computer Science ; Neurosciences & Neurology |
| 语种 | 英语 |
| WOS记录号 | WOS:001235699200001 |
| 出版者 | PERGAMON-ELSEVIER SCIENCE LTD |
| 资助机构 | National Science Foundation for National Science and Technology Major Project ; Distinguished Young Scholars ; National Natural Science Foundation of China ; Beijing Natural Science Foundation for Distinguished Young Scholars |
| 源URL | [http://ir.ia.ac.cn/handle/173211/58481]  |
| 专题 | 数字内容技术与服务研究中心_听觉模型与认知计算 |
| 通讯作者 | Li, Guoqi |
| 作者单位 | 1.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China 2.Tsinghua Univ, Dept Precis Instrument, Beijing 100084, Peoples R China 3.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China 4.Chinese Acad Sci, Inst Automat, Key Lab Brain Cognit & Brain Inspired Intelligence, Beijing 100190, Peoples R China |
| 推荐引用方式 GB/T 7714 | Su, Qiaoyi,He, Weihua,Wei, Xiaobao,et al. Multi-scale full spike pattern for semantic segmentation[J]. NEURAL NETWORKS,2024,176:12. |
| APA | Su, Qiaoyi,He, Weihua,Wei, Xiaobao,Xu, Bo,&Li, Guoqi.(2024).Multi-scale full spike pattern for semantic segmentation.NEURAL NETWORKS,176,12. |
| MLA | Su, Qiaoyi,et al."Multi-scale full spike pattern for semantic segmentation".NEURAL NETWORKS 176(2024):12. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。