TIM: An Efficient Temporal Interaction Module for Spiking Transformer
文献类型:会议论文
作者 | Shen, Sicheng1,3,5; Zhao, Dongcheng1,5![]() ![]() |
出版日期 | 2024 |
会议日期 | 2024-08 |
会议地点 | Jeju, korea |
英文摘要 | Spiking Neural Networks (SNNs), as the third generation of neural networks, have gained prominence for their biological plausibility and computational efficiency, especially in processing diverse datasets. The integration of attention mechanisms, inspired by advancements in neural network architectures, has led to the development of Spiking Transformers. These have shown promise in enhancing SNNs’ capabilities, particularly in the realms of both static and neuromorphic datasets. Despite their progress, a discernible gap exists in these systems, specifically in the Spiking Self Attention (SSA) mechanism’s effectiveness in leveraging the temporal processing potential of SNNs. To address this, we introduce the Temporal Interaction Module (TIM), a novel, convolutionbased enhancement designed to augment the temporal data processing abilities within SNN architectures. TIM’s integration into existing SNN frameworks is seamless and efficient, requiring minimal additional parameters while significantly boosting their temporal information handling capabilities. Through rigorous experimentation, TIM has demonstrated its effectiveness in exploiting temporal information, leading to state-of-the-art performance across various neuromorphic datasets. The code is available at https://github.com/BrainCog- X/Brain-Cog/tree/main/examples/TIM. |
源URL | [http://ir.ia.ac.cn/handle/173211/57254] ![]() |
专题 | 类脑智能研究中心_类脑认知计算 |
通讯作者 | Zeng, Yi |
作者单位 | 1.Brain-inspired Cognitive Intelligence Lab, Institute of Automation, Chinese Academy of Sciences 2.School of Artificial Intelligence, University of Chinese Academy of Sciences 3.School of Future Technology, University of Chinese Academy of Sciences 4.Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, CAS 5.Center for Long-term Artificial Intelligence |
推荐引用方式 GB/T 7714 | Shen, Sicheng,Zhao, Dongcheng,Shen, Guobin,et al. TIM: An Efficient Temporal Interaction Module for Spiking Transformer[C]. 见:. Jeju, korea. 2024-08. |
入库方式: OAI收割
来源:自动化研究所
其他版本
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。