中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
TIM: An Efficient Temporal Interaction Module for Spiking Transformer

文献类型:会议论文

作者Shen, Sicheng1,3,5; Zhao, Dongcheng1,5; Shen, Guobin1,3,5; Zeng, Yi1,2,3,4,5
出版日期2024
会议日期2024-08
会议地点Jeju, korea
英文摘要

Spiking Neural Networks (SNNs), as the third generation  of neural networks, have gained prominence  for their biological plausibility and computational  efficiency, especially in processing diverse  datasets. The integration of attention mechanisms,  inspired by advancements in neural network architectures,  has led to the development of Spiking Transformers. These have shown promise in  enhancing SNNs’ capabilities, particularly in the  realms of both static and neuromorphic datasets. Despite their progress, a discernible gap exists in  these systems, specifically in the Spiking Self Attention (SSA) mechanism’s effectiveness in leveraging  the temporal processing potential of SNNs. To address this, we introduce the Temporal Interaction Module (TIM), a novel, convolutionbased  enhancement designed to augment the temporal  data processing abilities within SNN architectures. TIM’s integration into existing SNN frameworks  is seamless and efficient, requiring minimal  additional parameters while significantly boosting  their temporal information handling capabilities. Through rigorous experimentation, TIM has  demonstrated its effectiveness in exploiting temporal  information, leading to state-of-the-art performance  across various neuromorphic datasets. The  code is available at https://github.com/BrainCog- X/Brain-Cog/tree/main/examples/TIM.

源URL[http://ir.ia.ac.cn/handle/173211/57254]  
专题类脑智能研究中心_类脑认知计算
通讯作者Zeng, Yi
作者单位1.Brain-inspired Cognitive Intelligence Lab, Institute of Automation, Chinese Academy of Sciences
2.School of Artificial Intelligence, University of Chinese Academy of Sciences
3.School of Future Technology, University of Chinese Academy of Sciences
4.Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, CAS
5.Center for Long-term Artificial Intelligence
推荐引用方式
GB/T 7714
Shen, Sicheng,Zhao, Dongcheng,Shen, Guobin,et al. TIM: An Efficient Temporal Interaction Module for Spiking Transformer[C]. 见:. Jeju, korea. 2024-08.

入库方式: OAI收割

来源:自动化研究所

浏览0
下载0
收藏0
其他版本

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。