中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 自动化研究所
  3. 中国科学院自动化研究所
  4. 脑图谱与类脑智能实验室
  5. 类脑认知计算
Dynamically Optimizing Network Structure Based on Synaptic Pruning in the Brain

文献类型：期刊论文

作者Zhao, Feifei1,2; Zeng, Yi1,2,3,4,5
刊名FRONTIERS IN SYSTEMS NEUROSCIENCE
出版日期2021-06-04
卷号15页码:8
关键词synaptic pruning developmental neural network optimizing network structure accelerating learning compressing network
DOI10.3389/fnsys.2021.620558
通讯作者Zeng, Yi(yi.zeng@ia.ac.cn)
英文摘要Most neural networks need to predefine the network architecture empirically, which may cause over-fitting or under-fitting. Besides, a large number of parameters in a fully connected network leads to the prohibitively expensive computational cost and storage overhead, which makes the model hard to be deployed on mobile devices. Dynamically optimizing the network architecture by pruning unused synapses is a promising technique for solving this problem. Most existing pruning methods focus on reducing the redundancy of deep convolutional neural networks by pruning unimportant filters or weights, at the cost of accuracy drop. In this paper, we propose an effective brain-inspired synaptic pruning method to dynamically modulate the network architecture and simultaneously improve network performance. The proposed model is biologically inspired as it dynamically eliminates redundant connections based on the synaptic pruning rules used during the brain's development. Connections are pruned if they are not activated or less activated multiple times consecutively. Extensive experiments demonstrate the effectiveness of our method on classification tasks of different complexity with the MNIST, Fashion MNIST, and CIFAR-10 datasets. Experimental results reveal that even for a compact network, the proposed method can also remove up to 59-90% of the connections, with relative improvement in learning speed and accuracy.
WOS关键词NEURAL-NETWORKS ; PLASTICITY ; RECOGNITION ; MEMORY
资助项目National Key Research and Development Program[2020AAA0104305] ; Strategic Priority Research Program of the Chinese Academy of Sciences[XDB32070100] ; Beijing Municipal Commission of Science and Technology[Z181100001518006] ; Key Research Program of Frontier Sciences, Chinese Academy of Sciences[ZDBS-LY-JSC013] ; Beijing Academy of Artificial Intelligence
WOS研究方向Neurosciences & Neurology
语种英语
WOS记录号WOS:000662925300001
出版者FRONTIERS MEDIA SA
资助机构National Key Research and Development Program ; Strategic Priority Research Program of the Chinese Academy of Sciences ; Beijing Municipal Commission of Science and Technology ; Key Research Program of Frontier Sciences, Chinese Academy of Sciences ; Beijing Academy of Artificial Intelligence
源URL[http://ir.ia.ac.cn/handle/173211/45337]  
专题类脑智能研究中心_类脑认知计算
通讯作者Zeng, Yi
作者单位1.Chinese Acad Sci, Res Ctr Brain Inspired Intelligence, Inst Automat, Beijing, Peoples R China
2.Chinese Acad Sci, Inst Automat, Beijing, Peoples R China
3.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing, Peoples R China
4.Chinese Acad Sci, Ctr Excellence Brain Sci & Intelligence Technol, Shanghai, Peoples R China
5.Univ Chinese Acad Sci, Beijing, Peoples R China
推荐引用方式
GB/T 7714		 Zhao, Feifei,Zeng, Yi. Dynamically Optimizing Network Structure Based on Synaptic Pruning in the Brain[J]. FRONTIERS IN SYSTEMS NEUROSCIENCE,2021,15:8.
APA Zhao, Feifei,&Zeng, Yi.(2021).Dynamically Optimizing Network Structure Based on Synaptic Pruning in the Brain.FRONTIERS IN SYSTEMS NEUROSCIENCE,15,8.
MLA Zhao, Feifei,et al."Dynamically Optimizing Network Structure Based on Synaptic Pruning in the Brain".FRONTIERS IN SYSTEMS NEUROSCIENCE 15(2021):8.

入库方式： OAI收割

来源：自动化研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。