Neural Encoding for Human Visual Cortex With Deep Neural Networks Learning "What" and "Where"
文献类型：期刊论文
| 作者 | Wang, Haibao2,3 ; Huang, Lijie2,3; Du, Changde2,3; Li, Dan2,3; Wang, Bo2,3; He, Huiguang1,2,3
|
| 刊名 | IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS
 |
| 出版日期 | 2021-12-01 |
| 卷号 | 13期号:4页码:827-840 |
| 关键词 | Visualization Feature extraction Encoding Brain modeling Biological neural networks Sociology Statistics Deep neural network (DNN) neural encoding regularization "what" and "where" |
| ISSN号 | 2379-8920 |
| DOI | 10.1109/TCDS.2020.3007761 |
| 通讯作者 | He, Huiguang(huiguang.he@ia.ac.cn) |
| 英文摘要 | Neural encoding, a crucial aspect to understand the human brain information processing system, aims to establish a quantitative relationship between the stimuli and the evoked brain activities. In the field of visual neuroscience, with the ability to explain how neurons in the primary visual cortex work, population receptive field (pRF) models have enjoyed high popularity and made reliable progress in recent years. However, existing models rely on either the inflexible prior assumptions about pRF or the clumsy parameter estimation methods, severely limiting the expressiveness and interpretability. In this article, we propose a novel neural encoding framework by learning "what" and "where" with deep neural networks. It involves two separate aspects: 1) the spatial characteristic ("where") and 2) feature selection ("what") of neuron populations in the visual cortex. Specifically, our approach first encodes visual stimuli into hierarchically intermediate features through a pretrained deep neural network (DNN), then converts DNN features into refined features with the channel attention and spatial receptive field (RF) to learn "where", and finally regresses refined features simultaneously onto voxel activities to learn "what". The sparsity regularization and smoothness regularization are adopted in our modeling approach so that the crucial RF can be estimated automatically without prior assumptions about shapes. Furthermore, an attempt is made to extend the voxel-wise modeling approach to multi-voxel joint encoding models, and we show that it is conducive to rescuing voxels with poor signal-to-noise characteristics. Extensive empirical results demonstrate that the method developed herein provides an effective strategy to establish neural encoding for the human visual cortex, with the weaker prior constraints but the higher encoding performance. |
| WOS关键词 | RECEPTIVE-FIELD ; REPRESENTATION ; ATTENTION ; IMAGES ; MODELS |
| 资助项目 | National Natural Science Foundation of China[61976209] ; National Natural Science Foundation of China[61906188] ; CAS International Collaboration Key Project[173211KYSB20190024] ; Strategic Priority Research Program of CAS[XDB32040000] |
| WOS研究方向 | Computer Science ; Robotics ; Neurosciences & Neurology |
| 语种 | 英语 |
| WOS记录号 | WOS:000728925200012 |
| 出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| 资助机构 | National Natural Science Foundation of China ; CAS International Collaboration Key Project ; Strategic Priority Research Program of CAS |
| 源URL | [http://ir.ia.ac.cn/handle/173211/46767]  |
| 专题 | 类脑智能研究中心_神经计算及脑机交互 |
| 通讯作者 | He, Huiguang |
| 作者单位 | 1.Chinese Acad Sci, Ctr Excellence Brain Sci & Intelligence Technol, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China 3.Chinese Acad Sci, Res Ctr Brain Inspired Intelligence, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China |
| 推荐引用方式 GB/T 7714 | Wang, Haibao,Huang, Lijie,Du, Changde,et al. Neural Encoding for Human Visual Cortex With Deep Neural Networks Learning "What" and "Where"[J]. IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS,2021,13(4):827-840. |
| APA | Wang, Haibao,Huang, Lijie,Du, Changde,Li, Dan,Wang, Bo,&He, Huiguang.(2021).Neural Encoding for Human Visual Cortex With Deep Neural Networks Learning "What" and "Where".IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS,13(4),827-840. |
| MLA | Wang, Haibao,et al."Neural Encoding for Human Visual Cortex With Deep Neural Networks Learning "What" and "Where"".IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS 13.4(2021):827-840. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。