Towards understanding residual and dilated dense neural networks via convolutional sparse coding
文献类型：期刊论文
| 作者 | Zhang, Zhiyang3,4; Zhang, Shihua1,2,3,4
|
| 刊名 | NATIONAL SCIENCE REVIEW
 |
| 出版日期 | 2021-03-01 |
| 卷号 | 8期号:3页码:13 |
| 关键词 | convolutional neural network convolutional sparse coding residual neural network mixed-scale dense neural network dilated convolution dense connection |
| ISSN号 | 2095-5138 |
| DOI | 10.1093/nsr/nwaa159 |
| 英文摘要 | Convolutional neural network (CNN) and its variants have led to many state-of-the-art results in various fields. However, a clear theoretical understanding of such networks is still lacking. Recently, a multilayer convolutional sparse coding (ML-CSC) model has been proposed and proved to equal such simply stacked networks (plain networks). Here, we consider the initialization, the dictionary design and the number of iterations to be factors in each layer that greatly affect the performance of the ML-CSC model. Inspired by these considerations, we propose two novel multilayer models: the residual convolutional sparse coding (Res-CSC) model and the mixed-scale dense convolutional sparse coding (MSD-CSC) model. They are closely related to the residual neural network (ResNet) and the mixed-scale (dilated) dense neural network (MSDNet), respectively. Mathematically, we derive the skip connection in the ResNet as a special case of a new forward propagation rule for the ML-CSC model. We also find a theoretical interpretation of dilated convolution and dense connection in the MSDNet by analyzing the MSD-CSC model, which gives a clear mathematical understanding of each. We implement the iterative soft thresholding algorithm and its fast version to solve the Res-CSC and MSD-CSC models. The unfolding operation can be employed for further improvement. Finally, extensive numerical experiments and comparison with competing methods demonstrate their effectiveness. |
| 资助项目 | National Key Research and Development Program of China[2019YFA0709501] ; National Natural Science Foundation of China[11661141019] ; National Natural Science Foundation of China[61621003] ; National Ten Thousand Talent Program for Young Top-notch Talents ; CAS Frontier Science Research Key Project for Top Young Scientists[QYZDB-SSW-SYS008] |
| WOS研究方向 | Science & Technology - Other Topics |
| 语种 | 英语 |
| WOS记录号 | WOS:000649002700009 |
| 出版者 | OXFORD UNIV PRESS |
| 源URL | [http://ir.amss.ac.cn/handle/2S8OKBNM/58731]  |
| 专题 | 应用数学研究所 |
| 通讯作者 | Zhang, Shihua |
| 作者单位 | 1.Univ Chinese Acad Sci, Chinese Acad Sci, Hangzhou Inst Adv Study, Key Lab Syst Biol, Hangzhou 310024, Peoples R China 2.Chinese Acad Sci, Ctr Excellence Anim Evolut & Genet, Kunming 650223, Yunnan, Peoples R China 3.Univ Chinese Acad Sci, Sch Math Sci, Beijing 100049, Peoples R China 4.Chinese Acad Sci, Acad Math & Syst Sci, Natl Ctr Math & Interdisciplinary Sci, Ctr Excellence Math Sci,Key Lab Random Complex St, Beijing 100190, Peoples R China |
| 推荐引用方式 GB/T 7714 | Zhang, Zhiyang,Zhang, Shihua. Towards understanding residual and dilated dense neural networks via convolutional sparse coding[J]. NATIONAL SCIENCE REVIEW,2021,8(3):13. |
| APA | Zhang, Zhiyang,&Zhang, Shihua.(2021).Towards understanding residual and dilated dense neural networks via convolutional sparse coding.NATIONAL SCIENCE REVIEW,8(3),13. |
| MLA | Zhang, Zhiyang,et al."Towards understanding residual and dilated dense neural networks via convolutional sparse coding".NATIONAL SCIENCE REVIEW 8.3(2021):13. |
入库方式： OAI收割
来源：数学与系统科学研究院
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。