中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 自动化研究所
  3. 中国科学院自动化研究所
  4. 多模态人工智能系统全国重点实验室
  5. 先进控制与自动化团队
Enhancing Representation Power of Deep Neural Networks With Negligible Parameter Growth for Industrial Applications

文献类型：期刊论文

作者Chen, Liangming1,2; Jin, Long1,2; Shang, Mingsheng2,3; Wang, Fei-Yue4
刊名IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS
出版日期2024-05-01
页码12
关键词Residual neural networks Vectors Noise Biological neural networks Medical services Defect detection Agriculture Deep neural networks (DNNs) industrial applications ordinary differential equation (ODE) representation power
ISSN号2168-2216
DOI10.1109/TSMC.2024.3387408
通讯作者Jin, Long(jinlongsysu@foxmail.com)
英文摘要In industrial applications where computational resources are finite and data noises are prevalent, the representation power of deep neural networks (DNNs) is crucial. Traditional network structures often require a significant increase in the parameter amount to enhance the representation power, making it difficult to achieve effective representation under parameter amount constraints. In order to alleviate this problem, this work leverages the ordinary differential equation (ODE) interpretation of deep residual networks, elucidating the relationship between the fine-grained connectivity modes of blocks in DNNs and the representation power. We build a bridge from the order of numerical methods and the order of ODEs to the representation power of DNNs. Besides, we show that higher-order ODEs can be approximated by k -step methods incorporating trainable coefficients. Empirically, we validate our theoretical insights by demonstrating the superior representation power of our proposed network structures through enhanced performance on industrial tasks, such as surface defect detection, critical temperature prediction of superconductors, and image classification under noises. The proposed method provides a new approach to the design of network structures for robust and accurate DNNs, enhancing the representation power with a negligible number of additional parameters. The code is publicly available at https://github.com/LongJin-lab/Order-and-Representation-Power.
WOS关键词APPROXIMATION
资助项目National Natural Science Foundation of China
WOS研究方向Automation & Control Systems ; Computer Science
语种英语
WOS记录号WOS:001214317700001
出版者IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
资助机构National Natural Science Foundation of China
源URL[http://ir.ia.ac.cn/handle/173211/57014]  
专题自动化研究所_复杂系统管理与控制国家重点实验室_先进控制与自动化团队
通讯作者Jin, Long
作者单位1.Lanzhou Univ, Sch Informat Sci & Engn, Lanzhou 730000, Peoples R China
2.Chinese Acad Sci, Chongqing Inst Green & Intelligent Technol, Chongqing Key Lab Big Data & Intelligent Comp, Chongqing 400714, Peoples R China
3.Univ Chinese Acad Sci, Chongqing Sch, Chongqing 400714, Peoples R China
4.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China
推荐引用方式
GB/T 7714		 Chen, Liangming,Jin, Long,Shang, Mingsheng,et al. Enhancing Representation Power of Deep Neural Networks With Negligible Parameter Growth for Industrial Applications[J]. IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS,2024:12.
APA Chen, Liangming,Jin, Long,Shang, Mingsheng,&Wang, Fei-Yue.(2024).Enhancing Representation Power of Deep Neural Networks With Negligible Parameter Growth for Industrial Applications.IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS,12.
MLA Chen, Liangming,et al."Enhancing Representation Power of Deep Neural Networks With Negligible Parameter Growth for Industrial Applications".IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS (2024):12.

入库方式： OAI收割

来源：自动化研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。