Incorporating NODE with pre-trained neural differential operator for learning dynamics
文献类型：期刊论文
| 作者 | Gong, Shiqi3; Meng, Qi2; Wang, Yue2; Wu, Lijun2; Chen, Wei1; Ma, Zhiming3; Liu, Tie-Yan2 |
| 刊名 | NEUROCOMPUTING
 |
| 出版日期 | 2023-04-01 |
| 卷号 | 528页码:48-58 |
| 关键词 | Neural ODE Learning dynamics Neural operator |
| ISSN号 | 0925-2312 |
| DOI | 10.1016/j.neucom.2023.01.040 |
| 英文摘要 | Learning dynamics governed by differential equations is crucial for predicting and controlling the sys-tems in science and engineering. Neural Ordinary Differential Equation (NODE), a deep learning model integrated with differential equations, is popular in learning dynamics recently due to its robustness to irregular samples and its flexibility to high-dimensional input. However, the training of NODE is sen-sitive to the precision of the numerical solver, which makes the convergence of NODE unstable, especially for ill-conditioned dynamical systems. In this paper, to reduce the reliance on the numerical solver, we propose to enhance the supervised signal in the training of NODE. Specifically, we pre-train a neural dif-ferential operator (NDO) to output an estimation of the derivatives to serve as an additional supervised signal. The NDO is pre-trained on a class of basis functions and learns the mapping between the trajectory samples of these functions to their derivatives. To leverage both the trajectory signal and the estimated derivatives from NDO, we propose an algorithm called NDO-NODE, in which the loss function contains two terms: the fitness on the true trajectory samples and the fitness on the estimated derivatives that are outputted by the pre-trained NDO. Experiments on various kinds of dynamics show that our proposed NDO-NODE can consistently improve the forecasting accuracy with one pre-trained NDO. Especially for the stiff ODEs, we observe that NDO-NODE can capture the transitions in the dynamics more accurately compared with other regularization methods.(c) 2023 Elsevier B.V. All rights reserved. |
| WOS研究方向 | Computer Science |
| 语种 | 英语 |
| WOS记录号 | WOS:000923797800001 |
| 出版者 | ELSEVIER |
| 源URL | [http://119.78.100.204/handle/2XEOYT63/19938]  |
| 专题 | 中国科学院计算技术研究所期刊论文 |
| 通讯作者 | Meng, Qi |
| 作者单位 | 1.Chinese Acad Sci, Inst Comp Technol, 6 Kexueyuan South Rd, Beijing 100190, Peoples R China 2.Microsoft Res, AI4Science, Beijing 100080, Peoples R China 3.Chinese Acad Sci, Acad Math & Syst Sci, Zhongguancun East Rd, Beijing 100190, Peoples R China |
| 推荐引用方式 GB/T 7714 | Gong, Shiqi,Meng, Qi,Wang, Yue,et al. Incorporating NODE with pre-trained neural differential operator for learning dynamics[J]. NEUROCOMPUTING,2023,528:48-58. |
| APA | Gong, Shiqi.,Meng, Qi.,Wang, Yue.,Wu, Lijun.,Chen, Wei.,...&Liu, Tie-Yan.(2023).Incorporating NODE with pre-trained neural differential operator for learning dynamics.NEUROCOMPUTING,528,48-58. |
| MLA | Gong, Shiqi,et al."Incorporating NODE with pre-trained neural differential operator for learning dynamics".NEUROCOMPUTING 528(2023):48-58. |
入库方式： OAI收割
来源：计算技术研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。