Toward a stable and low-resource PLM-based medical diagnostic system via prompt tuning and MoE structure
文献类型：期刊论文
| 作者 | Dong, Bowen3; Wang, Zhuo3; Li, Zhenyu3; Duan, Zhichao3; Xu, Jiacheng3; Pan, Tengyu3; Zhang, Rui3; Liu, Ning2; Li, Xiuxing4,5; Wang, Jie1 |
| 刊名 | SCIENTIFIC REPORTS
 |
| 出版日期 | 2023-08-03 |
| 卷号 | 13期号:1页码:15 |
| ISSN号 | 2045-2322 |
| DOI | 10.1038/s41598-023-39543-2 |
| 英文摘要 | Machine learning (ML) has been extensively involved in assistant disease diagnosis and prediction systems to emancipate the serious dependence on medical resources and improve healthcare quality. Moreover, with the booming of pre-training language models (PLMs), the application prospect and promotion potential of machine learning methods in the relevant field have been further inspired. PLMs have recently achieved tremendous success in diverse text processing tasks, whereas limited by the significant semantic gap between the pre-training corpus and the structured electronic health records (EHRs), PLMs cannot converge to anticipated disease diagnosis and prediction results. Unfortunately, establishing connections between PLMs and EHRs typically requires the extraction of curated predictor variables from structured EHR resources, which is tedious and labor-intensive, and even discards vast implicit information.In this work, we propose an Input Prompting and Discriminative language model with the Mixture-of-experts framework (IPDM) by promoting the model's capabilities to learn knowledge from heterogeneous information and facilitating the feature-aware ability of the model. Furthermore, leveraging the prompt-tuning mechanism, IPDM can inherit the impacts of the pre-training in downstream tasks exclusively through minor modifications. IPDM remarkably outperforms existing models, proved by experiments on one disease diagnosis task and two disease prediction tasks. Finally, experiments with few-feature and few-sample demonstrate that IPDM achieves significant stability and impressive performance in predicting chronic diseases with unclear early-onset characteristics or sudden diseases with insufficient data, which verifies the superiority of IPDM over existing mainstream methods, and reveals the IPDM can powerfully address the aforementioned challenges via establishing a stable and low-resource medical diagnostic system for various clinical scenarios. |
| 资助项目 | National Key Research and Development Program of China[2020YFA0804503] ; National Key Research and Development Program of China[2020YFA0804501] ; National Natural Science Foundation of China[62272264] ; Beijing Academy of Artificial Intelligence (BAAI) ; Alzheimer's Disease Neuroimaging Initiative (ADNI) (National Institutes of Health)[U01 AG024904] ; DOD ADNI (Department of Defense)[W81XWH-12-2-0012] |
| WOS研究方向 | Science & Technology - Other Topics |
| 语种 | 英语 |
| WOS记录号 | WOS:001042854100035 |
| 出版者 | NATURE PORTFOLIO |
| 源URL | [http://119.78.100.204/handle/2XEOYT63/21177]  |
| 专题 | 中国科学院计算技术研究所期刊论文_英文 |
| 通讯作者 | Wang, Jianyong |
| 作者单位 | 1.Peking Union Med Coll Hosp, Dept Neurol, State Key Lab Complex Severe & Rare Dis, Beijing, Peoples R China 2.Shandong Univ, Sch Software, Jinan, Peoples R China 3.Tsinghua Univ, Dept Comp Sci & Technol, Beijing, Peoples R China 4.Univ Chinese Acad Sci, Beijing, Peoples R China 5.Chinese Acad Sci ICT CAS, Key Lab Intelligent Informat Proc, Inst Comp Technol, Beijing, Peoples R China |
| 推荐引用方式 GB/T 7714 | Dong, Bowen,Wang, Zhuo,Li, Zhenyu,et al. Toward a stable and low-resource PLM-based medical diagnostic system via prompt tuning and MoE structure[J]. SCIENTIFIC REPORTS,2023,13(1):15. |
| APA | Dong, Bowen.,Wang, Zhuo.,Li, Zhenyu.,Duan, Zhichao.,Xu, Jiacheng.,...&Wang, Jianyong.(2023).Toward a stable and low-resource PLM-based medical diagnostic system via prompt tuning and MoE structure.SCIENTIFIC REPORTS,13(1),15. |
| MLA | Dong, Bowen,et al."Toward a stable and low-resource PLM-based medical diagnostic system via prompt tuning and MoE structure".SCIENTIFIC REPORTS 13.1(2023):15. |
入库方式： OAI收割
来源：计算技术研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。