中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
Knowledge matters: Chest radiology report generation with general and specific knowledge

文献类型:期刊论文

作者Yang, Shuxin1,5; Wu, Xian2; Ge, Shen2; Zhou, S. Kevin3,4,5; Xiao, Li1,5
刊名MEDICAL IMAGE ANALYSIS
出版日期2022-08-01
卷号80页码:11
关键词Chest radiology report generation Knowledge graph Multi-head attention
ISSN号1361-8415
DOI10.1016/j.media.2022.102510
英文摘要Automatic chest radiology report generation is critical in clinics which can relieve experienced radiologists from the heavy workload and remind inexperienced radiologists of misdiagnosis or missed diagnose. Existing approaches mainly formulate chest radiology report generation as an image captioning task and adopt the encoder-decoder framework. However, in the medical domain, such pure data-driven approaches suffer from the following problems: 1) visual and textual bias problem; 2) lack of expert knowledge. In this paper, we propose a knowledge-enhanced radiology report generation approach introduces two types of medical knowledge: 1) General knowledge, which is input independent and provides the broad knowledge for report generation; 2) Specific knowledge, which is input dependent and provides the fine-grained knowledge for chest X-ray report generation. To fully utilize both the general and specific knowledge, we also propose a knowledge-enhanced multi-head attention mechanism. By merging the visual features of the radiology image with general knowledge and specific knowledge, the proposed model can improve the quality of generated reports. The experimental results on the publicly available IU-Xray dataset show that the proposed knowledge-enhanced approach outperforms state-of-the-art methods in almost all metrics. And the results of MIMIC-CXR dataset show that the proposed knowledge-enhanced approach is on par with state-of-the-art methods. Ablation studies also demonstrate that both general and specific knowledge can help to improve the performance of chest radiology report generation.(c) 2022 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license( http://creativecommons.org/licenses/by-nc-nd/4.0/ )
资助项目CCF-Tencent Open Fund ; National Natural Science Foundation of China[31900979]
WOS研究方向Computer Science ; Engineering ; Radiology, Nuclear Medicine & Medical Imaging
语种英语
WOS记录号WOS:000871059300006
出版者ELSEVIER
源URL[http://119.78.100.204/handle/2XEOYT63/19768]  
专题中国科学院计算技术研究所期刊论文
通讯作者Wu, Xian; Zhou, S. Kevin; Xiao, Li
作者单位1.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
2.Tencent Med AI Lab, Beijing 100094, Peoples R China
3.Univ Sci & Technol China, Suzhou Inst Adv Res Ctr Med Imaging Robot & Analyt, Suzhou 215123, Peoples R China
4.Univ Sci & Technol China, Sch Biomed Engn, Suzhou 215123, Peoples R China
5.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc Chinese Acad Sci, Beijing 100190, Peoples R China
推荐引用方式
GB/T 7714
Yang, Shuxin,Wu, Xian,Ge, Shen,et al. Knowledge matters: Chest radiology report generation with general and specific knowledge[J]. MEDICAL IMAGE ANALYSIS,2022,80:11.
APA Yang, Shuxin,Wu, Xian,Ge, Shen,Zhou, S. Kevin,&Xiao, Li.(2022).Knowledge matters: Chest radiology report generation with general and specific knowledge.MEDICAL IMAGE ANALYSIS,80,11.
MLA Yang, Shuxin,et al."Knowledge matters: Chest radiology report generation with general and specific knowledge".MEDICAL IMAGE ANALYSIS 80(2022):11.

入库方式: OAI收割

来源:计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。