中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
Radiology report generation with a learned knowledge base and multi-modal alignment

文献类型：期刊论文

作者Yang, Shuxin5,6; Wu, Xian1; Ge, Shen1; Zheng, Zhuozhao2,4; Zhou, S. Kevin3,6; Xiao, Li5,6,7
刊名MEDICAL IMAGE ANALYSIS
出版日期2023-05-01
卷号86页码:10
ISSN号1361-8415
关键词Radiology report generation Knowledge base Multi-modal alignment
DOI10.1016/j.media.2023.102798
英文摘要In clinics, a radiology report is crucial for guiding a patient's treatment. However, writing radiology reports is a heavy burden for radiologists. To this end, we present an automatic, multi-modal approach for report generation from a chest x-ray. Our approach, motivated by the observation that the descriptions in radiology reports are highly correlated with specific information of the x-ray images, features two distinct modules: (i) Learned knowledge base: To absorb the knowledge embedded in the radiology reports, we build a knowledge base that can automatically distill and restore medical knowledge from textual embedding without manual labor; (ii) Multi-modal alignment: to promote the semantic alignment among reports, disease labels, and images, we explicitly utilize textual embedding to guide the learning of the visual feature space. We evaluate the performance of the proposed model using metrics from both natural language generation and clinic efficacy on the public IU-Xray and MIMIC-CXR datasets. Our ablation study shows that each module contributes to improving the quality of generated reports. Furthermore, the assistance of both modules, our approach outperforms state-of-the-art methods over almost all the metrics. Code is available at https://github.com/LX-doctorAI1/M2KT.
资助项目Open Fund Project of Guangdong Academy of Medical Sciences, China[YKY-KF202206] ; [62271465]
WOS研究方向Computer Science ; Engineering ; Radiology, Nuclear Medicine & Medical Imaging
语种英语
出版者ELSEVIER
WOS记录号WOS:001054264400001
源URL[http://119.78.100.204/handle/2XEOYT63/21381]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Xiao, Li
作者单位1.Tencent Med AI Lab, Beijing, Peoples R China
2.Beijing Tsinghua Changgung Hosp, Dept Radiol, Beijing 102218, Peoples R China
3.Univ Sci & Technol China, Sch Biomed Engn Suzhou Inst Adv Res, Ctr Med Imaging Robot & Analyt Comp & LEarning MIR, Suzhou 215123, Peoples R China
4.Tsinghua Univ, Sch Med, Beijing 100084, Peoples R China
5.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
6.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China
7.Beijing Univ Posts & Telecommun, Sch Artificial Intelligence, Beijing 100876, Peoples R China
推荐引用方式
GB/T 7714		 Yang, Shuxin,Wu, Xian,Ge, Shen,et al. Radiology report generation with a learned knowledge base and multi-modal alignment[J]. MEDICAL IMAGE ANALYSIS,2023,86:10.
APA Yang, Shuxin,Wu, Xian,Ge, Shen,Zheng, Zhuozhao,Zhou, S. Kevin,&Xiao, Li.(2023).Radiology report generation with a learned knowledge base and multi-modal alignment.MEDICAL IMAGE ANALYSIS,86,10.
MLA Yang, Shuxin,et al."Radiology report generation with a learned knowledge base and multi-modal alignment".MEDICAL IMAGE ANALYSIS 86(2023):10.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。