中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
Explainability for Large Language Models: A Survey

文献类型：期刊论文

作者Zhao, Haiyan2; Chen, Hanjie3; Yang, Fan4; Liu, Ninghao5; Deng, Huiqi6; Cai, Hengyi7; Wang, Shuaiqiang1; Yin, Dawei1; Du, Mengnan2
刊名ACM TRANSACTIONS ON INTELLIGENT SYSTEMS AND TECHNOLOGY
出版日期2024-04-01
卷号15期号:2页码:38
关键词Explainability interpretability large language models
ISSN号2157-6904
DOI10.1145/3639372
英文摘要Large language models (LLMs) have demonstrated impressive capabilities in natural language processing. However, their internal mechanisms are still unclear and this lack of transparency poses unwanted risks for downstream applications. Therefore, understanding and explaining these models is crucial for elucidating their behaviors, limitations, and social impacts. In this article, we introduce a taxonomy of explainability techniques and provide a structured overview ofmethods for explaining Transformer-based language models. We categorize techniques based on the training paradigms of LLMs: traditional fine-tuning-based paradigm and prompting-based paradigm. For each paradigm, we summarize the goals and dominant approaches for generating local explanations of individual predictions and global explanations of overall model knowledge. We also discuss metrics for evaluating generated explanations and discuss how explanations can be leveraged to debug models and improve performance. Lastly, we examine key challenges and emerging opportunities for explanation techniques in the era of LLMs in comparison to conventional deep learning models.
WOS研究方向Computer Science
语种英语
WOS记录号WOS:001208775700001
出版者ASSOC COMPUTING MACHINERY
源URL[http://119.78.100.204/handle/2XEOYT63/39000]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Zhao, Haiyan
作者单位1.10 Shangdi 10th St, Beijing 100085, Peoples R China
2.New Jersey Inst Technol, 323 Dr Martin Luther King Jr Blvd, Newark, NJ 07102 USA
3.Johns Hopkins Univ, 3400 N Charles St, Baltimore, MD 21218 USA
4.Wake Forest Univ, 1834 Wake Forest Rd, Winston Salem, NC 27109 USA
5.Univ Georgia, Herty Dr, Athens, GA 30602 USA
6.Shanghai Jiao Tong Univ, 800 Dongchuan RD, Shanghai 200240, Peoples R China
7.Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China
推荐引用方式
GB/T 7714		 Zhao, Haiyan,Chen, Hanjie,Yang, Fan,et al. Explainability for Large Language Models: A Survey[J]. ACM TRANSACTIONS ON INTELLIGENT SYSTEMS AND TECHNOLOGY,2024,15(2):38.
APA Zhao, Haiyan.,Chen, Hanjie.,Yang, Fan.,Liu, Ninghao.,Deng, Huiqi.,...&Du, Mengnan.(2024).Explainability for Large Language Models: A Survey.ACM TRANSACTIONS ON INTELLIGENT SYSTEMS AND TECHNOLOGY,15(2),38.
MLA Zhao, Haiyan,et al."Explainability for Large Language Models: A Survey".ACM TRANSACTIONS ON INTELLIGENT SYSTEMS AND TECHNOLOGY 15.2(2024):38.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。