中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 合肥物质科学研究院
  3. 中国科学院合肥物质科学研究院
History Reuse and Bag-of-Words Loss for Long Summary Generation

文献类型：期刊论文

作者Liu, Qing1,2; Chen, Lei1; Yuan, Yuan1; Wu, Huarui3
刊名IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING
出版日期2021
卷号29
关键词History Decoding Training Predictive models Postal services Computational modeling Vocabulary Abstractive summarization long summaries history reuse bag-of-words word order deviation
ISSN号2329-9290
DOI10.1109/TASLP.2021.3100281
通讯作者Chen, Lei(chenlei@iim.ac.cn) ; Wu, Huarui(wuhr@nercita.org.cn)
英文摘要Recurrent Neural Network (RNN) based abstractive text summarization models have made great progress over the past few years, largely triggered by the encoder-decoder architecture. However, there has been little work improving the generation of relatively long summaries. In this paper, we concentrate on two prominent problems in long summary generation. First, although significant efforts have been made to assist the encoder in handling long sequences, the decoder struggles with long sequences owing to the limited storage capacity of RNN. We propose a simple and effective approach called history reuse, which first mines critical information from the history summary sequence and then transmits the information to the decoder. Second, since encoder-decoder models are typically trained to produce exactly the same summary as the target summary, certain word order deviations between the predicted summary and target summary are excessively punished. Accordingly, we introduce a fully differentiable loss called bag-of-words (BoW) loss, which takes advantage of the feature of BoW discarding word order information in texts, and computes the difference between the two summaries at the BoW space. Experiments on two benchmark datasets, CNN/Daily Mail and Pubmed, demonstrate that our methods significantly improve the baseline.
资助项目Beijing Municipal Science and Technology Project[Z191100004019007] ; National Natural Science Foundation of China[31771677] ; National Natural Science Foundation of China[32071901]
WOS研究方向Acoustics ; Engineering
语种英语
WOS记录号WOS:000684688000001
出版者IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
资助机构Beijing Municipal Science and Technology Project ; National Natural Science Foundation of China
源URL[http://ir.hfcas.ac.cn:8080/handle/334002/124425]  
专题中国科学院合肥物质科学研究院
通讯作者Chen, Lei; Wu, Huarui
作者单位1.Chinese Acad Sci, HFIPS, Inst Intelligent Machines, Hefei 230031, Peoples R China
2.Univ Sci & Technol China, Hefei 230026, Peoples R China
3.Beijing Res Ctr Informat Technol Agr, Beijing 100097, Peoples R China
推荐引用方式
GB/T 7714		 Liu, Qing,Chen, Lei,Yuan, Yuan,et al. History Reuse and Bag-of-Words Loss for Long Summary Generation[J]. IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING,2021,29.
APA Liu, Qing,Chen, Lei,Yuan, Yuan,&Wu, Huarui.(2021).History Reuse and Bag-of-Words Loss for Long Summary Generation.IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING,29.
MLA Liu, Qing,et al."History Reuse and Bag-of-Words Loss for Long Summary Generation".IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING 29(2021).

入库方式： OAI收割

来源：合肥物质科学研究院

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。