Introspection unit in memory network: Learning to generalize inference in OOV scenarios
文献类型：期刊论文
| 作者 | Yang, Qichuan2; He, Zhiqiang1,2,3; Zhan, Zhiqiang1; Zhang, Yang3; Li, Rang3; Hu, Changjian3 |
| 刊名 | NEUROCOMPUTING
 |
| 出版日期 | 2020-02-28 |
| 卷号 | 379页码:30-40 |
| 关键词 | Memory network Out of vocabulary Introspection unit Language inference |
| ISSN号 | 0925-2312 |
| DOI | 10.1016/j.neucom.2019.07.111 |
| 英文摘要 | Inference in natural language processing (NLP) is a tough task. Although plenty of models have been proposed in recent years, they are usually restricted to infer within a limited vocabulary or handcrafted training templates. In this paper, we propose the introspection unit (IU), a new neural module which can be incorporated with memory networks to deal with inference tasks in out of vocabulary (OOV) and rare named entities (RNEs) scenarios. Specifically, when encountering a new word, IU compares its part-of-speech context with the training dataset to extract a similar sample, and then embeds the new word into a target position to construct a simulated sample. The target position is located by the result of part-of-speech tagging. Finally, using the simulated sample, IU helps memory networks to learn the context and characteristic of the new word. In experiments, we evaluate the effectiveness of IU with the memory network on four inference datasets: a name OOV dataset, a place OOV dataset, a more challenging synthetical mixture OOV dataset and a realistic dialogue dataset. The experimental results demonstrate that IU effectively generalizes the inference ability of memory networks to OOV scenarios and improves the inference accuracies significantly. Furthermore, we visualize both the introspection process and the effect of IU in word embeddings and memories. (C) 2019 Elsevier B.V. All rights reserved. |
| WOS研究方向 | Computer Science |
| 语种 | 英语 |
| WOS记录号 | WOS:000507464700003 |
| 出版者 | ELSEVIER |
| 源URL | [http://119.78.100.204/handle/2XEOYT63/15002]  |
| 专题 | 中国科学院计算技术研究所期刊论文_英文 |
| 通讯作者 | He, Zhiqiang; Zhang, Yang |
| 作者单位 | 1.Chinese Acad Sci, Univ Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China 2.Beihang Univ, Sch Comp Sci & Engn, Beijing, Peoples R China 3.Lenovo Ltd, Lenovo Corp Res & Dev, Beijing, Peoples R China |
| 推荐引用方式 GB/T 7714 | Yang, Qichuan,He, Zhiqiang,Zhan, Zhiqiang,et al. Introspection unit in memory network: Learning to generalize inference in OOV scenarios[J]. NEUROCOMPUTING,2020,379:30-40. |
| APA | Yang, Qichuan,He, Zhiqiang,Zhan, Zhiqiang,Zhang, Yang,Li, Rang,&Hu, Changjian.(2020).Introspection unit in memory network: Learning to generalize inference in OOV scenarios.NEUROCOMPUTING,379,30-40. |
| MLA | Yang, Qichuan,et al."Introspection unit in memory network: Learning to generalize inference in OOV scenarios".NEUROCOMPUTING 379(2020):30-40. |
入库方式： OAI收割
来源：计算技术研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。