中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
Refocused Attention: Long Short-Term Rewards Guided Video Captioning

文献类型：期刊论文

作者Dong, Jiarong1,2; Gao, Ke1; Chen, Xiaokai1,2; Cao, Juan1
刊名NEURAL PROCESSING LETTERS
出版日期2020-10-01
卷号52期号:2页码:935-948
关键词Video captioning Hierarchical attention Reinforcement learning Reward
ISSN号1370-4621
DOI10.1007/s11063-019-10030-y
英文摘要The adaptive cooperation of visual model and language model is essential for video captioning. However, due to the lack of proper guidance for each time step in end-to-end training, the over-dependence of language model often results in the invalidation of attention-based visual model, which is called 'Attention Defocus' problem in this paper. Based on an important observation that the recognition precision of entity word can reflect the effectiveness of the visual model, we propose a novel strategy called refocused attention to optimize the training and cooperating of visual model and language model, using ingenious guidance at appropriate time step. The strategy consists of a short-term-reward guided local entity recognition and a long-term-reward guided global relation understanding, neither requires any external training data. Moreover, a framework with hierarchical visual representations and hierarchical attention is established to fully exploit the potential strength of the proposed learning strategy. Extensive experiments demonstrate that the ingenious guidance strategy together with the optimized structure outperform state-of-the-art video captioning methods with relative improvements 7.7% in BLEU-4 and 5.0% in CIDEr-D on MSVD dataset, even without multi-modal features.
WOS研究方向Computer Science
语种英语
WOS记录号WOS:000574296700001
出版者SPRINGER
源URL[http://119.78.100.204/handle/2XEOYT63/15637]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Gao, Ke
作者单位1.Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China
2.Univ Chinese Acad Sci, Beijing, Peoples R China
推荐引用方式
GB/T 7714		 Dong, Jiarong,Gao, Ke,Chen, Xiaokai,et al. Refocused Attention: Long Short-Term Rewards Guided Video Captioning[J]. NEURAL PROCESSING LETTERS,2020,52(2):935-948.
APA Dong, Jiarong,Gao, Ke,Chen, Xiaokai,&Cao, Juan.(2020).Refocused Attention: Long Short-Term Rewards Guided Video Captioning.NEURAL PROCESSING LETTERS,52(2),935-948.
MLA Dong, Jiarong,et al."Refocused Attention: Long Short-Term Rewards Guided Video Captioning".NEURAL PROCESSING LETTERS 52.2(2020):935-948.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。