Adaptive Spatial Location With Balanced Loss for Video Captioning
文献类型：期刊论文
| 作者 | Li, Linghui1; Zhang, Yongdong2; Tang, Sheng2; Xie, Lingxi3; Li, Xiaoyong1; Tian, Qi3 |
| 刊名 | IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
 |
| 出版日期 | 2022 |
| 卷号 | 32期号:1页码:17-30 |
| 关键词 | Task analysis Redundancy Feature extraction Visualization Detectors Computer vision Training Convolutional neural network recurrent neural network video captioning adaptive spatial location balanced loss |
| ISSN号 | 1051-8215 |
| DOI | 10.1109/TCSVT.2020.3045735 |
| 英文摘要 | Many pioneering approaches have verified the effectiveness of utilizing the global temporal and local object information for video understanding tasks and have achieved significant progress. However, existing methods utilize object detectors to extract all objects overall video frames. This may bring performance degradation due to the information redundancy both spatially and temporally. To address this problem, we propose an adaptive spatial location module for the video captioning task which dynamically predicts an important position of each video frame in the procedure of generating the description sentence. The proposed adaptive spatial location method not only makes our model focus on local object information, but also reduces time and memory consumption brought by the temporal redundancy in extensive video frames and improves the accuracy of generated description. Besides, we propose a balanced loss function to address the class imbalance problem existing in training data. The proposed balanced loss assigns different weight to each word of ground-truth sentence in the training process which can generate more diversified description sentences. Extensive experimental results on the MSVD and MSR-VTT dataset show that the proposed method achieves competitive performance compared to state-of-the-art methods. |
| 资助项目 | National Nature Science Foundation of China[61525206] ; National Nature Science Foundation of China[61871004] ; 242 Project[2020A077] ; National Natural Science Foundation of China (NSFC)-General Technology Fundamental Research Joint Fund[U1836215] ; China Postdoctoral Science Foundation[2020TQ0055] ; Foundation of Key Laboratory of Trustworthy Distributed Computing and Service (BUPT), Ministry of Education |
| WOS研究方向 | Engineering |
| 语种 | 英语 |
| WOS记录号 | WOS:000742183600006 |
| 出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| 源URL | [http://119.78.100.204/handle/2XEOYT63/18257]  |
| 专题 | 中国科学院计算技术研究所期刊论文_英文 |
| 通讯作者 | Zhang, Yongdong |
| 作者单位 | 1.Beijing Univ Posts & Telecommun, Key Lab Trustworthy Distributed Comp & Serv, Minist Educ, Beijing 100876, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 3.Huawei Technol Inc, Shenzhen 518129, Peoples R China |
| 推荐引用方式 GB/T 7714 | Li, Linghui,Zhang, Yongdong,Tang, Sheng,et al. Adaptive Spatial Location With Balanced Loss for Video Captioning[J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,2022,32(1):17-30. |
| APA | Li, Linghui,Zhang, Yongdong,Tang, Sheng,Xie, Lingxi,Li, Xiaoyong,&Tian, Qi.(2022).Adaptive Spatial Location With Balanced Loss for Video Captioning.IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,32(1),17-30. |
| MLA | Li, Linghui,et al."Adaptive Spatial Location With Balanced Loss for Video Captioning".IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 32.1(2022):17-30. |
入库方式： OAI收割
来源：计算技术研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。