Learning explicit video attributes from mid-level representation for video captioning
文献类型：期刊论文
| 作者 | Fudong Nian; Teng Li; Yan Wang; Xinyu Wu; Bingbing Ni; Changsheng Xu |
| 刊名 | Computer Vision and Image Understanding
 |
| 出版日期 | 2017 |
| 文献子类 | 期刊论文 |
| 英文摘要 | Recent works on video captioning mainly learn the map from low-level visual features to language description directly without explicitly representing the high-level semantic video concepts (e.g. objects, actions in the annotated sentences). To bridge the semantic gap, in this paper, addressing it, we propose a novel video attribute representation learning algorithm for video concept understanding and utilize the learned explicit video attribute representation to improve video captioning performance. To achieve it, firstly, inspired by the success of spectrogram in audio processing, a novel mid-level video representation named “video response map” (VRM) is proposed, by which the frame sequence could be represented by a single image representation. Therefore, the video attributes representation learning could be converted to a well-studied multi-label image classification problem. Then in the captions prediction step, the learned video attributes feature is integrated with the single frame feature to improve previous sequence-to-sequence language generation model by adjusting the LSTM (Long-Short Term Memory) input units. The proposed video captioning framework could both handle variable frame inputs and utilize high-level semantic video attribute features. Experimental results on video captioning tasks show that the proposed method, utilizing only RGB frames as input without extra video or text training data, could achieve competitive performance with state-of-the-art methods. Furthermore, the extensive experimental evaluations on the UCF-101 action classification benchmark well demonstrate the representation capability of the proposed VRM. |
| URL标识 | 查看原文 |
| 语种 | 英语 |
| WOS记录号 | WOS:000418726800011 |
| 源URL | [http://ir.siat.ac.cn:8080/handle/172644/11729]  |
| 专题 | 深圳先进技术研究院_集成所 |
| 作者单位 | Computer Vision and Image Understanding |
| 推荐引用方式 GB/T 7714 | Fudong Nian,Teng Li,Yan Wang,et al. Learning explicit video attributes from mid-level representation for video captioning[J]. Computer Vision and Image Understanding,2017. |
| APA | Fudong Nian,Teng Li,Yan Wang,Xinyu Wu,Bingbing Ni,&Changsheng Xu.(2017).Learning explicit video attributes from mid-level representation for video captioning.Computer Vision and Image Understanding. |
| MLA | Fudong Nian,et al."Learning explicit video attributes from mid-level representation for video captioning".Computer Vision and Image Understanding (2017). |
入库方式： OAI收割
来源：深圳先进技术研究院
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。