Unsupervised Video Summarization via Relation-Aware Assignment Learning
文献类型：期刊论文
| 作者 | Gao, Junyu1,2,3 ; Yang, Xiaoshan1,2,3; Zhang, Yingying1,2,3; Xu, Changsheng1,2,3
|
| 刊名 | IEEE TRANSACTIONS ON MULTIMEDIA
 |
| 出版日期 | 2021 |
| 卷号 | 23页码:3203-3214 |
| ISSN号 | 1520-9210 |
| 关键词 | Feature extraction Training Optimization Semantics Recurrent neural networks Task analysis Graph neural network unsupervised learning video summarization |
| DOI | 10.1109/TMM.2020.3021980 |
| 通讯作者 | Xu, Changsheng(csxu@nlpr.ia.ac.cn) |
| 英文摘要 | We address the problem of unsupervised video summarization that automatically selects key video clips. Most state-of-the-art approaches suffer from two issues: (1) they model video clips without explicitly exploiting their relations, and (2) they learn soft importance scores over all the video clips to generate the summary representation. However, a meaningful video summary should be inferred by taking the relation-aware context of the original video into consideration, and directly selecting a subset of clips with a hard assignment. In this paper, we propose to exploit clip-clip relations to learn relation-aware hard assignments for selecting key clips in an unsupervised manner. First, we consider the clips as graph nodes to construct an assignment-learning graph. Then, we utilize the magnitude of the node features to generate hard assignments as the summary selection. Finally, we optimize the whole framework via a proposed multi-task loss including a reconstruction constraint, and a contrastive constraint. Extensive experimental results on three popular benchmarks demonstrate the favourable performance of our approach. |
| 资助项目 | National Key Research and Development Program of China[2018AAA0102200] ; National Natural Science Foundation of China[61720106006] ; National Natural Science Foundation of China[61721004] ; National Natural Science Foundation of China[61832002] ; National Natural Science Foundation of China[61702511] ; National Natural Science Foundation of China[61751211] ; National Natural Science Foundation of China[61532009] ; National Natural Science Foundation of China[U1836220] ; National Natural Science Foundation of China[U1705262] ; National Natural Science Foundation of China[61872424] ; National Natural Science Foundation of China[61936005] ; Key Research Program of Frontier Sciences of CAS[QYZDJSSWJSC039] ; Research Program of National Laboratory of Pattern Recognition[Z-2018007] |
| WOS研究方向 | Computer Science ; Telecommunications |
| 语种 | 英语 |
| 出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| WOS记录号 | WOS:000698902000020 |
| 资助机构 | National Key Research and Development Program of China ; National Natural Science Foundation of China ; Key Research Program of Frontier Sciences of CAS ; Research Program of National Laboratory of Pattern Recognition |
| 源URL | [http://ir.ia.ac.cn/handle/173211/45732]  |
| 专题 | 自动化研究所_模式识别国家重点实验室_多媒体计算与图形学团队 |
| 通讯作者 | Xu, Changsheng |
| 作者单位 | 1.Chinese Acad Sci, Natl Lab Pattern Recognit, Inst Automat, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China 3.PengCheng Lab, Shenzhen 518066, Peoples R China |
| 推荐引用方式 GB/T 7714 | Gao, Junyu,Yang, Xiaoshan,Zhang, Yingying,et al. Unsupervised Video Summarization via Relation-Aware Assignment Learning[J]. IEEE TRANSACTIONS ON MULTIMEDIA,2021,23:3203-3214. |
| APA | Gao, Junyu,Yang, Xiaoshan,Zhang, Yingying,&Xu, Changsheng.(2021).Unsupervised Video Summarization via Relation-Aware Assignment Learning.IEEE TRANSACTIONS ON MULTIMEDIA,23,3203-3214. |
| MLA | Gao, Junyu,et al."Unsupervised Video Summarization via Relation-Aware Assignment Learning".IEEE TRANSACTIONS ON MULTIMEDIA 23(2021):3203-3214. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。