中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 自动化研究所
  3. 中国科学院自动化研究所
  4. 多模态人工智能系统全国重点实验室
  5. 视频内容安全团队
Chinese Title Generation for Short Videos: Dataset, Metric and Algorithm

文献类型：期刊论文

作者Zhang, Ziqi1,2; Ma, Zongyang1,2; Yuan, Chunfeng1,2; Chen, Yuxin1,2; Wang, Peijin3; Qi, Zhongang4; Hao, Chenglei5; Li, Bing1,2; Shan, Ying4; Hu, Weiming1,2,6
刊名IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
出版日期2024-07-01
卷号46期号:7页码:5192-5208
关键词Videos Task analysis Measurement Semantics Benchmark testing Electronic commerce Annotations Video and language short video multi-modal benchmark video titling title evaluation text-video retrieval
ISSN号0162-8828
DOI10.1109/TPAMI.2024.3365739
通讯作者Li, Bing(bli@nlpr.ia.ac.cn)
英文摘要Previous work for video captioning aims to objectively describe the video content but the captions lack human interest and attractiveness, limiting its practical application scenarios. The intention of video title generation (video titling) is to produce attractive titles, but there is a lack of benchmarks. This work offers CREATE, the first large-scale Chinese shoRt vidEo retrievAl and Title gEneration dataset, to assist research and applications in video titling, video captioning, and video retrieval in Chinese. CREATE comprises a high-quality labeled 210 K dataset and two web-scale 3 M and 10 M pre-training datasets, covering 51 categories, 50K+ tags, 537K+ manually annotated titles and captions, and 10M+ short videos with original video information. This work presents ACTEr, a unique Attractiveness-Consensus-based Title Evaluation, to objectively evaluate the quality of video title generation. This metric measures the semantic correlation between the candidate (model-generated title) and references (manual-labeled titles) and introduces attractive consensus weights to assess the attractiveness and relevance of the video title. Accordingly, this work proposes a novel multi-modal ALignment WIth Generation model, ALWIG, as one strong baseline to aid future model development. With the help of a tag-driven video-text alignment module and a GPT-based generation module, this model achieves video titling, captioning, and retrieval simultaneously. We believe that the release of the CREATE dataset, ACTEr metric, and ALWIG model will encourage in-depth research on the analysis and creation of Chinese short videos.
资助项目National Key Research and Development Program of China
WOS研究方向Computer Science ; Engineering
语种英语
WOS记录号WOS:001240147800016
出版者IEEE COMPUTER SOC
资助机构National Key Research and Development Program of China
源URL[http://ir.ia.ac.cn/handle/173211/59076]  
专题自动化研究所_模式识别国家重点实验室_视频内容安全团队
通讯作者Li, Bing
作者单位1.Chinese Acad Sci, Inst Automat, State Key Lab Multimodal Artificial Intelligence, Beijing 100190, Peoples R China
2.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China
3.Chinese Acad Sci, Aerosp Informat Res Inst, Beijing 100045, Peoples R China
4.ARC Lab Tencent PCG, Shenzhen 518000, Peoples R China
5.Huake Xingsheng Elect Power Engn Technol, Beijing 100045, Peoples R China
6.Shanghai Tech Univ, Sch Informat Sci & Technol, Shanghai 201210, Peoples R China
7.Birkbeck Coll, Comp Sci & Informat Syst, London WC1E 7HX, England
推荐引用方式
GB/T 7714		 Zhang, Ziqi,Ma, Zongyang,Yuan, Chunfeng,et al. Chinese Title Generation for Short Videos: Dataset, Metric and Algorithm[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2024,46(7):5192-5208.
APA Zhang, Ziqi.,Ma, Zongyang.,Yuan, Chunfeng.,Chen, Yuxin.,Wang, Peijin.,...&Maybank, Stephen.(2024).Chinese Title Generation for Short Videos: Dataset, Metric and Algorithm.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,46(7),5192-5208.
MLA Zhang, Ziqi,et al."Chinese Title Generation for Short Videos: Dataset, Metric and Algorithm".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 46.7(2024):5192-5208.

入库方式： OAI收割

来源：自动化研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。