Debiased Video-Text Retrieval via Soft Positive Sample Calibration
文献类型：期刊论文
| 作者 | Zhang, Huaiwen1,4,6 ; Yang, Yang1,4,6; Qi, Fan5; Qian, Shengsheng2,3; Xu, Changsheng2,3
|
| 刊名 | IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
 |
| 出版日期 | 2023-09-01 |
| 卷号 | 33期号:9页码:5257-5270 |
| ISSN号 | 1051-8215 |
| 关键词 | debias soft positive samples |
| DOI | 10.1109/TCSVT.2023.3248873 |
| 通讯作者 | Qian, Shengsheng(shengsheng.qian@nlpr.ia.ac.cn) |
| 英文摘要 | With the emergence of enormous videos on various video apps, semantic video-text retrieval has become a critical task for improving the user experience. The primary paradigm for video-text retrieval learns the semantic video-text representations in a common space by pulling the positive samples close to the query and pushing the negative samples away. However, in practice, the video-text datasets contain only the annotations of positive samples. The negative samples are randomly drawn from the entire dataset. There may exist soft positive samples, which are sampled as negatives but share the same semantics as positive samples. Indiscriminately enforcing the model to push all the negative samples away from the query leads to inaccurate supervision and then misleads the video-text feature representation learning. In this paper, we introduce debiased video-text retrieval objectives that calibrate the punishment of soft positive samples. In particular, we propose a novel uncertainty measure framework to estimate the credibility of negative samples for each instance. Then, the reliability of negative samples is used to find the soft positive samples and rescale their contribution within video-text retrieval losses, including triplet loss and contrastive loss. Experimental results on five widely used datasets demonstrate that our debiased video-text retrieval objectives achieve significant performance improvements and establish a new state-of-the-art. |
| 资助项目 | National Natural Science Foundation of China[62206137] ; National Natural Science Foundation of China[62206200] ; National Natural Science Foundation of China[62276257] ; National Natural Science Foundation of China[62036012] ; National Natural Science Foundation of China[62066033] ; National Natural Science Foundation of Inner Mongolia[2022MS06025] ; Program for Young Talents of Science and Technology in Universities of Inner Mongolia Autonomous Region[NJYT23105] ; Applied Technology Research and Development Program of Inner Mongolia Autonomous Region[2020GG0046] ; Applied Technology Research and Development Program of Inner Mongolia Autonomous Region[2021GG0158] ; Applied Technology Research and Development Program of Inner Mongolia Autonomous Region[2020PT0002] ; National Natural Science Foundation of China[62206137] ; National Natural Science Foundation of China[62206200] ; National Natural Science Foundation of China[62276257] ; National Natural Science Foundation of China[62036012] ; National Natural Science Foundation of China[62066033] ; National Natural Science Foundation of Inner Mongolia[2022MS06025] ; Program for Young Talents of Science and Technology in Universities of Inner Mongolia Autonomous Region[NJYT23105] ; Applied Technology Research and Development Program of Inner Mongolia Autonomous Region[2020GG0046] ; Applied Technology Research and Development Program of Inner Mongolia Autonomous Region[2021GG0158] ; Applied Technology Research and Development Program of Inner Mongolia Autonomous Region[2020PT0002] |
| WOS研究方向 | Engineering |
| 语种 | 英语 |
| 出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| WOS记录号 | WOS:001063316800060 |
| 资助机构 | National Natural Science Foundation of China ; National Natural Science Foundation of Inner Mongolia ; Program for Young Talents of Science and Technology in Universities of Inner Mongolia Autonomous Region ; Applied Technology Research and Development Program of Inner Mongolia Autonomous Region ; National Natural Science Foundation of China ; National Natural Science Foundation of Inner Mongolia ; Program for Young Talents of Science and Technology in Universities of Inner Mongolia Autonomous Region ; Applied Technology Research and Development Program of Inner Mongolia Autonomous Region |
| 源URL | [http://ir.ia.ac.cn/handle/173211/53152]  |
| 专题 | 多模态人工智能系统全国重点实验室 |
| 通讯作者 | Qian, Shengsheng |
| 作者单位 | 1.Inner Mongolia Key Lab Mongolian Informat Proc Tec, Hohhot 010021, Peoples R China 2.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China 3.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China 4.Natl & Local Joint Engn Res Ctr Intelligent Inform, Hohhot 010021, Peoples R China 5.Tianjin Univ Technol, Sch Comp Sci & Engn, Tianjin 300384, Peoples R China 6.Inner Mongolia Univ, Coll Comp Sci, Hohhot 010021, Peoples R China |
| 推荐引用方式 GB/T 7714 | Zhang, Huaiwen,Yang, Yang,Qi, Fan,et al. Debiased Video-Text Retrieval via Soft Positive Sample Calibration[J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,2023,33(9):5257-5270. |
| APA | Zhang, Huaiwen,Yang, Yang,Qi, Fan,Qian, Shengsheng,&Xu, Changsheng.(2023).Debiased Video-Text Retrieval via Soft Positive Sample Calibration.IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,33(9),5257-5270. |
| MLA | Zhang, Huaiwen,et al."Debiased Video-Text Retrieval via Soft Positive Sample Calibration".IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 33.9(2023):5257-5270. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。