Learning Proposal-Aware Re-Ranking for Weakly-Supervised Temporal Action Localization
文献类型：期刊论文
| 作者 | Hu, Yufan3,4; Fu, Jie2; Chen, Mengyuan2; Gao, Junyu2 ; Dong, Jianfeng1; Fan, Bin3,4; Liu, Hongmin3,4
|
| 刊名 | IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
 |
| 出版日期 | 2024 |
| 卷号 | 34期号:1页码:207-220 |
| 关键词 | Proposals Feature extraction Location awareness Videos Measurement Task analysis Optimization weakly-supervised temporal action localization Proposal-aware reranking |
| ISSN号 | 1051-8215 |
| DOI | 10.1109/TCSVT.2023.3283430 |
| 通讯作者 | Liu, Hongmin(hmliu_82@163.com) |
| 英文摘要 | Weakly-supervised temporal action localization (WTAL) aims to localize and classify action instances in untrimmed videos with only video-level labels available. Despite the remarkable success of existing methods, whose generated proposals are commonly far more than the ground-truth action instances, it still makes sense to improve the ranking accuracy of the generated proposals since users in real-world scenarios usually prioritize the action proposals with the highest confidence scores. The inaccuracy of the proposal ranking mainly comes from two aspects: For one thing, the traditional proposal generation manner entirely relies on snippet-level perception, resulting in a significant yet unnoticed gap with the target of proposal-level localization. For another, existing methods commonly employ a hand-crafted proposal generation manner, a post-process that does not participate in model optimization. To address the above issues, we propose an end-to-end trained two-stage method, termed as Learning Proposal-aware Re-ranking (LPR) for WTAL. In the first stage, we design a proposal-aware feature learning module to inject the proposal-aware contextual information into each snippet, and then the enhanced features are utilized for predicting initial proposals. Furthermore, to perform effective and efficient proposal re-ranking, in the second stage, we contrast the proposals attached with high confidence scores with our constructed multi-scale foreground/background prototypes for further optimization. Evaluated by both the vanilla and Top- $k$ mAP metrics, results of extensive experiments on two popular benchmarks demonstrate the effectiveness of our proposed method. |
| WOS关键词 | NETWORK ; VIDEO ; RETRIEVAL ; ATTENTION |
| 资助项目 | Beijing Natural Science Foundation |
| WOS研究方向 | Engineering |
| 语种 | 英语 |
| WOS记录号 | WOS:001138814400041 |
| 出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| 资助机构 | Beijing Natural Science Foundation |
| 源URL | [http://ir.ia.ac.cn/handle/173211/55512]  |
| 专题 | 多模态人工智能系统全国重点实验室 |
| 通讯作者 | Liu, Hongmin |
| 作者单位 | 1.Zhejiang Gongshang Univ, Coll Comp & Informat Engn, Hangzhou 310018, Peoples R China 2.Chinese Acad Sci, Inst Automat, State Key Lab Multimodal Artificial Intelligence S, Beijing 100190, Peoples R China 3.Univ Sci & Technol Beijing, Inst Artificial Intelligence, Beijing 100083, Peoples R China 4.Univ Sci & Technol Beijing, Key Lab Intelligent Bion Unmanned Syst, Minist Educ, Sch Intelligence Sci & Technol, Beijing 100083, Peoples R China |
| 推荐引用方式 GB/T 7714 | Hu, Yufan,Fu, Jie,Chen, Mengyuan,et al. Learning Proposal-Aware Re-Ranking for Weakly-Supervised Temporal Action Localization[J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,2024,34(1):207-220. |
| APA | Hu, Yufan.,Fu, Jie.,Chen, Mengyuan.,Gao, Junyu.,Dong, Jianfeng.,...&Liu, Hongmin.(2024).Learning Proposal-Aware Re-Ranking for Weakly-Supervised Temporal Action Localization.IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,34(1),207-220. |
| MLA | Hu, Yufan,et al."Learning Proposal-Aware Re-Ranking for Weakly-Supervised Temporal Action Localization".IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 34.1(2024):207-220. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。