Exploring Rich Semantics for Open-Set Action Recognition
文献类型：期刊论文
| 作者 | Hu, Yufan1; Gao, Junyu2 ; Dong, Jianfeng3; Fan, Bin1; Liu, Hongmin1
|
| 刊名 | IEEE TRANSACTIONS ON MULTIMEDIA
 |
| 出版日期 | 2024 |
| 卷号 | 26页码:5410-5421 |
| 关键词 | Semantics Prototypes Knowledge graphs Visualization Task analysis Uncertainty Training Open-set action recognition video action recognition semantic relation modeling |
| ISSN号 | 1520-9210 |
| DOI | 10.1109/TMM.2023.3333206 |
| 通讯作者 | Liu, Hongmin(hmliu_82@163.com) |
| 英文摘要 | Open-set action recognition (OSAR) aims to learn a recognition framework capable of both classifying known classes and identifying unknown actions in open-set scenarios. Existing OSAR methods typically reside in a data-driven paradigm, which ignore the rich semantics in both known and unknown categories. In fact, we humans have the capability of leveraging the captured semantic information, i.e., knowledge and experience, to incisively distinguish samples from known and unknown classes. Motivated by this observation, in this paper, we propose a Unified Semantic Exploration (USE) framework for recognizing actions in open-set scenarios. Specifically, we explore the explicit knowledge semantics by simulating the unknown classes with knowledge-guided virtual classes based on an external knowledge graph, which enables the model to simulate open-set perception during model training. Besides, we propose to learn the implicit data semantics by transferring the knowledge structure of action categories to the visual prototype space for semantic structure preservation. Extensive experiments on several action recognition benchmarks validate the effectiveness of our proposed method. |
| 资助项目 | National Natural Science Foundation of China |
| WOS研究方向 | Computer Science ; Telecommunications |
| 语种 | 英语 |
| WOS记录号 | WOS:001189435600012 |
| 出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| 资助机构 | National Natural Science Foundation of China |
| 源URL | [http://ir.ia.ac.cn/handle/173211/58098]  |
| 专题 | 自动化研究所_模式识别国家重点实验室_多媒体计算与图形学团队 |
| 通讯作者 | Liu, Hongmin |
| 作者单位 | 1.Univ Sci & Technol Beijing, Sch Intelligence & Technol, Beijing 100083, Peoples R China 2.Chinese Acad Sci, Inst Automat, State Key Lab Multimodal Artificial Intelligence S, Beijing 100190, Peoples R China 3.Zhejiang Gongshang Univ, Coll Comp & Informat Engn, Hangzhou 310018, Peoples R China |
| 推荐引用方式 GB/T 7714 | Hu, Yufan,Gao, Junyu,Dong, Jianfeng,et al. Exploring Rich Semantics for Open-Set Action Recognition[J]. IEEE TRANSACTIONS ON MULTIMEDIA,2024,26:5410-5421. |
| APA | Hu, Yufan,Gao, Junyu,Dong, Jianfeng,Fan, Bin,&Liu, Hongmin.(2024).Exploring Rich Semantics for Open-Set Action Recognition.IEEE TRANSACTIONS ON MULTIMEDIA,26,5410-5421. |
| MLA | Hu, Yufan,et al."Exploring Rich Semantics for Open-Set Action Recognition".IEEE TRANSACTIONS ON MULTIMEDIA 26(2024):5410-5421. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。