Toward Egocentric Compositional Action Anticipation with Adaptive Semantic Debiasing
文献类型：期刊论文
| 作者 | Zhang, Tianyu2,3; Min, Weiqing2,3; Liu, Tao2,3; Jiang, Shuqiang2,3; Rui, Yong1 |
| 刊名 | ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS
 |
| 出版日期 | 2024-05-01 |
| 卷号 | 20期号:5页码:21 |
| 关键词 | Egocentric video understanding compositional action anticipation semantic bias adaptive counterfactual analysis |
| ISSN号 | 1551-6857 |
| DOI | 10.1145/3633333 |
| 英文摘要 | Predicting the unknown from the first-person perspective is expected as a necessary step toward machine intelligence, which is essential for practical applications including autonomous driving and robotics. As a human-level task, egocentric action anticipation aims at predicting an unknown action seconds before it is performed from the first-person viewpoint. Egocentric actions are usually provided as verb-noun pairs; however, predicting the unknown action may be trapped in insufficient training data for all possible combinations. Therefore, it is crucial for intelligent systems to use limited known verb-noun pairs to predict new combinations of actions that have never appeared, which is known as compositional generalization. In this article, we are the first to explore the egocentric compositional action anticipation problem, which is more in line with real-world settings but neglected by existing studies. Whereas prediction results are prone to suffer from semantic bias considering the distinct difference between training and test distributions, we further introduce a general and flexible adaptive semantic debiasing framework that is compatible with different deep neural networks. To capture and mitigate semantic bias, we can imagine one counterfactual situation where no visual representations have been observed and only semantic patterns of observation are used to predict the next action. Instead of the traditional counterfactual analysis scheme that reduces semantic bias in a mindless way, we devise a novel counterfactual analysis scheme to adaptively amplify or penalize the effect of semantic experience by considering the discrepancy both among categories and among examples. We also demonstrate that the traditional counterfactual analysis scheme is a special case of the devised adaptive counterfactual analysis scheme. We conduct experiments on three large-scale egocentric video datasets. Experimental results verify the superiority and effectiveness of our proposed solution. |
| 资助项目 | National Key Research and Development Project of New Generation Artificial Intelligence of China[2018AAA0102500] |
| WOS研究方向 | Computer Science |
| 语种 | 英语 |
| WOS记录号 | WOS:001192177900002 |
| 出版者 | ASSOC COMPUTING MACHINERY |
| 源URL | [http://119.78.100.204/handle/2XEOYT63/38746]  |
| 专题 | 中国科学院计算技术研究所期刊论文_英文 |
| 通讯作者 | Zhang, Tianyu |
| 作者单位 | 1.Lenovo Grp, 6 Shangdi West Rd, Beijing, Peoples R China 2.Univ Chinese Acad Sci, 80 Zhongguancun East Rd, Beijing, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, 6 Kexueyuan South Rd, Beijing, Peoples R China |
| 推荐引用方式 GB/T 7714 | Zhang, Tianyu,Min, Weiqing,Liu, Tao,et al. Toward Egocentric Compositional Action Anticipation with Adaptive Semantic Debiasing[J]. ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS,2024,20(5):21. |
| APA | Zhang, Tianyu,Min, Weiqing,Liu, Tao,Jiang, Shuqiang,&Rui, Yong.(2024).Toward Egocentric Compositional Action Anticipation with Adaptive Semantic Debiasing.ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS,20(5),21. |
| MLA | Zhang, Tianyu,et al."Toward Egocentric Compositional Action Anticipation with Adaptive Semantic Debiasing".ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS 20.5(2024):21. |
入库方式： OAI收割
来源：计算技术研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。