Self-Regulated Learning for Egocentric Video Activity Anticipation
文献类型：期刊论文
| 作者 | Qi, Zhaobo2,3; Wang, Shuhui2; Su, Chi4; Su, Li3; Huang, Qingming2,3,5; Tian, Qi1 |
| 刊名 | IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
 |
| 出版日期 | 2023-06-01 |
| 卷号 | 45期号:6页码:6715-6730 |
| 关键词 | Predictive models Dairy products Semantics Feature extraction Visualization Activity recognition Task analysis Egocentric video activity anticipaiton third-person video activity anticipaiton contrastive learning multi-task learning self-regulated learning |
| ISSN号 | 0162-8828 |
| DOI | 10.1109/TPAMI.2021.3059923 |
| 英文摘要 | Future activity anticipation is a challenging problem in egocentric vision. As a standard future activity anticipation paradigm, recursive sequence prediction suffers from the accumulation of errors. To address this problem, we propose a simple and effective Self-Regulated Learning framework, which aims to regulate the intermediate representation consecutively to produce representation that (a) emphasizes the novel information in the frame of the current time-stamp in contrast to previously observed content, and (b) reflects its correlation with previously observed frames. The former is achieved by minimizing a contrastive loss, and the latter can be achieved by a dynamic reweighing mechanism to attend to informative frames in the observed content with a similarity comparison between feature of the current frame and observed frames. The learned final video representation can be further enhanced by multi-task learning which performs joint feature learning on the target activity labels and the automatically detected action and object class tokens. SRL sharply outperforms existing state-of-the-art in most cases on two egocentric video datasets and two third-person video datasets. Its effectiveness is also verified by the experimental fact that the action and object concepts that support the activity semantics can be accurately identified. |
| 资助项目 | National Key R&D Program of China[2018AAA0102003] ; National Natural Science Foundation of China[62022083] ; National Natural Science Foundation of China[61672497] ; National Natural Science Foundation of China[61620106009] ; National Natural Science Foundation of China[61836002] ; National Natural Science Foundation of China[61931008] ; Key Research Program of Frontier Sciences, CAS[QYZDJ-SSWSYS013] ; Beijing Nova Program[Z201100006820023] ; Fundamental Research Funds for the Central Universities |
| WOS研究方向 | Computer Science ; Engineering |
| 语种 | 英语 |
| WOS记录号 | WOS:000982475600010 |
| 出版者 | IEEE COMPUTER SOC |
| 源URL | [http://119.78.100.204/handle/2XEOYT63/21229]  |
| 专题 | 中国科学院计算技术研究所期刊论文_英文 |
| 通讯作者 | Wang, Shuhui; Huang, Qingming |
| 作者单位 | 1.Huawei Technol, Cloud BU, Shenzhen, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 3.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing 101408, Peoples R China 4.Kingsoft Cloud, Beijing 100085, Peoples R China 5.Peng Cheng Lab, Shenzhen 518066, Peoples R China |
| 推荐引用方式 GB/T 7714 | Qi, Zhaobo,Wang, Shuhui,Su, Chi,et al. Self-Regulated Learning for Egocentric Video Activity Anticipation[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2023,45(6):6715-6730. |
| APA | Qi, Zhaobo,Wang, Shuhui,Su, Chi,Su, Li,Huang, Qingming,&Tian, Qi.(2023).Self-Regulated Learning for Egocentric Video Activity Anticipation.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,45(6),6715-6730. |
| MLA | Qi, Zhaobo,et al."Self-Regulated Learning for Egocentric Video Activity Anticipation".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 45.6(2023):6715-6730. |
入库方式： OAI收割
来源：计算技术研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。