Temporal Dynamic Concept Modeling Network for Explainable Video Event Recognition
文献类型：期刊论文
| 作者 | Zhang, Weigang5; Qi, Zhaobo5; Wang, Shuhui3,4; Su, Chi2; Su, Li6; Huang, Qingming1,7 |
| 刊名 | ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS
 |
| 出版日期 | 2023-11-01 |
| 卷号 | 19期号:6页码:22 |
| ISSN号 | 1551-6857 |
| 关键词 | Event recognition temporal concept receptive field dynamic convolution |
| DOI | 10.1145/3568312 |
| 英文摘要 | Recently, with the vigorous development of deep learning and multimedia technology, intelligent urban computing has received more and more extensive attention from academia and industry. Unfortunately, most of the related technologies are black-box paradigms that lack interpretability. Among them, video event recognition is a basic technology. Event contains multiple concepts and their rich interactions, which can assist us to construct explainable event recognition methods. However, the crucial concepts needed to recognize events have various temporal existing patterns, and the relationship between events and the temporal characteristics of concepts has not been fully exploited. This brings great challenges for concept-based event categorization. To address the above issues, we introduce the temporal concept receptive field, which is the length of the temporal window size required to capture key concepts for concept-based event recognition methods. Accordingly, we introduce the temporal dynamic convolution (TDC) to model the temporal concept receptive field dynamically according to different events. Its core idea is to combine the results of multiple convolution layers with the learned coefficients from two complementary perspectives. These convolution layers contain a variety of kernel sizes, which can provide temporal concept receptive fields of different lengths. Similarly, we also propose the cross-domain temporal dynamic convolution (CrTDC) with the help of the rich relationship between different concepts. Different coefficients can help us to capture suitable temporal concept receptive field sizes and highlight crucial concepts to obtain accurate and complete concept representations for event analysis. Based on the TDC and CrTDC, we introduce the temporal dynamic concept modeling network (TDCMN) for explainable video event recognition. We evaluate TDCMN on large-scale and challenging datasets FCVID, ActivityNet, and CCV. Experimental results show that TDCMN significantly improves the event recognition performance of concept-based methods, and the explainability of our method inspires us to construct more explainable models from the perspective of the temporal concept receptive field. |
| 资助项目 | Technology and Innovation Major Project of the Ministry of Science and Technology of China[2020AAA0108400] ; Technology and Innovation Major Project of the Ministry of Science and Technology of China[2020AAA0108402] ; National Natural Science Foundation of China[61976069] ; National Natural Science Foundation of China[U21B2038] ; National Natural Science Foundation of China[62236008] ; National Natural Science Foundation of China[62022083] ; National Natural Science Foundation of China[61836002] ; National Natural Science Foundation of China[61931008] ; Beijing Nova Program[Z201100006820023] ; Fundamental Research Funds for the Central Universities |
| WOS研究方向 | Computer Science |
| 语种 | 英语 |
| 出版者 | ASSOC COMPUTING MACHINERY |
| WOS记录号 | WOS:001035785200041 |
| 源URL | [http://119.78.100.204/handle/2XEOYT63/21363]  |
| 专题 | 中国科学院计算技术研究所期刊论文_英文 |
| 通讯作者 | Qi, Zhaobo; Huang, Qingming |
| 作者单位 | 1.Peng Cheng Lab, Beijing 101478, Peoples R China 2.SmartMore, Beijing, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China 4.Peng Cheng Lab, Beijing 100190, Peoples R China 5.Harbin Inst Technol, Weihai 264209, Peoples R China 6.Univ Chinese Acad Sci, Beijing 101478, Peoples R China 7.Univ Chinese Acad Sci, Inst Comp Technol, Chinese Acad Sci, Beijing 101478, Peoples R China |
| 推荐引用方式 GB/T 7714 | Zhang, Weigang,Qi, Zhaobo,Wang, Shuhui,et al. Temporal Dynamic Concept Modeling Network for Explainable Video Event Recognition[J]. ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS,2023,19(6):22. |
| APA | Zhang, Weigang,Qi, Zhaobo,Wang, Shuhui,Su, Chi,Su, Li,&Huang, Qingming.(2023).Temporal Dynamic Concept Modeling Network for Explainable Video Event Recognition.ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS,19(6),22. |
| MLA | Zhang, Weigang,et al."Temporal Dynamic Concept Modeling Network for Explainable Video Event Recognition".ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS 19.6(2023):22. |
入库方式： OAI收割
来源：计算技术研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。