中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 自动化研究所
  3. 中国科学院自动化研究所
  4. 多模态人工智能系统全国重点实验室
  5. 多媒体计算与图形学团队
CI-GNN: Building a Category-Instance Graph for Zero-Shot Video Classification

文献类型：期刊论文

作者Gao, Junyu1,2,3; Xu, Changsheng1,2,3
刊名IEEE TRANSACTIONS ON MULTIMEDIA
出版日期2020-12-01
卷号22期号:12页码:3088-3100
关键词Semantics Task analysis Visualization Training Message passing Pattern recognition Neural networks Zero-shot video classification graph neural network zero-shot learning
ISSN号1520-9210
DOI10.1109/TMM.2020.2969787
通讯作者Xu, Changsheng(csxu@nlpr.ia.ac.cn)
英文摘要With the ever-growing video categories, Zero-Shot Learning (ZSL) in video classification has drawn considerable attention in recent years. To transfer the learned knowledge from seen categories to unseen categories, most existing methods resort to an implicit model that learns a projection between visual features and semantic category-representations. However, such methods ignore the explicit relationships among video instances and categories, which impede the direct information propagation in a Category-Instance graph (CI-graph) consisting of both instances and categories. In fact, exploring the structure of the CI-graph can capture the invariances of the ZSL task with good generality for unseen instances. Inspired by these observations, we propose an end-to-end framework to directly and collectively model the relationships between category-instance, category-category, and instance-instance in the CI-graph. Specifically, to construct node features of this graph, we adopt object semantics as a bridge to generate unified representations for both videos and categories. Motivated by the favorable performance of Graph Neural Networks (GNNs), we design a Category-Instance GNN (CI-GNN) to adaptively model the structure of the CI-graph and propagate information among categories and videos. With the task-driven message passing process, the learned model is able to transfer label information from categories towards unseen videos. Extensive experiments on four video datasets demonstrate the favorable performance of the proposed framework.
WOS关键词LEARNING FRAMEWORK ; NEURAL-NETWORK
资助项目National Natural Science Foundation of China[61720106006] ; National Natural Science Foundation of China[61721004] ; National Natural Science Foundation of China[61832002] ; National Natural Science Foundation of China[61532009] ; National Natural Science Foundation of China[U1705262] ; National Natural Science Foundation of China[U1836220] ; National Natural Science Foundation of China[61702511] ; Key Research Program of Frontier Sciences, CAS[QYZDJSSWJSC039] ; Research Program of National Laboratory of Pattern Recognition[Z-2018007]
WOS研究方向Computer Science ; Telecommunications
语种英语
WOS记录号WOS:000591817700006
出版者IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
资助机构National Natural Science Foundation of China ; Key Research Program of Frontier Sciences, CAS ; Research Program of National Laboratory of Pattern Recognition
源URL[http://ir.ia.ac.cn/handle/173211/42515]  
专题自动化研究所_模式识别国家重点实验室_多媒体计算与图形学团队
通讯作者Xu, Changsheng
作者单位1.PengCheng Lab, Shenzhen 518066, Peoples R China
2.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China
3.Chinese Acad Sci, Natl Lab Pattern Recognit, Inst Automat, Beijing 100190, Peoples R China
推荐引用方式
GB/T 7714		 Gao, Junyu,Xu, Changsheng. CI-GNN: Building a Category-Instance Graph for Zero-Shot Video Classification[J]. IEEE TRANSACTIONS ON MULTIMEDIA,2020,22(12):3088-3100.
APA Gao, Junyu,&Xu, Changsheng.(2020).CI-GNN: Building a Category-Instance Graph for Zero-Shot Video Classification.IEEE TRANSACTIONS ON MULTIMEDIA,22(12),3088-3100.
MLA Gao, Junyu,et al."CI-GNN: Building a Category-Instance Graph for Zero-Shot Video Classification".IEEE TRANSACTIONS ON MULTIMEDIA 22.12(2020):3088-3100.

入库方式： OAI收割

来源：自动化研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。