InsPro: Propagating Instance Query and Proposal for Online Video Instance Segmentation
文献类型:会议论文
作者 | He, Fei2,4; Zhang, Haoyang3; Gao, Naiyu3; Jia, Jian2,4; Shan, Yanhu3; Zhao, Xin2,4; Huang, Kaiqi1,2,4 |
出版日期 | 2022-11 |
会议日期 | 2022-11 |
会议地点 | New Orleans |
卷号 | 35 |
页码 | 19370-19383 |
英文摘要 | Video instance segmentation (VIS) aims at segmenting and tracking objects in videos. Prior methods typically generate frame-level or clip-level object instances first and then associate them by either additional tracking heads or complex instance matching algorithms. This explicit instance association approach increases system complexity and fails to fully exploit temporal cues in videos. In this paper, we design a simple, fast and yet effective query-based framework for online VIS. Relying on an instance query and proposal propagation mechanism with several specially developed components, this framework can perform accurate instance association implicitly. Specifically, we generate frame-level object instances based on a set of instance query-proposal pairs propagated from previous frames. This instance query-proposal pair is learned to bind with one specific object across frames through conscientiously developed strategies. When using such a pair to predict an object instance on the current frame, not only the generated instance is automatically associated with its precursors on previous frames, but the model gets a good prior for predicting the same object. In this way, we naturally achieve implicit instance association in parallel with segmentation and elegantly take advantage of temporal clues in videos. To show the effectiveness of our method InsPro, we evaluate it on two popular VIS benchmarks, i.e., YouTube-VIS 2019 and YouTube-VIS 2021. Without bells-and-whistles, our InsPro with ResNet-50 backbone achieves 43.2 AP and 37.6 AP on these two benchmarks respectively, outperforming all other online VIS methods. |
源文献作者 | Sanmi Koyejo ; Shakir Mohamed |
会议录 | Neural Information Processing Systems (NeurIPS) |
会议录出版者 | Neural Information Processing Systems |
会议录出版地 | The United States of America |
源URL | [http://ir.ia.ac.cn/handle/173211/51944] |
专题 | 智能系统与工程 |
通讯作者 | Zhao, Xin |
作者单位 | 1.CAS Center for Excellence in Brain Science and Intelligence Technology 2.CRISE, Institute of Automation, Chinese Academy of Sciences 3.Horizon Robotics 4.School of Artificial Intelligence, University of Chinese Academy of Sciences |
推荐引用方式 GB/T 7714 | He, Fei,Zhang, Haoyang,Gao, Naiyu,et al. InsPro: Propagating Instance Query and Proposal for Online Video Instance Segmentation[C]. 见:. New Orleans. 2022-11. |
入库方式: OAI收割
来源:自动化研究所
其他版本
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。