Importance First: Generating Scene Graph of Human Interest
文献类型：期刊论文
| 作者 | Wang, Wenbin1,2; Wang, Ruiping1,2; Shan, Shiguang1,2; Chen, Xilin1,2 |
| 刊名 | INTERNATIONAL JOURNAL OF COMPUTER VISION
 |
| 出版日期 | 2023-06-09 |
| 页码 | 27 |
| 关键词 | Key relationship Hierarchical entity tree Hierarchical contextual propagation Relationship ranking Spatial scale Visual saliency |
| ISSN号 | 0920-5691 |
| DOI | 10.1007/s11263-023-01817-7 |
| 英文摘要 | Scene graph aims to faithfully reveal humans' perception of image content. When humans look at a scene, they usually focus on their interested parts in a special priority. This innate habit indicates a hierarchical preference about human perception. Therefore, we argue to generate the Scene Graph of Interest which should be hierarchically constructed, so that the important primary content is firstly presented while the secondary one is presented on demand. To achieve this goal, we propose the Tree-Guided Importance Ranking (TGIR) model. We represent the scene with a hierarchical structure by firstly detecting objects in the scene and organizing them into a Hierarchical Entity Tree (HET) according to their spatial scale, considering that larger objects are more likely to be noticed instantly. After that, the scene graph is generated guided by structural information of HET which is modeled by the elaborately designed Hierarchical Contextual Propagation (HCP) module. To further highlight the key relationship in the scene graph, all relationships are re-ranked through additionally estimating their importance by the Relationship Ranking Module (RRM). To train RRM, the most direct way is to collect the key relationship annotation, which is the so-called Direct Supervision scheme. As collecting annotation may be cumbersome, we further utilize two intuitive and effective cues, visual saliency and spatial scale, and treat them as Approximate Supervision, according to the findings that these cues are positively correlated with relationship importance. With these readily available cues, the RRM is still able to estimate the importance even without key relationship annotation. Experiments indicate that our method not only achieves state-of-the-art performances on scene graph generation, but also is expert in mining image-specific relationships which play a great role in serving subsequent tasks such as image captioning and cross-modal retrieval. |
| 资助项目 | Natural Science Foundation of China[U21B2025] ; Natural Science Foundation of China[U19B2036] ; Natural Science Foundation of China[61922080] ; National Key R &D Program of China[2021ZD0111901] |
| WOS研究方向 | Computer Science |
| 语种 | 英语 |
| WOS记录号 | WOS:001005476600002 |
| 出版者 | SPRINGER |
| 源URL | [http://119.78.100.204/handle/2XEOYT63/21217]  |
| 专题 | 中国科学院计算技术研究所期刊论文_英文 |
| 通讯作者 | Wang, Ruiping |
| 作者单位 | 1.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing, Peoples R China |
| 推荐引用方式 GB/T 7714 | Wang, Wenbin,Wang, Ruiping,Shan, Shiguang,et al. Importance First: Generating Scene Graph of Human Interest[J]. INTERNATIONAL JOURNAL OF COMPUTER VISION,2023:27. |
| APA | Wang, Wenbin,Wang, Ruiping,Shan, Shiguang,&Chen, Xilin.(2023).Importance First: Generating Scene Graph of Human Interest.INTERNATIONAL JOURNAL OF COMPUTER VISION,27. |
| MLA | Wang, Wenbin,et al."Importance First: Generating Scene Graph of Human Interest".INTERNATIONAL JOURNAL OF COMPUTER VISION (2023):27. |
入库方式： OAI收割
来源：计算技术研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。