中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
Image De-occlusion via Event-enhanced Multi-modal Fusion Hybrid Network

文献类型:期刊论文

作者Si-Qi Li1,2,3,4; Yue Gao1,2,3,4; Qiong-Hai Dai1,2,3,5
刊名Machine Intelligence Research
出版日期2022
卷号19期号:4页码:307-318
关键词Event camera multi-modal fusion image de-occlusion spiking neural network (SNN) image reconstruction
ISSN号2731-538X
DOI10.1007/s11633-022-1350-3
英文摘要

Seeing through dense occlusions and reconstructing scene images is an important but challenging task. Traditional frame based image de-occlusion methods may lead to fatal errors when facing extremely dense occlusions due to the lack of valid information available from the limited input occluded frames. Event cameras are bio-inspired vision sensors that record the brightness changes at each pixel asynchronously with high temporal resolution. However, synthesizing images solely from event streams is ill-posed since only the brightness changes are recorded in the event stream, and the initial brightness is unknown. In this paper, we propose an event-en hanced multi-modal fusion hybrid network for image de-occlusion, which uses event streams to provide complete scene information and frames to provide color and texture information. An event stream encoder based on the spiking neural network (SNN) is proposed to en code and denoise the event stream efficiently. A comparison loss is proposed to generate clearer results. Experimental results on a large scale event-based and frame-based image de-occlusion dataset demonstrate that our proposed method achieves state-of-the-art performance.

源URL[http://ir.ia.ac.cn/handle/173211/55947]  
专题自动化研究所_学术期刊_International Journal of Automation and Computing
作者单位1.Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing 100084, China
2.Institute for Brain and Cognitive Sciences, Tsinghua University, Beijing 100084, China
3.Beijing Laboratory of Brain and Cognitive Intelligence, Beijing Municipal Education Commission, Tsinghua University, Beijing 100084, China
4.Key Laboratory for Information System Security, School of Software, Tsinghua University, Beijing 100084, China
5.Department of Automation, Tsinghua University, Beijing 100084, China
推荐引用方式
GB/T 7714
Si-Qi Li,Yue Gao,Qiong-Hai Dai. Image De-occlusion via Event-enhanced Multi-modal Fusion Hybrid Network[J]. Machine Intelligence Research,2022,19(4):307-318.
APA Si-Qi Li,Yue Gao,&Qiong-Hai Dai.(2022).Image De-occlusion via Event-enhanced Multi-modal Fusion Hybrid Network.Machine Intelligence Research,19(4),307-318.
MLA Si-Qi Li,et al."Image De-occlusion via Event-enhanced Multi-modal Fusion Hybrid Network".Machine Intelligence Research 19.4(2022):307-318.

入库方式: OAI收割

来源:自动化研究所

浏览0
下载0
收藏0
其他版本

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。