Explicit 3D reconstruction from images with dynamic graph learning and rendering-guided diffusion
文献类型：期刊论文
| 作者 | Wu, Di1,2; Zhou, Linli2 ; Li, Jincheng1,2; Xiong, Jianqiao1,2; Song, Liangtu2
|
| 刊名 | NEUROCOMPUTING
 |
| 出版日期 | 2024-10-07 |
| 卷号 | 601 |
| 关键词 | Image-based reconstruction Graph learning Diffusion model |
| ISSN号 | 0925-2312 |
| DOI | 10.1016/j.neucom.2024.128206 |
| 通讯作者 | Wu, Di(wdcs@mail.ustc.edu.cn) |
| 英文摘要 | High-quality 3D reconstruction is becoming increasingly important in a variety of fields. Recently, implicit representation methods have made significant progress in image-based 3D reconstruction. However, these methods tend to yield entangled neural representations which lack support for standard 3D pipelines, and their reconstruction results usually experience a sharp drop in quality when input views are reduced. To obtain high-quality 3D content, we propose an explicit 3D reconstruction method that directly extracts textured meshes from images and remains robust using reduced input views. Our central components include a dynamic graph convolutional network (GCN) and a rendering-guided diffusion model. The dynamic GCN aims to improve mesh reconstruction quality by effectively aggregating features from vertex neighborhoods. The aggregation is accelerated through sampling geometric-related neighbors with different SDF signs, which gradually converges in quantity during training. The rendering-guided diffusion model learns prior distributions for unseen regions to improve reconstruction performance using sparse-view inputs. It uses the rendered image under an interpolated camera pose as conditioned input and its diffusion strength can be controlled with the rendering loss of explicit reconstruction. In addition, the rendering-guided diffusion model can be jointly trained to generate plausible novel views with 3D consistency. Experiments demonstrate that our method can produce high-quality explicit reconstruction results and maintain realistic reconstruction using sparse-view inputs. |
| 资助项目 | Hefei Institutes of Physical Science, China[ZKBB202103] |
| WOS研究方向 | Computer Science |
| 语种 | 英语 |
| WOS记录号 | WOS:001279994900001 |
| 出版者 | ELSEVIER |
| 资助机构 | Hefei Institutes of Physical Science, China |
| 源URL | [http://ir.hfcas.ac.cn:8080/handle/334002/137378]  |
| 专题 | 中国科学院合肥物质科学研究院 |
| 通讯作者 | Wu, Di |
| 作者单位 | 1.Univ Sci & Technol China, Hefei 230026, Peoples R China 2.Chinese Acad Sci, Hefei Inst Phys Sci, Hefei 230031, Peoples R China |
| 推荐引用方式 GB/T 7714 | Wu, Di,Zhou, Linli,Li, Jincheng,et al. Explicit 3D reconstruction from images with dynamic graph learning and rendering-guided diffusion[J]. NEUROCOMPUTING,2024,601. |
| APA | Wu, Di,Zhou, Linli,Li, Jincheng,Xiong, Jianqiao,&Song, Liangtu.(2024).Explicit 3D reconstruction from images with dynamic graph learning and rendering-guided diffusion.NEUROCOMPUTING,601. |
| MLA | Wu, Di,et al."Explicit 3D reconstruction from images with dynamic graph learning and rendering-guided diffusion".NEUROCOMPUTING 601(2024). |
入库方式： OAI收割
来源：合肥物质科学研究院
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。