Reparameterizing and dynamically quantizing image features for image generation
文献类型：期刊论文
| 作者 | Sun, Mingzhen1,2; Wang, Weining2 ; Zhu, Xinxin2; Liu, Jing1,2
|
| 刊名 | PATTERN RECOGNITION
 |
| 出版日期 | 2024-02-01 |
| 卷号 | 146页码:11 |
| ISSN号 | 0031-3203 |
| 关键词 | Vector quantization Variational auto-encoder Unconditional image generation Text-to-image generation Autoregressive generation |
| DOI | 10.1016/j.patcog.2023.109962 |
| 通讯作者 | Liu, Jing(jliu@nlpr.ia.ac.cn) |
| 英文摘要 | For autoregressive image generation, vector-quantized VAEs (VQ-VAEs) quantize image features with discrete codebook entries and reconstruct images from quantized features. However, they treat each codebook entry separately, which causes losses of image details. In this paper, we propose to reparameterize image features with weight vectors to treat all codebook entries as an entity, and present a novel dynamically vector quantized VAE (DVQ-VAE) to quantize reparameterized image features. Specifically, each image feature corresponds to a weight vector and we sum weighted codebook entries to obtain values of image features. In this way, image features can incorporate information from different codebook entries. Additionally, a novel continuous weight regularization loss is proposed to improve the reconstruction of image details. Our method achieves competitive results with prior state-of-the-art works for image generation and extensive experiments are conducted to take a deep insight into our DVQ-VAE. |
| 资助项目 | National Key Research and De-velopment Program of China[2022ZD0118801] ; National Natural Science Foundation of China[U21B2043] ; National Natural Science Foundation of China[62102419] ; National Natural Science Foundation of China[62102416] |
| WOS研究方向 | Computer Science ; Engineering |
| 语种 | 英语 |
| 出版者 | ELSEVIER SCI LTD |
| WOS记录号 | WOS:001086812100001 |
| 资助机构 | National Key Research and De-velopment Program of China ; National Natural Science Foundation of China |
| 源URL | [http://ir.ia.ac.cn/handle/173211/54358]  |
| 专题 | 紫东太初大模型研究中心 自动化研究所_模式识别国家重点实验室_图像与视频分析团队 |
| 通讯作者 | Liu, Jing |
| 作者单位 | 1.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing, Peoples R China 2.Chinese Acad Sci, Inst Automat, Beijing, Peoples R China |
| 推荐引用方式 GB/T 7714 | Sun, Mingzhen,Wang, Weining,Zhu, Xinxin,et al. Reparameterizing and dynamically quantizing image features for image generation[J]. PATTERN RECOGNITION,2024,146:11. |
| APA | Sun, Mingzhen,Wang, Weining,Zhu, Xinxin,&Liu, Jing.(2024).Reparameterizing and dynamically quantizing image features for image generation.PATTERN RECOGNITION,146,11. |
| MLA | Sun, Mingzhen,et al."Reparameterizing and dynamically quantizing image features for image generation".PATTERN RECOGNITION 146(2024):11. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。