RISTRA: Recursive Image Super-Resolution Transformer With Relativistic Assessment
文献类型：期刊论文
| 作者 | Zhou, Xiaoqiang1; Huang, Huaibo2,3,4 ; Wang, Zilei1; He, Ran2,3,4
|
| 刊名 | IEEE TRANSACTIONS ON MULTIMEDIA
 |
| 出版日期 | 2024 |
| 卷号 | 26页码:6475-6487 |
| 关键词 | Super resolution vision transformer parameter sharing |
| ISSN号 | 1520-9210 |
| DOI | 10.1109/TMM.2024.3352400 |
| 通讯作者 | He, Ran(rhe@nlpr.ia.ac.cn) |
| 英文摘要 | Many recent image restoration methods use Transformer as the backbone network and redesign the Transformer blocks. Differently, we explore the parameter-sharing mechanism over Transformer blocks and propose a dynamic recursive process to address the image super-resolution task efficiently. We firstly present a Recursive Image Super-resolution Transformer (RIST). By sharing the weights across different blocks, a plain forward process through the whole Transformer network can be folded into recursive iterations through a Transformer block. Such a parameter-sharing based recursive process can not only reduce the model size greatly, but also enable restoring images progressively. Features in the recursive process are modeled as a sequence and propagated with a temporal attention network. Besides, by analyzing the prediction variation across different iterations in RIST, we design a dynamic recursive process that can allocate adaptive computation costs to different samples. Specifically, a quality assessment network estimates the restoration quality and terminates the recursive process dynamically. We propose a relativistic learning strategy to simplify the objective from absolute image quality assessment to relativistic quality comparison. The proposed Recursive Image Super-resolution Transformer with Relativistic Assessment (RISTRA) reduces the model size greatly with the parameter-sharing mechanism, and achieves an instance-wise dynamic restoration process as well. Extensive experiments on several image super-resolution benchmarks show the superiority of our approach over state-of-the-art counterparts. |
| 资助项目 | National Natural Science Foundation of China |
| WOS研究方向 | Computer Science ; Telecommunications |
| 语种 | 英语 |
| WOS记录号 | WOS:001200272600053 |
| 出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| 资助机构 | National Natural Science Foundation of China |
| 源URL | [http://ir.ia.ac.cn/handle/173211/58642]  |
| 专题 | 多模态人工智能系统全国重点实验室 |
| 通讯作者 | He, Ran |
| 作者单位 | 1.Univ Sci & Technol China, Dept Automat, Hefei 230027, Peoples R China 2.Chinese Acad Sci, Inst Automation, State Key Lab Multimodal Artificial Intelligence S, Beijing 100190, Peoples R China 3.Chinese Acad Sci, Inst Automat, Ctr Res Intelligent Percept & Comp, Beijing 100190, Peoples R China 4.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100190, Peoples R China |
| 推荐引用方式 GB/T 7714 | Zhou, Xiaoqiang,Huang, Huaibo,Wang, Zilei,et al. RISTRA: Recursive Image Super-Resolution Transformer With Relativistic Assessment[J]. IEEE TRANSACTIONS ON MULTIMEDIA,2024,26:6475-6487. |
| APA | Zhou, Xiaoqiang,Huang, Huaibo,Wang, Zilei,&He, Ran.(2024).RISTRA: Recursive Image Super-Resolution Transformer With Relativistic Assessment.IEEE TRANSACTIONS ON MULTIMEDIA,26,6475-6487. |
| MLA | Zhou, Xiaoqiang,et al."RISTRA: Recursive Image Super-Resolution Transformer With Relativistic Assessment".IEEE TRANSACTIONS ON MULTIMEDIA 26(2024):6475-6487. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。