中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 自动化研究所
  3. 中国科学院自动化研究所
  4. 多模态人工智能系统全国重点实验室
  5. 多媒体计算与图形学团队
Deep learning-based importance map for content-aware media retargeting

文献类型：期刊论文

作者Le, Thi-Ngoc-Hanh1; Lee, Tong-Yee1; Lin, Shih-Syun2; Dong, Weiming3
刊名MULTIMEDIA TOOLS AND APPLICATIONS
出版日期2024-02-15
页码22
关键词Retargeting A2R-Map Seam carving Warping
ISSN号1380-7501
DOI10.1007/s11042-024-18389-4
通讯作者Lee, Tong-Yee(tonylee@mail.ncku.edu.tw)
英文摘要We introduce a deep learning-driven framework for creating an adaptably applicable importance map (A2R-Map) that can be integrated with existing image and video retargeting operators. A conventional retargeting algorithm uses a heuristic approach to seek an off-the-self algorithm used into their retargeting system. The extracted importance map of the image does not match the characteristics of the input image; therefore, it affects the retargeting results and limits the performance of the retargeting method. Our designed framework attempts to minimize the artifacts/distortions caused by inappropriate energy, e.g., the shrunk phenomenon in warping-based results and carving-through-object distortion in the seam carving-based approach. Our proposed framework focuses on capturing sensitive distortion regions and activating their energy to solve this challenge. We verify the effectiveness of our proposed scheme by plugging it in three typical retargeting methods: seam carving-based, warping-based for image, and video retargeting. Extensive experiments and evaluations are conducted on two widely used databases. On the one hand, A2R-Map significantly reduces the time of importance map generation in retargeting systems to similar to 9\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sim 9$$\end{document} times compared to the baseline saliency map. On the other hand, our A2R-Map achieves improvement over the baseline methods with an average of 11% and 9% in terms of image and video quality, respectively. The experimental results and evaluations demonstrate that our strategy for A2R-Map substantially outperforms the previous works and significantly boosts the visual quality of video/image retargeting.
资助项目National Science and Technology Council[111-2221-E-006-112-MY3] ; National Science and Technology Council[110-2221-E-006-135-MY3] ; National Science and Technology Council[112-2221-E-019-063-MY3] ; National Science and Technology Council[110-2221-E-019-052-MY3] ; Republic of China (ROC), Taiwan ; National Natural Science Foundation of China[U20B2070] ; National Natural Science Foundation of China[61832016]
WOS研究方向Computer Science ; Engineering
语种英语
WOS记录号WOS:001162156800023
出版者SPRINGER
资助机构National Science and Technology Council ; Republic of China (ROC), Taiwan ; National Natural Science Foundation of China
源URL[http://ir.ia.ac.cn/handle/173211/57844]  
专题自动化研究所_模式识别国家重点实验室_多媒体计算与图形学团队
通讯作者Lee, Tong-Yee
作者单位1.Natl Cheng Kung Univ, Tainan, Taiwan
2.Natl Taiwan Ocean Univ, Hsinchu, Taiwan
3.Chinese Acad Sci, Inst Automat, Beijing, Peoples R China
推荐引用方式
GB/T 7714		 Le, Thi-Ngoc-Hanh,Lee, Tong-Yee,Lin, Shih-Syun,et al. Deep learning-based importance map for content-aware media retargeting[J]. MULTIMEDIA TOOLS AND APPLICATIONS,2024:22.
APA Le, Thi-Ngoc-Hanh,Lee, Tong-Yee,Lin, Shih-Syun,&Dong, Weiming.(2024).Deep learning-based importance map for content-aware media retargeting.MULTIMEDIA TOOLS AND APPLICATIONS,22.
MLA Le, Thi-Ngoc-Hanh,et al."Deep learning-based importance map for content-aware media retargeting".MULTIMEDIA TOOLS AND APPLICATIONS (2024):22.

入库方式： OAI收割

来源：自动化研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。