Adversarial-Metric Learning for Audio-Visual Cross-Modal Matching
文献类型：期刊论文
| 作者 | Zheng, Aihua2,3; Hu, Menglan2,3; Jiang, Bo2,3,4; Huang, Yan5 ; Yan, Yan1; Luo, Bin2,3
|
| 刊名 | IEEE TRANSACTIONS ON MULTIMEDIA
 |
| 出版日期 | 2022 |
| 卷号 | 24页码:338-351 |
| 关键词 | Visualization Task analysis Measurement Speech recognition Videos Location awareness Image recognition Adversarial learning audio-visual matching cross-modal learning metric learning |
| ISSN号 | 1520-9210 |
| DOI | 10.1109/TMM.2021.3050089 |
| 通讯作者 | Jiang, Bo(jiangbo@ahu.edu.cn) |
| 英文摘要 | Audio-visual matching aims to learn the intrinsic correspondence between image and audio clip. Existing works mainly concentrate on learning discriminative features, while ignore the cross-modal heterogeneous issue between audio and visual modalities. To deal with this issue, we propose a novel Adversarial-Metric Learning (AML) model for audio-visual matching. AML aims to generate a modality-independent representation for each person in each modality via adversarial learning, while simultaneously learns a robust similarity measure for cross-modality matching via metric learning. By integrating the discriminative modality-independent representation and robust cross-modality metric learning into an end-to-end trainable deep network, AML can overcome the heterogeneous issue with promising performance for audio-visual matching. Experiments on the various audio-visual learning tasks, including audio-visual matching, audio-visual verification and audio-visual retrieval on benchmark dataset demonstrate the effectiveness of the proposed AML model. The implementation codes are available on https://github.com/MLanHu/AML. |
| WOS关键词 | FACE ; IDENTITY ; SPEECH ; VOICE |
| 资助项目 | Major Project for New Generation of AI[2018AAA0100400] ; National Natural Science Foundation of China[61976002] ; National Natural Science Foundation of China[62076004] ; Natural Science Foundation of Anhui Higher Education Institutions of China[KJ2019A0033] ; Open Project Program of the National Laboratory of Pattern Recognition (NLPR)[201900046] ; Cooperative Research Project Program of Nanjing Artificial Intelligence Chip Research, Institute of Automation, Chinese Academy of Sciences |
| WOS研究方向 | Computer Science ; Telecommunications |
| 语种 | 英语 |
| WOS记录号 | WOS:000745524300026 |
| 出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| 资助机构 | Major Project for New Generation of AI ; National Natural Science Foundation of China ; Natural Science Foundation of Anhui Higher Education Institutions of China ; Open Project Program of the National Laboratory of Pattern Recognition (NLPR) ; Cooperative Research Project Program of Nanjing Artificial Intelligence Chip Research, Institute of Automation, Chinese Academy of Sciences |
| 源URL | [http://ir.ia.ac.cn/handle/173211/47343]  |
| 专题 | 自动化研究所_智能感知与计算研究中心 |
| 通讯作者 | Jiang, Bo |
| 作者单位 | 1.IIT, Dept Comp Sci, Chicago, IL 60616 USA 2.Minist Educ, Key Lab Intelligent Comp & Signal Proc, Anhui Prov Key Lab Multimodal Cognit Computat, Hefei, Peoples R China 3.Anhui Univ, Sch Comp Sci & Technol, Hefei 230601, Peoples R China 4.Anhui Univ, Inst Phys Sci & Informat Technol, Hefei 230601, Peoples R China 5.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China |
| 推荐引用方式 GB/T 7714 | Zheng, Aihua,Hu, Menglan,Jiang, Bo,et al. Adversarial-Metric Learning for Audio-Visual Cross-Modal Matching[J]. IEEE TRANSACTIONS ON MULTIMEDIA,2022,24:338-351. |
| APA | Zheng, Aihua,Hu, Menglan,Jiang, Bo,Huang, Yan,Yan, Yan,&Luo, Bin.(2022).Adversarial-Metric Learning for Audio-Visual Cross-Modal Matching.IEEE TRANSACTIONS ON MULTIMEDIA,24,338-351. |
| MLA | Zheng, Aihua,et al."Adversarial-Metric Learning for Audio-Visual Cross-Modal Matching".IEEE TRANSACTIONS ON MULTIMEDIA 24(2022):338-351. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。