中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 自动化研究所
  3. 中国科学院自动化研究所
  4. 多模态人工智能系统全国重点实验室
  5. 互联网大数据与安全信息学研究中心
Towards Human-Machine Recognition Alignment: An Adversarilly Robust Multimodal Retrieval Hashing Framework

文献类型：期刊论文

作者Zhang, Xingwei1,2; Zheng, Xiaolong1,2; Liu, Bin3; Wang, Xiao1,2; Mao, Wenji1,2; Zeng, Daniel Dajun1,2; Wang, Fei-Yue1,2
刊名IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS
出版日期2022-08-29
页码13
关键词Training Task analysis Semantics Perturbation methods Feature extraction Computational modeling Robustness Adversarial perturbation adversarially robust training deep hashing multimodal retrieval
ISSN号2329-924X
DOI10.1109/TCSS.2022.3199819
通讯作者Zheng, Xiaolong(xiaolong.zheng@ia.ac.cn)
英文摘要The multimodality nature of web data has necessitated complex multimodal information retrieval for a wide range of web applications. Deep neural networks (DNNs) have been widely employed to extract semantic features from raw samples to improve retrieval accuracy. In addition, hashing is widely used to improve computational and storage efficiency. As such, deep hashing frameworks have been applied for multimodal retrieval tasks. However, there is still a great recognitive gap between primate brain structure-inspired DNNs and humans. On computer vision tasks, well-crafted DNN models can be easily defeated by invisible small attacks, and this phenomenon indicates a large recognition gap between DNN models and humans. Recently, adversarial defense methods have been shown to improve the human-machine recognition alignment in several classification tasks. However, the robustness problem on the retrieval tasks, especially on the deep hashing-based multimodal retrieval models, is still not well studied. Therefore, in this article, we present an adversarially robust training mechanism to improve model robustness for the purpose of human-machine recognition alignment on retrieval tasks. Through extensive experimental results on several social multimodal retrieval benchmarks, we show that the robust training hashing framework proposed can mitigate the recognition gap on retrieval tasks. Our study highlights the necessity of robustness enhancement on deep hashing models.
WOS关键词NEURAL-NETWORKS ; MODELS
资助项目Ministry of Science and Technology of China[2020AAA0108401] ; Natural Science Foundation of China[72225011] ; Natural Science Foundation of China[71621002]
WOS研究方向Computer Science
语种英语
WOS记录号WOS:000849234000001
出版者IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
资助机构Ministry of Science and Technology of China ; Natural Science Foundation of China
源URL[http://ir.ia.ac.cn/handle/173211/50056]  
专题自动化研究所_复杂系统管理与控制国家重点实验室_互联网大数据与安全信息学研究中心
自动化研究所_复杂系统管理与控制国家重点实验室_先进控制与自动化团队
通讯作者Zheng, Xiaolong
作者单位1.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 101408, Peoples R China
2.Chinese Acad Sci, State Key Lab Management & Control Complex Syst, Inst Automat, Beijing 100190, Peoples R China
3.West Virginia Univ, Dept Management Informat Syst, Morgantown, WV 26506 USA
推荐引用方式
GB/T 7714		 Zhang, Xingwei,Zheng, Xiaolong,Liu, Bin,et al. Towards Human-Machine Recognition Alignment: An Adversarilly Robust Multimodal Retrieval Hashing Framework[J]. IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS,2022:13.
APA Zhang, Xingwei.,Zheng, Xiaolong.,Liu, Bin.,Wang, Xiao.,Mao, Wenji.,...&Wang, Fei-Yue.(2022).Towards Human-Machine Recognition Alignment: An Adversarilly Robust Multimodal Retrieval Hashing Framework.IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS,13.
MLA Zhang, Xingwei,et al."Towards Human-Machine Recognition Alignment: An Adversarilly Robust Multimodal Retrieval Hashing Framework".IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS (2022):13.

入库方式： OAI收割

来源：自动化研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。