Multimodal Similarity Gaussian Process Latent Variable Model
文献类型：期刊论文
| 作者 | Song, Guoli1,2,3; Wang, Shuhui2; Huang, Qingming1,2,3; Tian, Qi4 |
| 刊名 | IEEE TRANSACTIONS ON IMAGE PROCESSING
 |
| 出版日期 | 2017-09-01 |
| 卷号 | 26期号:9页码:4168-4181 |
| 关键词 | Multimodal learning Gaussian processes similarity preservation |
| ISSN号 | 1057-7149 |
| DOI | 10.1109/TIP.2017.2713045 |
| 英文摘要 | Data from real applications involve multiple modalities representing content with the same semantics from complementary aspects. However, relations among heterogeneous modalities are simply treated as observation-to-fit by existing work, and the parameterized modality specific mapping functions lack flexibility in directly adapting to the content divergence and semantic complicacy in multimodal data. In this paper, we build our work based on the Gaussian process latent variable model (GPLVM) to learn the non-parametric mapping functions and transform heterogeneous modalities into a shared latent space. We propose multimodal Similarity Gaussian Process latent variable model (m-SimGP), which learns the mapping functions between the intra-modal similarities and latent representation. We further propose multimodal distance-preserved similarity GPLVM (m-DSimGP) to preserve the intra-modal global similarity structure, and multimodal regularized similarity GPLVM (m-RSimGP) by encouraging similar/dissimilar points to be similar/dissimilar in the latent space. We propose m-DRSimGP, which combines the distance preservation in m-DSimGP and semantic preservation in m-RSimGP to learn the latent representation. The overall objective functions of the four models are solved by simple and scalable gradient decent techniques. They can be applied to various tasks to discover the nonlinear correlations and to obtain the comparable low-dimensional representation for heterogeneous modalities. On five widely used real-world data sets, our approaches outperform existing models on cross-modal content retrieval and multimodal classification. |
| 资助项目 | National Natural Science Foundation of China[61672497] ; National Natural Science Foundation of China[61332016] ; National Natural Science Foundation of China[61620106009] ; National Natural Science Foundation of China[61650202] ; National Natural Science Foundation of China[U1636214] ; National Natural Science Foundation of China[61572488] ; National Natural Science Foundation of China[61429201] ; National Basic Research Program of China (973 Program)[2015CB351802] ; Key Research Program of Frontier Sciences of CAS[QYZDJ-SSW-SYS013] ; ARO[W911NF-15-1-0290] ; Faculty Research Gift Awards by NEC Laboratory of America ; Faculty Research Gift Awards by NEC Laboratory of Blippar |
| WOS研究方向 | Computer Science ; Engineering |
| 语种 | 英语 |
| WOS记录号 | WOS:000404288000006 |
| 出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| 源URL | [http://119.78.100.204/handle/2XEOYT63/7107]  |
| 专题 | 中国科学院计算技术研究所期刊论文_英文 |
| 通讯作者 | Wang, Shuhui |
| 作者单位 | 1.Univ Chinese Acad Sci, Sch Comp & Control Engn, Beijing 100049, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 3.Univ Chinese Acad Sci, Key Lab Big Data Min & Knowledge Management, Beijing 101408, Peoples R China 4.Univ Texas San Antonio, Dept Comp Sci, San Antonio, TX 78249 USA |
| 推荐引用方式 GB/T 7714 | Song, Guoli,Wang, Shuhui,Huang, Qingming,et al. Multimodal Similarity Gaussian Process Latent Variable Model[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2017,26(9):4168-4181. |
| APA | Song, Guoli,Wang, Shuhui,Huang, Qingming,&Tian, Qi.(2017).Multimodal Similarity Gaussian Process Latent Variable Model.IEEE TRANSACTIONS ON IMAGE PROCESSING,26(9),4168-4181. |
| MLA | Song, Guoli,et al."Multimodal Similarity Gaussian Process Latent Variable Model".IEEE TRANSACTIONS ON IMAGE PROCESSING 26.9(2017):4168-4181. |
入库方式： OAI收割
来源：计算技术研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。