Bridging Music and Image via Cross-Modal Ranking Analysis
文献类型：期刊论文
| 作者 | Wu, Xixuan; Qiao, Yu; Wang, Xiaogang; Tang, Xiaoou |
| 刊名 | IEEE TRANSACTIONS ON MULTIMEDIA
 |
| 出版日期 | 2016 |
| 英文摘要 | Human perceptions of music and image are closely related to each other, since both can inspire similar human sensations, such as emotion, motion, and power. This paper aims to explore whether and how music and image can be automatically matched by machines. The main contributions are three aspects. First, we construct a benchmark dataset composed of more than 45 000 music-image pairs. Human labelers are recruited to annotate whether these pairs are well-matched or not. The results show that they generally agree with each other on the matching degree of music-image pairs. Secondly, we investigate suitable semantic representations of music and image for this cross-modal matching task. In particular, we adopt lyrics as a middle-media to connect music and image, and design a set of lyric-based attributes for image representation. Thirdly, we propose cross-modal ranking analysis (CMRA) to learn the semantic similarity between music and image with ranking labeling information. CMRA aims to find the optimal embedding spaces for both music and image in the sense of maximizing the ordinal margin between music-image pairs. The proposed method is able to learn the non-linear relationship between music and image, and to integrate heterogeneous ranking data from different modalities into a unified space. Experimental results demonstrate that the proposed method outperforms state-of-the-art cross-modal methods in the music-image matching task, and achieves a consistency rate of 91.5% with human labelers. |
| 收录类别 | SCI |
| 原文出处 | http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=7457690&tag=1 |
| 语种 | 英语 |
| 源URL | [http://ir.siat.ac.cn:8080/handle/172644/9800]  |
| 专题 | 深圳先进技术研究院_集成所 |
| 作者单位 | IEEE TRANSACTIONS ON MULTIMEDIA |
| 推荐引用方式 GB/T 7714 | Wu, Xixuan,Qiao, Yu,Wang, Xiaogang,et al. Bridging Music and Image via Cross-Modal Ranking Analysis[J]. IEEE TRANSACTIONS ON MULTIMEDIA,2016. |
| APA | Wu, Xixuan,Qiao, Yu,Wang, Xiaogang,&Tang, Xiaoou.(2016).Bridging Music and Image via Cross-Modal Ranking Analysis.IEEE TRANSACTIONS ON MULTIMEDIA. |
| MLA | Wu, Xixuan,et al."Bridging Music and Image via Cross-Modal Ranking Analysis".IEEE TRANSACTIONS ON MULTIMEDIA (2016). |
入库方式： OAI收割
来源：深圳先进技术研究院
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。