中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
Sycophancy in vision-language models: A systematic analysis and an inference-time mitigation framework

文献类型：期刊论文

作者Zhao, Yunpu2; Zhang, Rui3; Xiao, Junbin4; Ke, Changxin3; Hou, Ruibo1; Hao, Yifan3; Li, Ling5
刊名NEUROCOMPUTING
出版日期2026
卷号659页码:14
关键词Vision-language models Contrastive decoding Model hallucinations
ISSN号0925-2312
DOI10.1016/j.neucom.2025.131217
英文摘要Large Vision-Language Models (LVLMs) have shown significant capability in vision-language understanding. However, one critical issue that persists in these models is sycophancy, where models are unduly influenced by leading or deceptive prompts, resulting in biased outputs and hallucinations. Despite the rapid development of LVLMs, evaluating and mitigating sycophancy remains largely under-explored. In this work, we fill this gap by systematically analyzing sycophancy across multiple vision-language benchmarks and propose an inference-time mitigation framework. We curate leading queries and quantify the susceptibility of state-of-the-art LVLMs to prompt-induced bias, revealing consistent performance degradation and instability across models and tasks. Our analysis further uncovers model-specific behavioral traits, such as sentiment sensitivity and prediction polarity shifts under sycophancy. To mitigate these issues, we propose a training-free, model-agnostic framework that operates entirely at inference time. Our approach first employs a query neutralizer, leveraging a language model to suppress implicit sycophantic bias in user queries. We then introduce a sycophancy-aware contrastive decoding mechanism that dynamically recalibrates token-level output distributions by contrasting responses to neutralized and leading queries. Finally, an adaptive logits refinement module further modifies the contrasted logits by integrating both an adaptive plausibility filter and query sentiment scaler, ensuring coherent and robust generation. Extensive experiments demonstrate that this framework effectively mitigates sycophancy across all evaluated models, while maintaining performance on neutral prompts. Our results suggest that sycophancy in LVLMs is a general and urgent challenge, and that inference-time strategies offer a promising path toward trustworthy multimodal reasoning.
资助项目National Key R&D Program of China[2023YFB4502200] ; NSF of China[62302478] ; NSF of China[U22A2028] ; NSF of China[62341411] ; Strategic Priority Research Program of the Chinese Academy of Sciences[XDB0660200] ; Strategic Priority Research Program of the Chinese Academy of Sciences[XDB0660201] ; Strategic Priority Research Program of the Chinese Academy of Sciences[XDB0660202] ; CAS Project for Young Scientists in Basic Research[YSBR-029] ; Youth Innovation Promotion Association CAS
WOS研究方向Computer Science
语种英语
WOS记录号WOS:001601182700001
出版者ELSEVIER
源URL[http://119.78.100.204/handle/2XEOYT63/41592]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Zhang, Rui
作者单位1.Univ Illinois, Chicago, IL 61820 USA
2.Univ Sci & Technol China, Sch Comp Sci & Technol, Hefei 230026, Peoples R China
3.Chinese Acad Sci, Inst Comp Technol, State Key Lab Processors, Beijing 100190, Peoples R China
4.Natl Univ Singapore, Acad Beijing, Dept Comp Sci, Key Comp Technol, Singapore 119077, Singapore
5.Chinese Acad Sci, Intelligent Software Res Ctr, Inst Software, Beijing 100190, Peoples R China
推荐引用方式
GB/T 7714		 Zhao, Yunpu,Zhang, Rui,Xiao, Junbin,et al. Sycophancy in vision-language models: A systematic analysis and an inference-time mitigation framework[J]. NEUROCOMPUTING,2026,659:14.
APA Zhao, Yunpu.,Zhang, Rui.,Xiao, Junbin.,Ke, Changxin.,Hou, Ruibo.,...&Li, Ling.(2026).Sycophancy in vision-language models: A systematic analysis and an inference-time mitigation framework.NEUROCOMPUTING,659,14.
MLA Zhao, Yunpu,et al."Sycophancy in vision-language models: A systematic analysis and an inference-time mitigation framework".NEUROCOMPUTING 659(2026):14.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。