Benchmarking Gaslighting Negation Attacks Against Reasoning Models

arXiv — cs.CVThursday, December 18, 2025 at 5:00:00 AM
  • Recent research evaluated the vulnerability of leading reasoning models, including OpenAI's o4-mini, Claude-3.7-Sonnet, and Gemini-2.5-Flash, to gaslighting negation attacks, which significantly reduced their accuracy by 25-29% on average across multimodal benchmarks like MMMU, MathVista, and CharXiv. This highlights a critical gap in the robustness of these advanced AI systems against manipulative inputs.
  • The findings underscore the challenges faced by top-tier AI models in maintaining accuracy under adversarial conditions, raising concerns about their reliability in real-world applications where user feedback can be misleading or deceptive.
  • This situation reflects ongoing debates in the AI community regarding the biases inherent in large language models and their evaluation methods, as well as the need for improved benchmarks and diagnostic tools like GaslightingBench-R to better assess and enhance model resilience against adversarial prompts.
— via World Pulse Now AI Editorial System

Was this article worth reading? Share it

Recommended apps based on your readingExplore all apps
Continue Readings
**Revolutionizing Conversational AI: ChatGPT's App Store Launch**
PositiveArtificial Intelligence
OpenAI has launched an app store for its ChatGPT chatbot, enabling developers to create and distribute custom applications within the platform. This initiative aims to enhance user experience by providing access to a variety of new features, games, and tools directly through ChatGPT's interface.
Sources: OpenAI's new fundraising round could value it at as much as $830B; it aims to raise up to $100B and complete the round by the end of Q1 at the earliest (Wall Street Journal)
NeutralArtificial Intelligence
OpenAI is reportedly seeking to raise up to $100 billion in a new fundraising round, which could value the company at approximately $830 billion. The completion of this funding round is anticipated by the end of the first quarter of 2026 at the earliest. Concerns about a potential AI bubble have been raised, affecting the valuations of tech companies, including OpenAI.
OpenAI Is ‘Definitely Not’ Too Big to Fail, Economist Says
NeutralArtificial Intelligence
A leading economist has stated that OpenAI is 'definitely not' too big to fail, suggesting that the potential collapse of the AI bubble would not have catastrophic consequences. This perspective comes amid growing concerns about the sustainability of investments in artificial intelligence, particularly following OpenAI's significant market fluctuations.
In an experiment, Claude ran a vending machine in the WSJ newsroom and lost $1,000+ after it dropped prices to zero, gave away a free PlayStation, and more (Joanna Stern/Wall Street Journal)
NeutralArtificial Intelligence
In a recent experiment, Anthropic's AI model Claude operated a vending machine in the Wall Street Journal newsroom, resulting in a loss of over $1,000. The AI dropped prices to zero, distributed a free PlayStation, and made unusual purchases, including a live fish, showcasing its unpredictable behavior in a real-world scenario.
ChatGPT launches an app store, lets developers know it’s open for business
PositiveArtificial Intelligence
OpenAI has launched an app store for its ChatGPT chatbot, allowing developers to create and distribute custom applications within the platform. This initiative aims to enhance user experience by providing access to a variety of new features, tools, and games, marking a significant expansion of ChatGPT's functionality.
OpenAI Has Declared ‘Code Red’ Multiple Times, Executive Says
NeutralArtificial Intelligence
OpenAI CEO Sam Altman declared a 'code red' for the company's ChatGPT platform, emphasizing the urgent need for improvements amid rising competition from Google's Gemini 3. This declaration marks a significant moment for OpenAI, highlighting ongoing challenges in maintaining its leadership in the AI sector.
Why British politicians are flocking to American tech giants
PositiveArtificial Intelligence
Former British Chancellor George Osborne has joined OpenAI as managing director and head of OpenAI for Countries, a role focused on building partnerships with governments globally for AI initiatives. He will also lead Coinbase's internal advisory council. This move underscores a trend of political figures transitioning into significant roles within major tech companies.
GPT-5.2 tops OpenAI's new FrontierScience test but struggles with real research problems
NeutralArtificial Intelligence
OpenAI has introduced GPT-5.2, which has excelled in its new FrontierScience benchmark, outperforming previous models but revealing limitations in tackling real-world research challenges. This benchmark aims to assess AI capabilities at both Olympic and research levels.

Ready to build your own newsroom?

Subscribe to unlock a personalised feed, podcasts, newsletters, and notifications tailored to the topics you actually care about