Gemini 3 Pro and GPT-5 still fail at complex physics tasks designed for real scientific research

THE DECODERSunday, November 23, 2025 at 4:20:07 PM
Gemini 3 Pro and GPT-5 still fail at complex physics tasks designed for real scientific research
  • A new physics benchmark named CritPt has revealed that leading AI models, including Gemini 3 Pro and GPT-5, are unable to perform complex physics tasks at the level required for early-stage PhD research, indicating significant limitations in their capabilities as autonomous scientific tools.
  • This development is critical as it highlights the ongoing challenges faced by advanced AI models in achieving reliability and accuracy in scientific research, which is essential for their acceptance and integration into academic and professional environments.
  • The findings underscore a broader concern regarding the reliability of AI models, as Gemini 3 Pro, despite being recognized as a top performer in other benchmarks, still struggles with high hallucination rates, raising questions about the readiness of AI for complex scientific applications.
— via World Pulse Now AI Editorial System

Was this article worth reading? Share it

Recommended apps based on your readingExplore all apps
Continue Readings
Can A.I. Generate New Ideas?
NeutralArtificial Intelligence
OpenAI has launched GPT-5.2, its latest AI model, which is designed to enhance productivity and has shown mixed results in tests compared to its predecessor, GPT-5.1. This development comes amid increasing competition from Google's Gemini 3, which has rapidly gained a significant user base.
Measuring Iterative Temporal Reasoning with Time Puzzles
NeutralArtificial Intelligence
The introduction of Time Puzzles marks a significant advancement in evaluating iterative temporal reasoning in large language models (LLMs). This task combines factual temporal anchors with cross-cultural calendar relations, generating puzzles that challenge LLMs' reasoning capabilities. Despite the simplicity of the dataset, models like GPT-5 achieved only 49.3% accuracy, highlighting the difficulty of the task.
From Rows to Reasoning: A Retrieval-Augmented Multimodal Framework for Spreadsheet Understanding
PositiveArtificial Intelligence
A new framework called From Rows to Reasoning (FRTR) has been introduced to enhance the reasoning capabilities of Large Language Models (LLMs) when dealing with complex spreadsheets. This framework includes FRTR-Bench, a benchmark featuring 30 enterprise-grade Excel workbooks, which aims to improve the understanding of multimodal data by breaking down spreadsheets into granular components.
APEX-SWE
NeutralArtificial Intelligence
The AI Productivity Index for Software Engineering (APEX-SWE) has been introduced as a benchmark to evaluate the economic viability of frontier AI models in executing software engineering tasks. This benchmark assesses two innovative task types: integration tasks, which involve creating end-to-end systems, and observability tasks, which focus on debugging production failures using telemetry signals. Eight frontier models were evaluated, with Gemini 3 Pro achieving the highest Pass@1 score of 25%.
KidVis: Do Multimodal Large Language Models Possess the Visual Perceptual Capabilities of a 6-Year-Old?
NeutralArtificial Intelligence
A new benchmark called KidVis has been introduced to evaluate the visual perceptual capabilities of Multimodal Large Language Models (MLLMs), specifically assessing their performance against that of 6-7 year old children across six atomic visual capabilities. The results reveal a significant performance gap, with human children scoring an average of 95.32 compared to GPT-5's score of 67.33.

Ready to build your own newsroom?

Subscribe to unlock a personalised feed, podcasts, newsletters, and notifications tailored to the topics you actually care about