LiveOIBench: Can Large Language Models Outperform Human Contestants in Informatics Olympiads?

arXiv — cs.CLTuesday, December 23, 2025 at 5:00:00 AM
  • LiveOIBench has been introduced as a new benchmark designed to evaluate the coding capabilities of large language models (LLMs) against human contestants in Informatics Olympiads. This benchmark features 403 expert-curated competitive programming problems sourced from 72 official contests conducted between 2023 and 2025, each accompanied by an average of 60 test cases.
  • The development of LiveOIBench is significant as it addresses current limitations in coding benchmarks, such as the lack of challenging problems and insufficient test case coverage, thereby providing a more robust evaluation framework for LLMs.
  • This initiative reflects ongoing efforts to enhance the performance of LLMs in complex tasks, highlighting a broader trend in AI research focused on improving model accuracy and reliability. The introduction of various benchmarks, such as CIFE and Offscript, underscores the importance of rigorous evaluation methods in advancing AI capabilities.
— via World Pulse Now AI Editorial System

Was this article worth reading? Share it

Recommended apps based on your readingExplore all apps
Continue Readings
Can A.I. Generate New Ideas?
NeutralArtificial Intelligence
OpenAI has launched GPT-5.2, its latest AI model, which is designed to enhance productivity and has shown mixed results in tests compared to its predecessor, GPT-5.1. This development comes amid increasing competition from Google's Gemini 3, which has rapidly gained a significant user base.
Compliance-to-Code: Enhancing Financial Compliance Checking via Code Generation
NeutralArtificial Intelligence
The recent development in financial compliance checking involves the introduction of Compliance-to-Code, which leverages Regulatory Technology and Large Language Models to automate the conversion of complex regulatory text into executable compliance logic. This innovation aims to address the challenges posed by intricate financial regulations, particularly in the context of Chinese-language regulations, where existing models have shown suboptimal performance due to various limitations.
QuantEval: A Benchmark for Financial Quantitative Tasks in Large Language Models
NeutralArtificial Intelligence
The introduction of QuantEval marks a significant advancement in evaluating Large Language Models (LLMs) in financial quantitative tasks, focusing on knowledge-based question answering, mathematical reasoning, and strategy coding. This benchmark incorporates a backtesting framework that assesses the performance of model-generated strategies using financial metrics, providing a more realistic evaluation of LLM capabilities.
Measuring Iterative Temporal Reasoning with Time Puzzles
NeutralArtificial Intelligence
The introduction of Time Puzzles marks a significant advancement in evaluating iterative temporal reasoning in large language models (LLMs). This task combines factual temporal anchors with cross-cultural calendar relations, generating puzzles that challenge LLMs' reasoning capabilities. Despite the simplicity of the dataset, models like GPT-5 achieved only 49.3% accuracy, highlighting the difficulty of the task.
Focus, Merge, Rank: Improved Question Answering Based on Semi-structured Knowledge Bases
PositiveArtificial Intelligence
A new framework named FocusedRetriever has been introduced to enhance multi-hop question answering by leveraging Semi-Structured Knowledge Bases (SKBs), which connect unstructured content to structured data. This innovative approach integrates various components, including VSS-based entity search and LLM-based query generation, outperforming existing methods in the STaRK benchmark tests.
Improving Zero-shot ADL Recognition with Large Language Models through Event-based Context and Confidence
PositiveArtificial Intelligence
A recent study has proposed enhancements to zero-shot recognition of Activities of Daily Living (ADLs) using Large Language Models (LLMs) by implementing event-based segmentation and a novel method for estimating prediction confidence. This approach aims to improve the accuracy of sensor-based recognition systems in smart homes, which are crucial for applications in healthcare and safety management.
Reasoning Matters for 3D Visual Grounding
PositiveArtificial Intelligence
Recent advancements in Large Language Models (LLMs) have highlighted the importance of reasoning in 3D visual grounding, a task that remains challenging due to the limitations of current models. The proposed 3D visual grounding data pipeline aims to synthesize data automatically, enhancing the ability to predict referring objects in 3D environments.
Detecting High-Stakes Interactions with Activation Probes
NeutralArtificial Intelligence
A recent study published on arXiv explores the use of activation probes to detect high-stakes interactions in Large Language Models (LLMs), focusing on interactions that may lead to significant harm. The research evaluates various probe architectures trained on synthetic data, demonstrating their robust generalization to real-world scenarios and highlighting their computational efficiency compared to traditional monitoring methods.

Ready to build your own newsroom?

Subscribe to unlock a personalised feed, podcasts, newsletters, and notifications tailored to the topics you actually care about