I-RAVEN-X: Benchmarking Generalization and Robustness of Analogical and Mathematical Reasoning in Large Language and Reasoning Models

The introduction of I-RAVEN-X marks a significant advancement in evaluating the capabilities of Large Language Models (LLMs) and Large Reasoning Models (LRMs) in analogical and mathematical reasoning. By enhancing operand complexity and introducing perceptual uncertainty, this benchmark aims to provide a more rigorous assessment of these models' generalization and robustness. The findings indicate that LRMs outperform LLMs in productivity and systematicity, which is crucial for developing more effective AI systems that can handle complex reasoning tasks.