mmPISA-bench: Do LLMs Reason Equally Well Across 43 Languages?

The introduction of mmPISA-bench marks a significant advancement in multilingual reasoning benchmarks, derived from the OECD Programme for International Student Assessment (PISA). This benchmark consists of 25 multiple-choice questions translated into 43 languages, allowing for a comprehensive evaluation of large language models (LLMs) across diverse linguistic contexts.

The findings indicate that modern LLMs can reason effectively across all evaluated languages, achieving accuracy levels comparable to human test-takers, which underscores the potential of these models in educational assessments and multilingual applications.

This development highlights ongoing efforts to enhance the capabilities of LLMs in multilingual contexts, reflecting a broader trend in AI research that seeks to address performance disparities across languages and improve the reliability of machine translations, as seen in related benchmarks for summarization and reasoning.