METATR: A Multilingual, Evolving Benchmark for Automatic Text Recognition

A new benchmark called METATR has been introduced to evaluate Automatic Text Recognition (ATR) systems, particularly Vision-Large Language Models (vLLMs). This benchmark aims to address the limitations of existing datasets that primarily feature modern, printed texts in English, thereby enhancing the evaluation of ATR models across diverse document types.

The development of METATR is significant as it facilitates a more accurate assessment of ATR systems in real-world applications, ensuring that models can be effectively compared and selected based on their performance with varied document collections.