Hierarchical structure understanding in complex tables with VLLMs: a benchmark and experiments

The study on Vision Large Language Models (VLLMs) focuses on their ability to comprehend the hierarchical structures of tables found in scientific literature. Utilizing the PubTables-1M dataset, researchers introduced a benchmark known as Complex Hierarchical Tables (CHiTab), which consists of intricate tables with hierarchical headings. Through various prompt engineering strategies, the study assessed multiple state-of-the-art VLLMs, both in their standard forms and after fine-tuning for the task. The results indicated that even generic VLLMs, which were not specifically designed for table comprehension, could perform well in understanding these structures. Furthermore, the performance of VLLMs was compared to that of humans on a smaller set of tables, providing insights into the models' capabilities. This research not only highlights the potential of VLLMs in interpreting structured data but also offers guidance for future advancements in integrating such understanding into general-p…