Athena: Enhancing Multimodal Reasoning with Data-efficient Process Reward Models

EgoVITA has been introduced as a reinforcement learning framework designed to enhance the reasoning capabilities of multimodal large language models (MLLMs) by enabling them to plan and verify actions from both egocentric and exocentric perspectives. This dual-phase approach allows the model to predict future actions from a first-person viewpoint and subsequently verify these actions from a third-person perspective, addressing challenges in understanding dynamic visual contexts.

The introduction of LAST, or LeArning to Think in Space and Time, aims to enhance the capabilities of vision-language models (VLMs) by enabling them to better understand 3D spatial contexts and long video sequences using only 2D images as input. This approach contrasts with existing methods that typically address 3D and video tasks separately.

The recent introduction of BeMyEyes presents a modular, multi-agent framework aimed at enhancing Large Language Models (LLMs) by enabling them to collaborate with Vision Language Models (VLMs) for multimodal reasoning. This approach orchestrates the interaction between adaptable VLMs as perceivers and powerful LLMs as reasoners, facilitating improved perception and reasoning capabilities.

ChainV has been introduced as a framework that enhances multimodal reasoning by dynamically integrating visual hints into the reasoning process, addressing issues of redundancy in lengthy reasoning chains. The framework selects visual patches based on previous reasoning steps and refines them by identifying the most representative atomic visual hints, improving the efficiency of reasoning models.

EvoLMM, a self-evolving framework for large multimodal models, has been introduced to enhance reasoning capabilities without relying on human-annotated data. This framework consists of two cooperative agents: a Proposer that generates diverse questions and a Solver that answers them through a continuous self-rewarding process. This innovation aims to improve the autonomy and scalability of multimodal models.