Embodied Tree of Thoughts: Deliberate Manipulation Planning with Embodied World Model

A novel approach called SATGround has been introduced to enhance visual grounding in remote sensing through a structured localization mechanism that fine-tunes a pretrained vision-language model (VLM) on diverse instruction-following tasks. This method significantly improves the model's ability to localize objects in complex satellite imagery, achieving a 24.8% relative improvement over previous methods in visual grounding benchmarks.

A new framework named LaFG has been introduced for fine-grained image retrieval, which utilizes large language models (LLMs) and vision-language models (VLMs) to convert class names into detailed attribute-level descriptions. This approach aims to enhance the modeling of comparability among cross-category details, addressing limitations of existing methods that rely on sparse one-hot labels.

A new framework called AerialVP has been introduced to enhance image perception in UAVs by improving task prompts used in Vision-Language Models (VLMs). This framework addresses challenges such as target confusion and scale variations that arise from the complex nature of UAV imagery, which traditional VLMs struggle to interpret effectively.

CORE-3D introduces a novel approach to 3D scene understanding by utilizing context-aware open-vocabulary retrieval through embeddings, enhancing the accuracy of object-level masks in complex environments. This method leverages SemanticSAM and a refined CLIP encoding strategy to improve 3D semantic segmentation, addressing limitations of previous models that produced fragmented masks and inaccurate semantic assignments.

A new approach to procedural mistake detection (PMD) has been introduced, focusing on classifying task execution success through egocentric video analysis. This method emphasizes generating visual self-dialog rationales to enhance decision-making transparency, leveraging advanced vision-and-language models (VLMs) and establishing baseline metrics for coherence in generated rationales.

The introduction of PoSh, a new metric utilizing scene graphs, aims to enhance the evaluation of Vision-Language Models (VLMs) in generating detailed image descriptions. Traditional metrics like CIDEr and SPICE have struggled with longer texts, often failing to accurately assess compositional understanding and specific errors. PoSh seeks to provide a more interpretable and replicable scoring system, validated through the DOCENT dataset, which includes expert-written references for artwork.