MOCHA: Multi-modal Objects-aware Cross-arcHitecture Alignment

The introduction of MultiPriv marks a significant advancement in the evaluation of individual-level privacy reasoning within Vision-Language Models (VLMs). This benchmark addresses the inadequacies of current privacy assessments, which primarily focus on privacy perception rather than the ability of VLMs to link distributed information and construct individual profiles. The framework includes a novel bilingual multimodal dataset that features synthetic individual profiles linked to sensitive attributes.

PhyBlock has been introduced as a progressive benchmark aimed at evaluating vision-language models (VLMs) on their physical understanding and planning capabilities through robotic 3D block assembly tasks. This benchmark features a four-level cognitive hierarchy assembly task and includes 2,600 tasks to assess models on spatial reasoning and physical comprehension.

Recent research has examined whether Vision-Language Models (VLMs) comprehend visual persuasion, which influences human attitudes and decisions. A new dataset was created for binary persuasiveness judgment, introducing a taxonomy of Visual Persuasive Factors (VPFs) that includes various levels of visual cues. The analysis indicates that VLMs tend to overestimate high persuasiveness and struggle with low/mid-level features, while high-level semantic alignment is a strong predictor of human judgment.

The introduction of MirageTVQA, a new benchmark for evaluating Vision-Language Models (VLMs), highlights the significant performance gaps in existing datasets that primarily focus on monolingual and visually perfect tables. This benchmark includes nearly 60,000 QA pairs across 24 languages and incorporates realistic noise to better reflect real-world scenarios.

VLA-Pruner has been introduced as a novel method for token pruning in Vision-Language-Action (VLA) models, addressing the inefficiencies of existing approaches that focus solely on semantic salience. This method aims to enhance real-time deployment of VLA models by retaining critical information necessary for action generation while discarding redundant visual tokens.