Seeing is Believing: Rich-Context Hallucination Detection for MLLMs via Backward Visual Grounding

The article discusses a novel training-free pipeline called Foresee, designed for image forgery detection using vanilla multimodal large language models (MLLMs). As artificial intelligence-generated content technologies advance, traditional image forgery detection methods struggle with generalization and interpretability. Foresee aims to address these challenges by enabling lightweight inference without additional training, showcasing the inherent potential of MLLMs in image forgery analysis.

This study explores the use of Large Language Models (LLMs), specifically GPT-4o, for evaluating short-answer quizzes and project reports in an undergraduate Computational Linguistics course. The research involved approximately 50 students and 14 project teams, comparing LLM-generated scores with human evaluations from teaching assistants. Results indicated a strong correlation between LLM and human scores, achieving up to 0.98 correlation and exact score agreement in 55% of quiz cases, while showing variability in scoring open-ended responses.

The paper introduces UniSER, a foundational model designed for the unified removal of soft effects in digital images, such as lens flare, haze, shadows, and reflections. These effects often degrade image aesthetics while leaving underlying pixels visible. Existing solutions typically focus on individual issues, leading to specialized models that lack scalability. In contrast, UniSER leverages the commonality of semi-transparent occlusions to effectively address various soft effect degradations, enhancing image restoration capabilities beyond current generalist models that require detailed prom…

CAR-Scenes is a frame-level dataset designed for autonomous driving, facilitating the training and evaluation of vision-language models (VLMs) for scene-level understanding. The dataset comprises 5,192 annotated images from sources like Argoverse, Cityscapes, KITTI, and nuScenes, utilizing a comprehensive 28-key category/sub-category knowledge base. The annotations are generated through a GPT-4o-assisted pipeline with human verification, providing detailed attributes and supporting semantic retrieval and risk-aware scenario mining.

Recent advancements in multimodal large language models (MLLMs) have significantly improved vision-language understanding. However, their high computational demands hinder their use in resource-limited environments like robotics and personal assistants. Traditional Transformer-based methods face efficiency challenges due to quadratic complexity, and smaller models often fail to capture critical visual details for fine-grained reasoning tasks. Viper-F1 introduces a Hybrid State-Space Vision-Language Model that utilizes Liquid State-Space Dynamics and a Token-Grid Correlation Module to enhance e…

AdaTok introduces an innovative object-level token merging strategy for Adaptive Token compression, aimed at enhancing the efficiency of Multimodal Large Language Models (MLLMs). Traditional patch-level tokenization has resulted in excessive computational and memory demands, leading to misalignments with human cognitive processes. The proposed method significantly reduces token usage to 10% while maintaining nearly 96% of the original model's performance, addressing critical challenges in multimodal understanding and reasoning.

A new study introduces a scene graph-guided generative AI framework aimed at synthesizing realistic images of industrial hazard scenarios. This framework addresses the challenge of acquiring datasets for workplace hazards, which are difficult to capture in real-time. By analyzing historical Occupational Safety and Health Administration (OSHA) accident reports with GPT-4o, the study extracts structured hazard reasoning and creates object-level scene graphs. These graphs are utilized to guide a text-to-image diffusion model, generating accurate hazard scenes for evaluation.

This study evaluates the effectiveness of various large language models (LLMs) in restoring diacritics in Romanian texts, a crucial task for text processing in languages with rich diacritical marks. The models tested include OpenAI's GPT-3.5, GPT-4, Google's Gemini 1.0 Pro, and Meta's Llama family, among others. Results indicate that GPT-4o achieves high accuracy in diacritic restoration, outperforming a neutral baseline, while other models show variability. The findings emphasize the importance of model architecture, training data, and prompt design in enhancing natural language processing to…