VOST-SGG: VLM-Aided One-Stage Spatio-Temporal Scene Graph Generation

The Large Shapes and Textures dataset (LAS&T) has been introduced to enhance the capabilities of Large Vision-Language Models (LVLMs) in recognizing and representing shapes and textures across various contexts. This dataset, created through unsupervised extraction from natural images, serves as a benchmark for evaluating the performance of leading models like CLIP and DINO in shape recognition tasks.

A recent study introduces Function-word De-Attention (FDA) as a method to enhance the robustness of Vision-Language Models (VLMs) against cross-modal adversarial attacks by reducing the influence of function words. The FDA technique differentiates between original and function-word cross-attention, leading to improved alignment and robustness in VLMs. Comprehensive experiments demonstrate significant reductions in attack success rates with minimal performance drops across various models and tasks.

OpenSubject has been introduced as a large-scale video-derived dataset comprising 2.5 million samples and 4.35 million images, aimed at improving subject-driven image generation and manipulation. This dataset employs a four-stage pipeline that utilizes cross-frame identity priors to enhance the accuracy of generated images in complex scenes with multiple subjects.

A new benchmark called Know-Show has been introduced to evaluate the spatio-temporal grounded reasoning capabilities of large Video-Language Models (Video-LMs). This benchmark consists of five scenarios that assess how well these models can reason about actions while grounding their inferences in visual and temporal evidence, highlighting significant gaps between current models and human reasoning.