Intelligent Image Search Algorithms Fusing Visual Large Models

A novel vulnerability in vision-language models (VLMs) has been identified through the introduction of IAG, a method that enables multi-target backdoor attacks on VLM-based visual grounding systems. This technique utilizes dynamically generated, input-aware triggers that are text-guided, allowing for imperceptible manipulation of visual inputs while maintaining normal performance on benign samples.

The paper introduces TRANSPORTER, a model-independent approach designed to enhance video generation by transferring visual semantics from Vision Language Models (VLMs). This method addresses the challenge of understanding how VLMs derive their predictions, particularly in complex scenes with various objects and actions. TRANSPORTER generates videos that reflect changes in captions across diverse attributes and contexts.

A new generative framework has been proposed for enhancing low-light images and reducing blur, utilizing visual autoregressive modeling guided by perceptual priors from vision-language models. This approach addresses significant challenges in restoring dark images, which often suffer from low visibility, contrast, noise, and blur.

The recent paper titled 'Peregrine: One-Shot Fine-Tuning for FHE Inference of General Deep CNNs' addresses key challenges in adapting deep convolutional neural networks (CNNs) for fully homomorphic encryption (FHE) inference. It introduces a single-stage fine-tuning strategy and a generalized interleaved packing scheme to enhance the performance of CNNs while maintaining accuracy and supporting high-resolution image processing.

The introduction of DiffSeg30k marks a significant advancement in the detection of AI-generated content (AIGC) by providing a dataset of 30,000 diffusion-edited images with pixel-level annotations. This dataset allows for fine-grained detection of localized edits, addressing a gap in existing benchmarks that typically assess entire images without considering localized modifications.

A new framework has been introduced for automatic fashion captioning and hashtag generation, utilizing a retrieval-augmented approach that integrates multi-garment detection, attribute reasoning, and Large Language Model (LLM) prompting. This system aims to produce visually grounded and stylistically engaging text for fashion images, addressing the shortcomings of traditional end-to-end captioners in attribute fidelity and domain generalization.

A comprehensive multimodal room-monitoring system has been developed, integrating synchronized video and audio processing for real-time activity recognition and anomaly detection. The system has undergone two iterations, with the advanced version featuring multi-model audio ensembles and hybrid object detection methods, significantly enhancing its accuracy and robustness.