PHANTOM: A Large-Scale Dataset of Multimodal Adversarial Attacks for Vision-Language Models

A new dataset named PHANTOM has been introduced, providing a large-scale collection of pre-generated adversarial attacks specifically designed for vision-language models (VLMs). This dataset includes 47,524 adversarial samples across 10 high-level categories and 55 subcategories of harmful intents, aiming to enhance the accessibility of adversarial data for researchers.

The significance of this dataset lies in its potential to improve the robustness and alignment of VLMs, addressing the computational challenges associated with generating adversarial examples. By consolidating existing benchmarks and introducing new categories, it serves as a valuable resource for evaluating model performance.

This development reflects ongoing efforts in the AI community to address vulnerabilities in VLMs, particularly as concerns about their limitations and safety persist. The introduction of benchmarks like PHANTOM aligns with broader discussions on the need for comprehensive evaluation frameworks, as researchers seek to enhance model reliability and mitigate risks associated with adversarial attacks.