MemeArena: Automating Context-Aware Unbiased Evaluation of Harmfulness Understanding for Multimodal Large Language Models

MemeArena is a groundbreaking new tool designed to enhance the evaluation of multimodal large language models (mLLMs) in understanding harmful content on social media. As memes proliferate online, it's crucial for these models to accurately assess the nuanced nature of harmfulness in various contexts. Traditional evaluation methods often fall short, focusing solely on binary classifications. By introducing an agent-based arena-style evaluation, MemeArena aims to provide a more comprehensive understanding of harmfulness, which is essential for improving AI's interaction with diverse media.