VADER: Towards Causal Video Anomaly Understanding with Relation-Aware Large Language Models

A new study introduces a method for long video summarization through key moment extraction, utilizing Vision-Language Models (VLMs) to identify and select the most relevant clips from lengthy video content. This approach aims to enhance the efficiency of video analysis by generating compact visual descriptions and leveraging large language models (LLMs) for summarization. The evaluation is based on reference clips derived from the MovieSum dataset.

A new training framework for retrieval-augmented generation (RAG) models has been introduced, utilizing the Merlin-Arthur protocol to enhance the interaction between retrievers and large language models (LLMs). This approach aims to reduce hallucinations by ensuring that LLMs only provide answers supported by reliable evidence while rejecting insufficient or misleading context.

A new framework called Causal Judge Evaluation (CJE) has been introduced to address the statistical shortcomings of using large language models (LLMs) as judges in model assessments. CJE achieves a 99% pairwise ranking accuracy on 4,961 prompts from Chatbot Arena while significantly reducing costs by utilizing a calibrated judge with only 5% of oracle labels.