T2T-VICL: Unlocking the Boundaries of Cross-Task Visual In-Context Learning via Implicit Text-Driven VLMs

The paper discusses the challenges associated with Vision-Language Models (VLMs) in generating lengthy outputs with low information density, which leads to increased energy consumption and costs. It introduces a novel verbose-text induction attack (VTIA) that uses adversarial perturbations to optimize output token length, addressing the limitations of existing methods that merely delay the end of output without maximizing length.

The article presents an interpretability-guided framework for generating synthetic data in emotional text analysis, addressing the challenges of high costs and restrictions in accessing training data. Utilizing Shapley Additive Explanations (SHAP), the framework enhances the performance of large language models (LLMs) in emotion classification, particularly for underrepresented classes. However, it notes limitations in vocabulary richness and expression complexity in synthetic texts compared to authentic social media posts.

QueryGym is a new Python toolkit designed for large language model (LLM)-based query reformulation. It aims to provide a unified framework that enhances retrieval effectiveness by allowing consistent implementation, execution, and comparison of various LLM-based methods. The toolkit includes a Python API, a retrieval-agnostic interface for integration with backends like Pyserini and PyTerrier, and a centralized prompt management system.

HiViS (Hiding Visual Tokens from the Drafter for Speculative Decoding in Vision-Language Models) is a proposed framework aimed at enhancing the efficiency of Vision-Language Models (VLMs). It addresses the computational challenges posed by visual tokens by allowing the drafter to obtain visual information without explicitly processing these tokens. This approach ensures that the drafter's prefill sequence length aligns with that of the textual tokens, potentially improving inference speed and quality.

Recent advancements in language models have led to the development of Large Reasoning Models (LRMs) that articulate detailed thought processes before arriving at conclusions. Despite their enhanced performance on reasoning benchmarks, the fundamental capabilities and limitations of these models are not yet fully understood. Evaluations have primarily focused on final answer accuracy, often overlooking the reasoning processes involved.

The paper introduces FlipVQA-Miner, an automated pipeline designed to extract high-quality question-answer (QA) and visual question-answer (VQA) pairs from educational documents. This method combines layout-aware OCR with large language model (LLM)-based semantic parsing, addressing the challenge of transforming raw PDFs into AI-ready supervision. Experiments demonstrate that the approach yields accurate and aligned QA/VQA pairs, enhancing the utility of educational materials for training LLMs.

The paper presents Rationale-Bootstrapped Fine-Tuning (RB-FT) for enhancing video classification using Vision Language Models (VLMs). It addresses the challenge of limited domain-specific data by proposing a two-stage self-improvement approach. Initially, VLMs generate textual rationales for videos, which are then used for fine-tuning, followed by conventional supervised fine-tuning on task labels, resulting in improved classification effectiveness.

VLA-Pruner is a proposed method aimed at enhancing the efficiency of Vision-Language-Action (VLA) models by implementing temporal-aware dual-level visual token pruning. This approach addresses the high computational costs associated with processing continuous visual streams, which limits real-time deployment. By focusing on both high-level semantic understanding and low-level action execution, VLA-Pruner seeks to improve the performance of VLA models significantly.