Critical or Compliant? The Double-Edged Sword of Reasoning in Chain-of-Thought Explanations

FinCriticalED (Financial Critical Error Detection) is introduced as a visual benchmark for evaluating OCR and vision language models specifically on financial documents at the fact level. This benchmark addresses the challenges posed by the visually dense layouts of financial documents, where minor OCR errors can lead to significant misinterpretations. It provides 500 image-HTML pairs with expert-annotated financial facts, marking a shift from traditional metrics to a focus on factual correctness.

This paper surveys techniques aimed at enhancing the predictive accuracy of pretrained large language models (LLMs) by allocating additional computational resources during inference. It categorizes test-time scaling methods based on the decomposition of problems into subproblems and their topological organization, including sequential, parallel, or tree-structured approaches. The study unifies various methodologies such as Chain-of-Thought and Tree-of-Thought, analyzing their strengths and weaknesses while suggesting future research directions.

The study explores the temporal predictors of outcomes in reasoning language models, specifically focusing on the chain-of-thought (CoT) paradigm. It reveals that large language models (LLMs) can predict their correctness after only a few reasoning tokens, even when longer outputs are needed for definitive answers. The findings highlight a drop in predictive accuracy for harder questions, suggesting that internal self-assessment of success emerges early in the reasoning process, impacting interpretability and inference-time control.

Synthetic datasets are increasingly utilized in clinical contexts to address challenges such as data privacy and governance. This paper provides an overview of the creation and evaluation of synthetic datasets specifically for clinical dialogues, which are difficult to collect due to their sensitive nature. The authors also discuss the theoretical implications for the application of these datasets in medical dialogue processing.

A comparative study on training private clinical language models for ICD-9 coding reveals that differential privacy (DP) methods can compromise diagnostic accuracy. The research systematically compares four training pipelines using identical models and privacy budgets. Results indicate that knowledge distillation from DP-trained teachers significantly outperforms other methods, recovering up to 63% of non-private performance while maintaining strong empirical privacy.

The paper discusses the exploitative pricing practices in data markets for large language models (LLMs), which often marginalize data providers. It proposes a fairshare pricing mechanism based on data valuation to enhance seller participation and improve data quality. The framework aims to align incentives between buyers and sellers, ensuring optimal outcomes for both parties while maintaining market sustainability.

Annotation bias in NLP datasets poses significant challenges for the development of multilingual Large Language Models (LLMs), especially in culturally diverse contexts. Factors such as task framing, annotator subjectivity, and cultural mismatches can lead to distorted model outputs and increased social harms. A comprehensive framework is proposed to understand annotation bias, which includes instruction bias, annotator bias, and contextual and cultural bias. The article reviews detection methods and suggests mitigation strategies.

Recent advancements in Natural Language Processing (NLP) have led to the widespread use of language models, prompting research into the reflection and amplification of social biases, including gender and racial bias. However, there is a notable gap in the analysis of how queer sexualities are represented in NLP systems. A survey of 55 articles reveals that sexuality is often poorly defined, relying on normative assumptions about sexual and romantic identities, which raises concerns about the operationalization of sexuality in NLP bias research.