Representational Stability of Truth in Large Language Models

Cornell Tech has secured a $7 million investment from NASA and Schmidt Sciences aimed at modernizing arXiv, a preprint repository for scientific papers. This funding will facilitate the migration of arXiv to cloud infrastructure, upgrade its outdated codebase, and develop new tools to enhance the discovery of relevant preprints for researchers.

Recent evaluations of large language models (LLMs) have highlighted their vulnerability to flawed premises, which can lead to inefficient reasoning and unreliable outputs. The introduction of the Premise Critique Bench (PCBench) aims to assess the Premise Critique Ability of LLMs, focusing on their capacity to identify and articulate errors in input premises across various difficulty levels.

A recent position paper emphasizes the need for literary scholars to engage with research on large language model (LLM) interpretability, suggesting that the red team could serve as a platform for this ideological struggle. The paper argues that current interpretability standards are insufficient for evaluating LLMs.

Recent advancements in text-to-speech (TTS) technology have led to the development of a new model called Word-level TTS Alignment by ASR-driven Attentive Reward (W3AR), which utilizes fine-grained reward signals from automatic speech recognition (ASR) systems to enhance TTS synthesis. This model addresses the limitations of traditional evaluation methods that often overlook specific problematic words in utterances.

A recent study has introduced a novel method for evaluating large language models (LLMs) in zero-shot classification tasks by transforming binary classifications into pairwise comparisons. This approach utilizes the Elo rating system to rank instances, thereby enhancing classification performance and providing more informative results than traditional methods.

A new framework named Specification-Guided Moderation (SGM) has been introduced to enhance content moderation filters for large language models (LLMs). This framework allows for the automation of training data generation based on user-defined specifications, addressing the limitations of traditional safety-focused filters. SGM aims to provide scalable and application-specific alignment goals for LLMs.

A new framework named Reading Comprehension Exercise Generation (RCEG) has been proposed to leverage large language models (LLMs) for automatically generating personalized English reading comprehension exercises. This framework utilizes fine-tuned LLMs to create content candidates, which are then evaluated by a discriminator to select the highest quality output, significantly enhancing the educational content generation process.

Cross-lingual information retrieval (CLIR) is being systematically evaluated through various approaches, including document translation and multilingual dense retrieval with pretrained encoders. This research highlights the challenges posed by disparities in resources and weak semantic alignment in embedding models, revealing that dense retrieval models specifically trained for CLIR outperform traditional methods.