Breaking Expert Knowledge Limits: Self-Pruning for Large Language Models

Large Language Models (LLMs) have shown exceptional ability in text generation but often produce factually incorrect statements, known as 'hallucinations'. This study investigates hallucinations in conversational data across three low-resource languages: Hindi, Farsi, and Mandarin. The analysis of various LLMs, including GPT-3.5 and GPT-4o, reveals that while Mandarin has few hallucinated responses, Hindi and Farsi exhibit significantly higher rates of inaccuracies.

LiveCLKTBench is an automated generation pipeline designed to evaluate cross-lingual knowledge transfer in large language models (LLMs). It isolates and measures knowledge transfer by identifying time-sensitive knowledge entities, filtering them based on temporal occurrence, and generating factual questions translated into multiple languages. The evaluation of several LLMs across five languages reveals that cross-lingual transfer is influenced by linguistic distance and is often asymmetric.

ProRAC (Progression-based Reasoning about Actions and Change) is a neuro-symbolic framework that utilizes large language models (LLMs) to address reasoning about actions and changes (RAC) problems. The framework extracts essential elements from RAC problems, executes actions progressively to determine the final state, and evaluates queries against this state. Evaluations on various RAC benchmarks indicate that ProRAC demonstrates strong performance across diverse tasks and domains.

HSKBenchmark introduces a novel benchmark for modeling and assessing Chinese second language acquisition (SLA) using large language models (LLMs). This benchmark addresses the challenges of traditional language acquisition experiments, which are often impractical and ethically complex. HSKBenchmark encompasses HSK levels 3 to 6, featuring authentic textbooks and a comprehensive evaluation system, thereby enhancing the interpretability and scalability of LLMs in SLA.

HalluClean is a new framework designed to detect and correct hallucinations in large language models (LLMs). This task-agnostic approach enhances the reliability of LLM-generated text by decomposing the process into planning, execution, and revision stages. HalluClean utilizes minimal task-routing prompts for zero-shot generalization across various domains, significantly improving factual consistency in outputs.

Instruction tuning is a crucial method for aligning large language models (LLMs) with human intentions and safety requirements. This survey outlines the entire process, including data collection methods, fine-tuning strategies, and evaluation protocols. It categorizes data construction into expert annotation, distillation from larger models, and self-improvement mechanisms, each with unique trade-offs. The study also addresses challenges in evaluating model performance across multilingual and multimodal contexts.

MedBench v4 introduces a comprehensive benchmarking framework for evaluating Chinese medical language models, multimodal models, and intelligent agents. This cloud-based infrastructure features over 700,000 expert-curated tasks across various medical specialties. The evaluation process includes multi-stage refinement and clinician reviews, with results indicating that while base LLMs score an average of 54.1/100, safety and ethics ratings remain low at 18.4/100.

The study presents a benchmark for detecting long-term periodic workflows in human activities, addressing a gap in existing research. It includes 580 multimodal activity sequences and supports tasks such as unsupervised workflow detection and procedural anomaly detection. The proposed lightweight model aims to enhance understanding of complex human behaviors over extended periods.