AutoSynth: Automated Workflow Optimization for High-Quality Synthetic Dataset Generation via Monte Carlo Tree Search

arXiv — cs.LG•Thursday, November 13, 2025 at 5:00:00 AM

AutoSynth represents a significant advancement in the field of synthetic data generation, particularly for supervised fine-tuning of large language models (LLMs). Traditional methods often struggle with the cold start problem, requiring labeled datasets that are costly and time-consuming to produce. By employing a Monte Carlo Tree Search approach, AutoSynth eliminates the need for reference datasets, utilizing a novel dataset-free hybrid reward system. This system allows for meta-learning through two LLM-as-judge components that evaluate sample quality and workflow effectiveness. Experimental results indicate that while expert-designed workflows maintain a high human preference rate of 96-99%, AutoSynth-generated data still achieves a notable performance of 40-51%. Furthermore, AutoSynth significantly reduces the time required for dataset generation from 5-7 hours to just 30 minutes, showcasing its potential to streamline processes in educational tasks and beyond.

— via World Pulse Now AI Editorial System

Read Original

Was this article worth reading? Share it

Recommended Readings

arXiv — cs.CL5 hours ago

Automatic Fact-checking in English and Telugu

NeutralArtificial Intelligence

The research paper explores the challenge of false information and the effectiveness of large language models (LLMs) in verifying factual claims in English and Telugu. It presents a bilingual dataset and evaluates various approaches for classifying the veracity of claims. The study aims to enhance the efficiency of fact-checking processes, which are often labor-intensive and time-consuming.

Read full article

via arXiv — cs.CL

arXiv — cs.LG5 hours ago

Beat the long tail: Distribution-Aware Speculative Decoding for RL Training

PositiveArtificial Intelligence

The paper titled 'Beat the long tail: Distribution-Aware Speculative Decoding for RL Training' introduces a new framework called DAS, aimed at improving the efficiency of reinforcement learning (RL) rollouts for large language models (LLMs). The study identifies a bottleneck in the rollout phase, where long trajectories consume significant time. DAS employs an adaptive drafter and a length-aware speculation policy to optimize the rollout process without changing model outputs, enhancing the overall training efficiency.

Read full article

via arXiv — cs.LG

arXiv — cs.LG5 hours ago

Failure to Mix: Large language models struggle to answer according to desired probability distributions

NegativeArtificial Intelligence

Recent research indicates that large language models (LLMs) struggle to generate outputs that align with specified probability distributions. Experiments revealed that when asked to produce binary outputs with a target probability, LLMs consistently failed to meet these expectations, often defaulting to the most probable answer. This behavior undermines the probabilistic exploration necessary for scientific idea generation and selection, raising concerns about the effectiveness of current AI training methodologies.

Read full article

via arXiv — cs.LG

arXiv — cs.CL5 hours ago

DataSage: Multi-agent Collaboration for Insight Discovery with External Knowledge Retrieval, Multi-role Debating, and Multi-path Reasoning

PositiveArtificial Intelligence

DataSage is a novel multi-agent framework designed to enhance insight discovery in data analytics. It addresses limitations of existing data insight agents by incorporating external knowledge retrieval, a multi-role debating mechanism, and multi-path reasoning. These features aim to improve the depth of analysis and the accuracy of insights generated, thereby assisting organizations in making informed decisions in a data-driven environment.

Read full article

via arXiv — cs.CL

arXiv — cs.LG5 hours ago

FlakyGuard: Automatically Fixing Flaky Tests at Industry Scale

PositiveArtificial Intelligence

Flaky tests, which unpredictably pass or fail, hinder developer productivity and delay software releases. FlakyGuard is introduced as a solution that leverages large language models (LLMs) to automatically repair these tests. Unlike previous methods like FlakyDoctor, FlakyGuard effectively addresses the context problem by structuring code as a graph and selectively exploring relevant contexts. Evaluation of FlakyGuard on real-world tests indicates a repair success rate of 47.6%, with 51.8% of fixes accepted by developers, marking a significant improvement over existing approaches.

Read full article

via arXiv — cs.LG

arXiv — cs.CV5 hours ago

GMAT: Grounded Multi-Agent Clinical Description Generation for Text Encoder in Vision-Language MIL for Whole Slide Image Classification

PositiveArtificial Intelligence

The article presents a new framework called GMAT, which enhances Multiple Instance Learning (MIL) for whole slide image (WSI) classification. By integrating vision-language models (VLMs), GMAT aims to improve the generation of clinical descriptions that are more expressive and medically specific. This addresses limitations in existing methods that rely on large language models (LLMs) for generating descriptions, which often lack domain grounding and detailed medical specificity, thus improving alignment with visual features.

Read full article

via arXiv — cs.CV

arXiv — stat.MLa day ago

Silenced Biases: The Dark Side LLMs Learned to Refuse

NegativeArtificial Intelligence

Safety-aligned large language models (LLMs) are increasingly used in sensitive applications where fairness is crucial. Evaluating their fairness is complex, often relying on standard question-answer methods that misinterpret refusal responses as indicators of fairness. This paper introduces the concept of silenced biases, which are unfair preferences hidden within the models' latent space, masked by safety-alignment. Previous methods have limitations, prompting the need for new approaches to uncover these biases effectively.

Read full article

via arXiv — stat.ML

arXiv — cs.LGa day ago

Fair In-Context Learning via Latent Concept Variables

PositiveArtificial Intelligence

The paper titled 'Fair In-Context Learning via Latent Concept Variables' explores the in-context learning (ICL) capabilities of large language models (LLMs) in handling tabular data. It highlights the potential for LLMs to inherit biases from pre-training data, which can lead to discrimination in high-stakes applications. The authors propose an optimal demonstration selection method using latent concept variables to enhance task adaptation and fairness, alongside data augmentation strategies to minimize correlations between sensitive variables and predictive outcomes.

Read full article

via arXiv — cs.LG