arXiv:2605.29631v1 Announce Type: new 
Abstract: Randomized controlled trials are a cornerstone of medicine and the social sciences as they enable reliable estimates of causal effects. However, they are costly and time-consuming to conduct, motivating interest in predicting causal effects from existing experimental evidence. Recent advances in large language models (LLMs) have demonstrated strong performance on knowledge-intensive tasks, raising the question of whether these models can be used for forecasting causal effect sizes. To investigate this, we introduce Query2Effect, a new large-scale benchmark consisting of more than 72,000 natural language questions aligned with experiment descriptions, created to simulate realistic information-seeking scenarios by varying query specificity along dimensions of implicitness, abstraction, and ambiguity. We then propose a two-step framework that first generates a synthetic structured representation of a query before predicting effect size using a supervised encoder model. Experiments show that finetuning plays a crucial role in improving prediction performance, with absolute error reducing by -27% up to -71% compared to prompted out-of-the-box LLMs, and that our two-step framework is beneficial for out-of-domain generalization, highlighting the benefits of separating semantic interpretation from numerical effect estimation.

تم تقديم معيار جديد يسمى Query2Effect، يتكون من أكثر من 72,000 سؤال بلغة طبيعية متماشية مع أوصاف تجريبية، بهدف التنبؤ بالتأثيرات السببية من الأدلة التجريبية الموجودة. تسعى هذه المبادرة للاستفادة من التقدم في نماذج اللغة الكبيرة (LLMs) للتنبؤ بأحجام التأثيرات السببية، مما يعالج قيود التجارب العشوائية المضبوطة التقليدية.

Se ha introducido un nuevo benchmark llamado Query2Effect, que consiste en más de 72,000 preguntas en lenguaje natural alineadas con descripciones experimentales, con el objetivo de predecir efectos causales a partir de evidencia experimental existente. Esta iniciativa busca aprovechar los avances en modelos de lenguaje grandes (LLMs) para pronosticar tamaños de efectos causales, abordando las limitaciones de los ensayos controlados aleatorios tradicionales.

Un nouveau benchmark appelé Query2Effect a été introduit, comprenant plus de 72 000 questions en langage naturel alignées avec des descriptions expérimentales, visant à prédire les effets causaux à partir de preuves expérimentales existantes. Cette initiative cherche à tirer parti des avancées des grands modèles de langage (LLMs) pour prévoir les tailles des effets causaux, répondant ainsi aux limites des essais contrôlés randomisés traditionnels.

A new benchmark called Query2Effect has been introduced, consisting of over 72,000 natural language questions aligned with experimental descriptions, aimed at predicting causal effects from existing experimental evidence. This initiative seeks to leverage advancements in large language models (LLMs) to forecast causal effect sizes, addressing the limitations of traditional randomized controlled trials.