arXiv:2604.14585v2 Announce Type: replace-cross 
Abstract: Prompt optimization in compound AI systems is statistically indistinguishable from a coin flip: across 72 optimization runs on Claude Haiku 4.5 (6 methods $\times$ 4 tasks $\times$ 3 repeats), 49% score below zero-shot; on Amazon Nova Lite, the failure rate is even higher. Yet on one task, all six methods improve over zero-shot by up to $+6.8$ points. What distinguishes success from failure? We investigate with 18,000 grid evaluations and 144 optimization runs, testing two assumptions behind end-to-end optimization tools like TextGrad and DSPy, in the order they must be answered: (A) agent prompts interact, requiring joint rather than independent optimization, and (B) individual prompts are worth optimizing at all. Interaction effects are never significant ($p > 0.52$, all $F < 1.0$), and optimization helps only when the task has exploitable output structure: a format the model can produce but does not default to. We further give a mechanistic account: instruction-tuning compresses input phrasing into a narrow output distribution, eliminating the very phrasing-sensitivity that joint optimization assumes. We provide a two-stage diagnostic: an \$80 ANOVA pre-test for agent coupling, and a 10-minute headroom test that predicts whether optimization is worthwhile, turning a coin flip into an informed decision.

تشير الأبحاث الأخيرة إلى أن تحسين المطالبات في أنظمة الذكاء الاصطناعي المركبة، وخاصة على نماذج مثل Claude Haiku 4.5 وAmazon Nova Lite، يؤدي إلى نتائج غير متسقة، مع معدل فشل يقارب 50% عبر طرق تحسين مختلفة. أظهرت مهمة واحدة فقط تحسنًا كبيرًا، مما يشير إلى أن التحسين ليس فعالًا بشكل عالمي.

Investigaciones recientes indican que la optimización de prompts en sistemas de IA compuestos, específicamente en modelos como Claude Haiku 4.5 y Amazon Nova Lite, produce resultados inconsistentes, con una tasa de fracaso cercana al 50% en varios métodos de optimización. Solo una tarea mostró una mejora significativa, sugiriendo que la optimización no es universalmente efectiva.

Des recherches récentes montrent que l'optimisation des invites dans les systèmes d'IA composés, notamment sur des modèles comme Claude Haiku 4.5 et Amazon Nova Lite, donne des résultats incohérents, avec un taux d'échec proche de 50 % sur diverses méthodes d'optimisation. Une seule tâche a montré une amélioration significative, suggérant que l'optimisation n'est pas universellement efficace.

Recent research indicates that prompt optimization in compound AI systems, specifically on models like Claude Haiku 4.5 and Amazon Nova Lite, yields inconsistent results, with a near 50% failure rate across various optimization methods. Only one task showed significant improvement, suggesting that optimization is not universally effective.