arXiv:2606.12780v1 Announce Type: cross 
Abstract: Self-evolving agents are expected to improve through interaction without external supervision, but this remains difficult in partially observable environments where agents must explore actively, learn from limited feedback, and decide when to trust prior experience. Existing LLM-agent methods often rely on memory or planning modules, yet they rarely close the loop between them to continually refine an internal understanding of environment dynamics. We introduce ProPlay, a procedural world model that supports procedure-level preplay, where agents can rehearse future procedural paths using the learned world knowledge. Rather than representing experience as isolated rules or low-level action constraints, ProPlay abstracts successful trajectories into procedures and organizes them in a procedure graph that captures causal transitions among task stages. Each transition is associated with a reliability record embedding to estimate its task-specific contribution from past outcomes. Before each episode, ProPlay simulates future procedural trajectories over known graph structures as structured soft guidance; after execution, it refines the graph using environment feedback. Experiments on public benchmarks show that ProPlay consistently improves environment understanding and self-evolution capability over strong baselines. Our code has been released in https://github.com/antman9914/proplay.

تم تقديم ProPlay كنموذج عالمي إجرائي مصمم لوكلاء نماذج اللغة الكبيرة (LLM) القابلة للتطور الذاتي، مما يمكّنهم من تكرار مسارات إجرائية مستقبلية بناءً على المعرفة المكتسبة. تعالج هذه الابتكار التحديات المتعلقة بالاستكشاف النشط والتعلم في البيئات القابلة للملاحظة جزئيًا، مما يسمح للوكلاء بتحسين فهمهم للديناميات البيئية.

ProPlay se ha presentado como un modelo de mundo procedural diseñado para agentes de modelos de lenguaje de gran tamaño (LLM) en auto-evolución, permitiéndoles ensayar futuros caminos procedimentales basados en el conocimiento adquirido. Esta innovación aborda los desafíos de la exploración activa y el aprendizaje en entornos parcialmente observables, permitiendo a los agentes refinar su comprensión de los entornos dinámicos.

ProPlay a été introduit comme un modèle de monde procédural conçu pour des agents de modèles de langage de grande taille (LLM) en auto-évolution, leur permettant de répéter des chemins procéduraux futurs basés sur des connaissances acquises. Cette innovation répond aux défis de l'exploration active et de l'apprentissage dans des environnements partiellement observables, permettant aux agents de perfectionner leur compréhension des environnements dynamiques.

ProPlay has been introduced as a procedural world model designed for self-evolving large language model (LLM) agents, enabling them to rehearse future procedural paths based on learned knowledge. This innovation addresses the challenges of active exploration and learning in partially observable environments, allowing agents to refine their understanding of dynamic environments.