arXiv:2506.00727v2 Announce Type: replace 
Abstract: Background and Objective: Plane reformatting for four-dimensional phase contrast MRI (4D flow MRI) is time-consuming and prone to inter-observer variability, which limits fast cardiovascular flow assessment. Deep reinforcement learning (DRL) trains agents to iteratively adjust plane position and orientation, enabling accurate plane reformatting without the need for detailed landmarks, making it suitable for images with limited contrast and resolution such as 4D flow MRI. However, current DRL methods assume that test volumes share the same spatial alignment as the training data, limiting generalization across scanners and institutions. To address this limitation, we introduce AdaPR (Adaptive Plane Reformatting), a DRL framework that uses a local coordinate system to navigate volumes with arbitrary positions and orientations.
  Methods: We implemented AdaPR using the Asynchronous Advantage Actor-Critic (A3C) algorithm and validated it on 88 4D flow MRI datasets acquired from multiple vendors, including patients with congenital heart disease.
  Results: AdaPR achieved a mean angular error of 6.32 +/- 4.15 degrees and a distance error of 3.40 +/- 2.75 mm, outperforming global-coordinate DRL methods and alternative non-DRL methods. AdaPR maintained consistent accuracy under different volume orientations and positions. Flow measurements from AdaPR planes showed no significant differences compared to two manual observers, with excellent correlation (R^2 = 0.972 and R^2 = 0.968), comparable to inter-observer agreement (R^2 = 0.969).
  Conclusion: AdaPR provides robust, orientation-independent plane reformatting for 4D flow MRI, achieving flow quantification comparable to expert observers. Its adaptability across datasets and scanners makes it a promising candidate for medical imaging applications beyond 4D flow MRI.

تم تطوير إطار عمل جديد يسمى AdaPR (إعادة تشكيل الطائرة التكيفية) باستخدام التعلم العميق المعزز (DRL) لتحسين عملية إعادة تشكيل الطائرات لتصوير الرنين المغناطيسي بتباين الطور الرباعي الأبعاد (IRM 4D flow). تهدف هذه الطريقة إلى تقليل الوقت والتباين المرتبطين بالتقنيات التقليدية، مما يسمح بتقييمات أكثر دقة لتدفق القلب والأوعية الدموية دون الاعتماد على معالم مفصلة.

Se ha desarrollado un nuevo marco llamado AdaPR (Reformateo de Plano Adaptativo) utilizando Aprendizaje por Refuerzo Profundo (DRL) para mejorar el proceso de reformateo de planos para la IRM de contraste de fase en cuatro dimensiones (IRM 4D flow). Este método busca reducir el tiempo y la variabilidad asociados con las técnicas tradicionales, permitiendo evaluaciones más precisas del flujo cardiovascular sin depender de puntos de referencia detallados.

Un nouveau cadre appelé AdaPR (Reformatage de Plan Adaptatif) a été développé en utilisant l'apprentissage par renforcement profond (DRL) pour améliorer le processus de reformatage de plan pour l'IRM de contraste de phase en quatre dimensions (IRM 4D flow). Cette méthode vise à réduire le temps et la variabilité associés aux techniques traditionnelles, permettant des évaluations plus précises du flux cardiovasculaire sans dépendre de repères détaillés.

A new framework called AdaPR (Adaptive Plane Reformatting) has been developed using Deep Reinforcement Learning (DRL) to enhance the process of plane reformatting for four-dimensional phase contrast MRI (4D flow MRI). This method aims to reduce the time and variability associated with traditional techniques, allowing for more accurate cardiovascular flow assessments without relying on detailed landmarks.

Adaptive Plane Reformatting for 4D Flow MRI using Deep Reinforcement Learning

arXiv:2512.03805v1 Announce Type: new 
Abstract: Dynamic Algorithm Configuration (DAC) studies the efficient identification of control policies for parameterized optimization algorithms. Numerous studies have leveraged the robustness of decision-making in Reinforcement Learning (RL) to address the optimization challenges in algorithm configuration. However, applying RL to DAC is challenging and often requires extensive domain expertise. We conduct a comprehensive study of deep-RL algorithms in DAC through a systematic analysis of controlling the population size parameter of the (1+($\lambda$,$\lambda$))-GA on OneMax instances. Our investigation of DDQN and PPO reveals two fundamental challenges that limit their effectiveness in DAC: scalability degradation and learning instability. We trace these issues to two primary causes: under-exploration and planning horizon coverage, each of which can be effectively addressed through targeted solutions. To address under-exploration, we introduce an adaptive reward shifting mechanism that leverages reward distribution statistics to enhance DDQN agent exploration, eliminating the need for instance-specific hyperparameter tuning and ensuring consistent effectiveness across different problem scales. In dealing with the planning horizon coverage problem, we demonstrate that undiscounted learning effectively resolves it in DDQN, while PPO faces fundamental variance issues that necessitate alternative algorithmic designs. We further analyze the hyperparameter dependencies of PPO, showing that while hyperparameter optimization enhances learning stability, it consistently falls short in identifying effective policies across various configurations. Finally, we demonstrate that DDQN equipped with our adaptive reward shifting strategy achieves performance comparable to theoretically derived policies with vastly improved sample efficiency, outperforming prior DAC approaches by several orders of magnitude.

تم إجراء دراسة شاملة حول تطبيق خوارزميات التعلم العميق المعزز (RL) لتكوين الخوارزميات الديناميكية (DAC)، مع التركيز بشكل خاص على تحسين معلمة حجم السكان لخوارزمية (1+($\lambda$,$\lambda$))-GA على حالات OneMax. تحدد الدراسة تحديات كبيرة مثل تدهور القابلية للتوسع وعدم استقرار التعلم، المنسوبين إلى نقص الاستكشاف وتغطية أفق التخطيط.

Se ha llevado a cabo un estudio exhaustivo sobre la aplicación de algoritmos de aprendizaje por refuerzo profundo (RL) para la configuración dinámica de algoritmos (DAC), centrándose específicamente en la optimización del parámetro de tamaño de población del (1+($\lambda$,$\lambda$))-GA en instancias de OneMax. La investigación identifica desafíos significativos como la degradación de la escalabilidad y la inestabilidad del aprendizaje, atribuidos a la subexploración y a la cobertura del horizonte de planificación.

Une étude approfondie a été réalisée sur l'application des algorithmes d'apprentissage par renforcement profond (RL) pour la configuration dynamique des algorithmes (DAC), en se concentrant spécifiquement sur l'optimisation du paramètre de taille de population du (1+($\lambda$,$\lambda$))-GA sur des instances OneMax. La recherche identifie des défis significatifs tels que la dégradation de l'évolutivité et l'instabilité de l'apprentissage, attribués à une sous-exploration et à une couverture de l'horizon de planification.

A comprehensive study has been conducted on the application of deep reinforcement learning (RL) algorithms for dynamic algorithm configuration (DAC), specifically focusing on optimizing the population size parameter of the (1+($\lambda$,$\lambda$))-GA on OneMax instances. The research identifies significant challenges such as scalability degradation and learning instability, attributed to under-exploration and planning horizon coverage.

Deep Reinforcement Learning for Dynamic Algorithm Configuration: A Case Study on Optimizing OneMax with the (1+($\lambda$,$\lambda$))-GA

arXiv:2512.03891v1 Announce Type: cross 
Abstract: Active suspension systems are critical for enhancing vehicle comfort, safety, and stability, yet their performance is often limited by fixed hardware designs and control strategies that cannot adapt to uncertain and dynamic operating conditions. Recent advances in digital twins (DTs) and deep reinforcement learning (DRL) offer new opportunities for real-time, data-driven optimization across a vehicle's lifecycle. However, integrating these technologies into a unified framework remains an open challenge. This work presents a DT-based control co-design (CCD) framework for full-vehicle active suspensions using multi-generation design concepts. By integrating automatic differentiation into DRL, we jointly optimize physical suspension components and control policies under varying driver behaviors and environmental uncertainties. DRL also addresses the challenge of partial observability, where only limited states can be sensed and fed back to the controller, by learning optimal control actions directly from available sensor information. The framework incorporates model updating with quantile learning to capture data uncertainty, enabling real-time decision-making and adaptive learning from digital-physical interactions. The approach demonstrates personalized optimization of suspension systems under two distinct driving settings (mild and aggressive). Results show that the optimized systems achieve smoother trajectories and reduce control efforts by approximately 43% and 52% for mild and aggressive, respectively, while maintaining ride comfort and stability. Contributions include: developing a DT-enabled CCD framework integrating DRL and uncertainty-aware model updating for full-vehicle active suspensions, introducing a multi-generation design strategy for self-improving systems, and demonstrating personalized optimization of active suspension systems for distinct driver types.

تم تطوير إطار عمل جديد يستخدم تقنية التوأم الرقمي والتعلم العميق المعزز (DRL) لتحسين أنظمة التعليق النشطة للسيارات الكاملة. تتناول هذه الطريقة قيود أنظمة التعليق التقليدية من خلال تمكين التعديلات في الوقت الفعلي المستندة إلى البيانات، مما يعزز راحة السيارة وسلامتها واستقرارها في ظروف متنوعة.

Se ha desarrollado un nuevo marco que utiliza la tecnología Digital Twin y el Aprendizaje por Refuerzo Profundo (DRL) para optimizar las suspensiones activas de vehículos completos. Este enfoque aborda las limitaciones de los sistemas de suspensión tradicionales al permitir ajustes en tiempo real basados en datos, mejorando así el confort, la seguridad y la estabilidad del vehículo en diversas condiciones.

Un nouveau cadre utilisant la technologie Digital Twin et l'apprentissage par renforcement profond (DRL) a été développé pour optimiser les suspensions actives de véhicules complets. Cette approche répond aux limitations des systèmes de suspension traditionnels en permettant des ajustements en temps réel, basés sur des données, pour améliorer le confort, la sécurité et la stabilité du véhicule dans des conditions variées.

A new framework utilizing Digital Twin technology and Deep Reinforcement Learning (DRL) has been developed for optimizing full vehicle active suspensions. This approach addresses the limitations of traditional suspension systems by enabling real-time, data-driven adjustments to enhance vehicle comfort, safety, and stability under varying conditions.

Digital Twin-based Control Co-Design of Full Vehicle Active Suspensions via Deep Reinforcement Learning

arXiv:2505.08032v2 Announce Type: replace-cross 
Abstract: Adaptive beam switching is essential for mission-critical military and commercial 6G networks but faces major challenges from high carrier frequencies, user mobility, and frequent blockages. While existing machine learning (ML) solutions often focus on maximizing instantaneous throughput, this can lead to unstable policies with high signaling overhead. This paper presents an online Deep Reinforcement Learning (DRL) framework designed to learn an operationally stable policy. By equipping the DRL agent with an enhanced state representation that includes blockage history, and a stability-centric reward function, we enable it to prioritize long-term link quality over transient gains. Validated in a challenging 100-user scenario using the Sionna library, our agent achieves throughput comparable to a reactive Multi-Armed Bandit (MAB) baseline. Specifically, our proposed framework improves link stability by approximately 43% compared to a vanilla DRL approach, achieving operational reliability competitive with MAB while maintaining high data rates. This work demonstrates that by reframing the optimization goal towards operational stability, DRL can deliver efficient, reliable, and real-time beam management solutions for next-generation mission-critical networks.

تم تقديم إطار جديد للتعلم العميق المعزز (DRL) عبر الإنترنت لتحسين تبديل الشعاع التكيفي في شبكات 6G، حيث يتناول التحديات مثل الترددات الحاملة العالية وتحركات المستخدمين. يركز هذا الإطار على جودة الرابط على المدى الطويل بدلاً من المكاسب المؤقتة، محققًا تحسينًا بنسبة 43% في استقرار الرابط مقارنة بالطرق التقليدية.

Se ha introducido un nuevo marco de Aprendizaje por Refuerzo Profundo (DRL) en línea para mejorar el cambio adaptativo de haz en redes 6G, abordando desafíos como las altas frecuencias portadoras y la movilidad de los usuarios. Este marco prioriza la calidad del enlace a largo plazo sobre las ganancias a corto plazo, logrando una mejora del 43% en la estabilidad del enlace en comparación con los métodos tradicionales.

Un nouveau cadre d'apprentissage par renforcement profond (DRL) en ligne a été introduit pour améliorer le commutateur de faisceau adaptatif dans les réseaux 6G, abordant des défis tels que les hautes fréquences porteuses et la mobilité des utilisateurs. Ce cadre privilégie la qualité de lien à long terme par rapport aux gains à court terme, réalisant une amélioration de 43 % de la stabilité du lien par rapport aux méthodes traditionnelles.

A new online Deep Reinforcement Learning (DRL) framework has been introduced to enhance adaptive beam switching in 6G networks, addressing challenges such as high carrier frequencies and user mobility. This framework prioritizes long-term link quality over short-term gains, achieving a 43% improvement in link stability compared to traditional methods.

Adaptive Plane Reformatting for 4D Flow MRI using Deep Reinforcement Learning

Was this article worth reading? Share it

LucidQuery AI

Fluig AI

Attentive AI