arXiv:2512.10925v1 Announce Type: new 
Abstract: Autonomous navigation in underwater environments remains a major challenge due to the absence of GPS, degraded visibility, and the presence of submerged obstacles. This article investigates these issues through the case of the BlueROV2, an open platform widely used for scientific experimentation. We propose a deep reinforcement learning approach based on the Proximal Policy Optimization (PPO) algorithm, using an observation space that combines target-oriented navigation information, a virtual occupancy grid, and ray-casting along the boundaries of the operational area. The learned policy is compared against a reference deterministic kinematic planner, the Dynamic Window Approach (DWA), commonly employed as a robust baseline for obstacle avoidance. The evaluation is conducted in a realistic simulation environment and complemented by validation on a physical BlueROV2 supervised by a 3D digital twin of the test site, helping to reduce risks associated with real-world experimentation. The results show that the PPO policy consistently outperforms DWA in highly cluttered environments, notably thanks to better local adaptation and reduced collisions. Finally, the experiments demonstrate the transferability of the learned behavior from simulation to the real world, confirming the relevance of deep RL for autonomous navigation in underwater robotics.

تم اقتراح إطار عمل جديد يستخدم نهج التعلم المعزز الخاضع للإشراف بواسطة التوأم الرقمي للملاحة الذاتية تحت الماء، حيث يتناول التحديات التي تواجه منصة BlueROV2 في البيئات التي تفتقر إلى نظام تحديد المواقع العالمي. يستخدم الإطار خوارزمية تحسين السياسة القريبة (PPO) لتعزيز قدرات الملاحة، مع دمج مصادر بيانات متنوعة لتحسين تجنب العقبات.

Se ha propuesto un nuevo marco que utiliza un enfoque de Aprendizaje por Refuerzo Supervisado por Gemelo Digital para la navegación autónoma submarina, abordando específicamente los desafíos que enfrenta la plataforma BlueROV2 en entornos sin GPS. El marco emplea el algoritmo Proximal Policy Optimization (PPO) para mejorar las capacidades de navegación, integrando diversas fuentes de datos para una mejor evasión de obstáculos.

Un nouveau cadre utilisant une approche d'apprentissage par renforcement supervisé par jumeau numérique a été proposé pour la navigation autonome sous-marine, abordant spécifiquement les défis rencontrés par la plateforme BlueROV2 dans des environnements sans GPS. Le cadre utilise l'algorithme Proximal Policy Optimization (PPO) pour améliorer les capacités de navigation, intégrant diverses sources de données pour une meilleure évitement des obstacles.

A new framework utilizing a Digital Twin Supervised Reinforcement Learning approach has been proposed for autonomous underwater navigation, specifically addressing challenges faced by the BlueROV2 platform in GPS-denied environments. The framework employs the Proximal Policy Optimization (PPO) algorithm to enhance navigation capabilities, integrating various data sources for improved obstacle avoidance.

Digital Twin Supervised Reinforcement Learning Framework for Autonomous Underwater Navigation

One More Thing in AI – Your Shortcut to AI Mastery

Digital Twin Supervised Reinforcement Learning Framework for Autonomous Underwater Navigation

Was this article worth reading? Share it

One More Thing in AI

LucidQuery AI

Attentive AI

LangWatch

Dyad

LCW

Ready to build your own newsroom?