arXiv:2606.02979v1 Announce Type: new 
Abstract: We present a novel compact deep multi-task learning model to handle various autonomous driving perception tasks in one forward pass. The model performs multiple views of semantic segmentation, depth estimation, light detection and ranging (LiDAR) segmentation, and bird's eye view projection simultaneously without being supported by other models. We also provide an adaptive loss weighting algorithm to tackle the imbalanced learning issue that occurred due to plenty of given tasks. Through data pre-processing and intermediate sensor fusion techniques, the model can process and combine multiple input modalities retrieved from RGB cameras, dynamic vision sensors (DVS), and LiDAR placed at several positions on the ego vehicle. Therefore, a better understanding of a dynamically changing environment can be achieved. Based on the ablation study, the model variant trained with our proposed method achieves a better performance. Furthermore, a comparative study is also conducted to clarify its performance and effectiveness against the combination of some recent models. As a result, our model maintains better performance even with much fewer parameters. Hence, the model can inference faster with less GPU memory utilization. Moreover, the result tends to be consistent in 3 different CARLA simulation datasets and 1 real-world nuScenes-lidarseg dataset. To support future research, we share codes and other files publicly at https://github.com/oskarnatan/compact-perception.

تم تقديم نموذج جديد مدمج للتعلم العميق متعدد المهام لتحسين مهام الإدراك في القيادة الذاتية، مما يسمح بالمعالجة المتزامنة للتجزئة الدلالية، وتقدير العمق، وتجزئة LiDAR، وإسقاط الرؤية العلوية دون الاعتماد على نماذج إضافية. يتم تحقيق هذا التقدم من خلال خوارزمية وزن خسارة تكيفية وتقنيات دمج متعددة المستشعرات باستخدام بيانات من كاميرات RGB، وأجهزة استشعار الرؤية الديناميكية، وLiDAR.

Se ha presentado un nuevo modelo compacto de aprendizaje profundo multitarea para mejorar las tareas de percepción en la conducción autónoma, permitiendo el procesamiento simultáneo de segmentación semántica, estimación de profundidad, segmentación LiDAR y proyección en vista aérea sin depender de modelos adicionales. Este avance se logra a través de un algoritmo de ponderación de pérdida adaptativa y técnicas de fusión de múltiples sensores utilizando datos de cámaras RGB, sensores de visión dinámica y LiDAR.

Un nouveau modèle d'apprentissage profond compact et multitâche a été introduit pour améliorer les tâches de perception de la conduite autonome, permettant le traitement simultané de la segmentation sémantique, de l'estimation de profondeur, de la segmentation LiDAR et de la projection en vue aérienne sans dépendance à des modèles supplémentaires. Cette avancée est réalisée grâce à un algorithme de pondération de perte adaptatif et à des techniques de fusion multi-capteurs utilisant des données provenant de caméras RGB, de capteurs de vision dynamique et de LiDAR.

A novel compact deep multi-task learning model has been introduced to enhance autonomous driving perception tasks, enabling simultaneous processing of semantic segmentation, depth estimation, LiDAR segmentation, and bird's eye view projection without reliance on additional models. This advancement is achieved through an adaptive loss weighting algorithm and multi-sensor fusion techniques utilizing data from RGB cameras, dynamic vision sensors, and LiDAR.