arXiv:2511.12932v2 Announce Type: replace 
Abstract: With the rapid advancement of intelligent transportation systems, text-driven image generation and editing techniques have demonstrated significant potential in providing rich, controllable visual scene data for applications such as traffic monitoring and autonomous driving. However, several challenges remain, including insufficient semantic richness of generated traffic elements, limited camera viewpoints, low visual fidelity of synthesized images, and poor alignment between textual descriptions and generated content. To address these issues, we propose a unified text-driven framework for both image generation and editing, leveraging a controllable mask mechanism to seamlessly integrate the two tasks. Furthermore, we incorporate both vehicle-side and roadside multi-view data to enhance the geometric diversity of traffic scenes. Our training strategy follows a two-stage paradigm: first, we perform conceptual learning using large-scale coarse-grained text-image data; then, we fine-tune with fine-grained descriptive data to enhance text-image alignment and detail quality. Additionally, we introduce a mask-region-weighted loss that dynamically emphasizes small yet critical regions during training, thereby substantially enhancing the generation fidelity of small-scale traffic elements. Extensive experiments demonstrate that our method achieves leading performance in text-based image generation and editing within traffic scenes.

تم تقديم طريقة جديدة تُدعى Text2Traffic لتوليد وتحرير صور لمشاهد المرور، مما يعالج التحديات في أنظمة النقل الذكية. يُحسن هذا الإطار الموحد من الغنى الدلالي والوضوح البصري للصور المُنتَجة، وهو أمر حاسم لتطبيقات مثل مراقبة المرور والقيادة الذاتية.

Se ha introducido un nuevo método llamado Text2Traffic para generar y editar imágenes de escenas de tráfico, abordando los desafíos en los sistemas de transporte inteligente. Este marco unificado mejora la riqueza semántica y la fidelidad visual de las imágenes generadas, lo cual es crucial para aplicaciones como la monitorización del tráfico y la conducción autónoma.

Une nouvelle méthode appelée Text2Traffic a été introduite pour générer et éditer des images de scènes de circulation, répondant aux défis des systèmes de transport intelligents. Ce cadre unifié améliore la richesse sémantique et la fidélité visuelle des images générées, ce qui est crucial pour des applications telles que la surveillance du trafic et la conduite autonome.

A new method called Text2Traffic has been introduced for generating and editing images of traffic scenes, addressing challenges in intelligent transportation systems. This unified framework enhances the semantic richness and visual fidelity of generated images, which is crucial for applications like traffic monitoring and autonomous driving.

Text2Traffic: A Text-to-Image Generation and Editing Method for Traffic Scenes

arXiv:2511.17221v1 Announce Type: new 
Abstract: Learning 3D scene geometry and semantics from images is a core challenge in computer vision and a key capability for autonomous driving. Since large-scale 3D annotation is prohibitively expensive, recent work explores self-supervised learning directly from sensor data without manual labels. Existing approaches either rely on 2D rendering consistency, where 3D structure emerges only implicitly, or on discretized voxel grids from accumulated lidar point clouds, limiting spatial precision and scalability. We introduce QueryOcc, a query-based self-supervised framework that learns continuous 3D semantic occupancy directly through independent 4D spatio-temporal queries sampled across adjacent frames. The framework supports supervision from either pseudo-point clouds derived from vision foundation models or raw lidar data. To enable long-range supervision and reasoning under constant memory, we introduce a contractive scene representation that preserves near-field detail while smoothly compressing distant regions. QueryOcc surpasses previous camera-based methods by 26% in semantic RayIoU on the self-supervised Occ3D-nuScenes benchmark while running at 11.6 FPS, demonstrating that direct 4D query supervision enables strong self-supervised occupancy learning. https://research.zenseact.com/publications/queryocc/

تم تقديم QueryOcc كإطار عمل قائم على الاستقلال الذاتي للتعلم الذاتي الذي يتعلم الإشغال السيماني ثلاثي الأبعاد بشكل مستمر مباشرة من بيانات المستشعر، مما يعالج التحديات المتعلقة بالهندسة السيمانية والمشاهد ثلاثية الأبعاد في رؤية الكمبيوتر، خاصة لتطبيقات القيادة الذاتية.

Se ha presentado QueryOcc como un marco de auto-supervisión basado en consultas que aprende la ocupación semántica 3D continua directamente a partir de datos de sensores, abordando los desafíos de la geometría y la semántica de escenas 3D en la visión por computadora, especialmente para aplicaciones de conducción autónoma.

QueryOcc a été introduit comme un cadre d'auto-supervision basé sur des requêtes qui apprend l'occupation sémantique 3D continue directement à partir des données de capteurs, répondant aux défis de la géométrie et de la sémantique des scènes 3D dans la vision par ordinateur, en particulier pour les applications de conduite autonome.

QueryOcc has been introduced as a query-based self-supervised framework that learns continuous 3D semantic occupancy directly from sensor data, addressing the challenges of 3D scene geometry and semantics in computer vision, particularly for autonomous driving applications.

QueryOcc: Query-based Self-Supervision for 3D Semantic Occupancy

arXiv:2503.09878v4 Announce Type: replace 
Abstract: Vision foundation models (VFMs) such as DINO have led to a paradigm shift in 2D camera-based perception towards extracting generalized features to support many downstream tasks. Recent works introduce self-supervised cross-modal knowledge distillation (KD) as a way to transfer these powerful generalization capabilities into 3D LiDAR-based models. However, they either rely on highly complex distillation losses, pseudo-semantic maps, or limit KD to features useful for semantic segmentation only. In this work, we propose CleverDistiller, a self-supervised, cross-modal 2D-to-3D KD framework introducing a set of simple yet effective design choices: Unlike contrastive approaches relying on complex loss design choices, our method employs a direct feature similarity loss in combination with a multi layer perceptron (MLP) projection head to allow the 3D network to learn complex semantic dependencies throughout the projection. Crucially, our approach does not depend on pseudo-semantic maps, allowing for direct knowledge transfer from a VFM without explicit semantic supervision. Additionally, we introduce the auxiliary self-supervised spatial task of occupancy prediction to enhance the semantic knowledge, obtained from a VFM through KD, with 3D spatial reasoning capabilities. Experiments on standard autonomous driving benchmarks for 2D-to-3D KD demonstrate that CleverDistiller achieves state-of-the-art performance in both semantic segmentation and 3D object detection (3DOD) by up to 10% mIoU, especially when fine tuning on really low data amounts, showing the effectiveness of our simple yet powerful KD strategy

يمثل تقديم CleverDistiller تقدمًا كبيرًا في مجال تقطير المعرفة عبر الأنماط الذاتية، مما يتيح نقل الميزات من نماذج الرؤية الأساسية ثنائية الأبعاد إلى نماذج تعتمد على LiDAR ثلاثية الأبعاد. يستخدم هذا الإطار خسارة مباشرة لتشابه الميزات ورأس إسقاط متعدد الطبقات، مما يعزز تعلم الاعتماديات الدلالية المعقدة في تطبيقات القيادة الذاتية.

La introducción de CleverDistiller marca un avance significativo en la destilación de conocimientos cross-modales auto-supervisados, permitiendo la transferencia de características de modelos de fundación visual 2D a modelos basados en LiDAR 3D. Este marco utiliza una pérdida de similitud de características directa y una cabeza de proyección de perceptrón multicapa, mejorando el aprendizaje de dependencias semánticas complejas en aplicaciones de conducción autónoma.

L'introduction de CleverDistiller représente une avancée significative dans la distillation de connaissances cross-modales auto-supervisées, permettant le transfert de caractéristiques des modèles de fondation visuelle 2D vers des modèles basés sur LiDAR 3D. Ce cadre utilise une perte de similarité de caractéristiques directe et une tête de projection à perceptron multicouche, améliorant l'apprentissage des dépendances sémantiques complexes dans les applications de conduite autonome.

The introduction of CleverDistiller marks a significant advancement in self-supervised cross-modal knowledge distillation, enabling the transfer of features from 2D vision foundation models to 3D LiDAR-based models. This framework utilizes a direct feature similarity loss and a multi-layer perceptron projection head, enhancing the learning of complex semantic dependencies in autonomous driving applications.

Text2Traffic: A Text-to-Image Generation and Editing Method for Traffic Scenes

Was this article worth reading? Share it