arXiv:2507.14643v2 Announce Type: replace 
Abstract: Modern multispectral feature fusion for object detection faces two critical limitations: (1) Excessive preference for local complementary features over cross-modal shared semantics adversely affects generalization performance; and (2) The trade-off between the receptive field size and computational complexity present critical bottlenecks for scalable feature modeling. Addressing these issues, a novel Multispectral State-Space Feature Fusion framework, dubbed MS2Fusion, is proposed based on the state space model (SSM), achieving efficient and effective fusion through a dual-path parametric interaction mechanism. More specifically, the first cross-parameter interaction branch inherits the advantage of cross-attention in mining complementary information with cross-modal hidden state decoding in SSM. The second shared-parameter branch explores cross-modal alignment with joint embedding to obtain cross-modal similar semantic features and structures through parameter sharing in SSM. Finally, these two paths are jointly optimized with SSM for fusing multispectral features in a unified framework, allowing our MS2Fusion to enjoy both functional complementarity and shared semantic space. In our extensive experiments on mainstream benchmarks including FLIR, M3FD and LLVIP, our MS2Fusion significantly outperforms other state-of-the-art multispectral object detection methods, evidencing its superiority. Moreover, MS2Fusion is general and applicable to other multispectral perception tasks. We show that, even without specific design, MS2Fusion achieves state-of-the-art results on RGB-T semantic segmentation and RGBT salient object detection, showing its generality. The source code will be available at https://github.com/61s61min/MS2Fusion.git.

تقدم دراسة حديثة نهجًا مبتكرًا لدمج الميزات متعددة الأطياف في كشف الكائنات، حيث تتناول تحديات كبيرة في هذا المجال. من خلال معالجة الاعتماد المفرط على الميزات المحلية والتوازن بين حجم المجال الاستقبالي وتعقيد الحسابات، يعد هذا الأسلوب الجديد بتحسين أداء التعميم وقابلية التوسع. هذه الخطوة مهمة لأنها قد تؤدي إلى أنظمة كشف كائنات أكثر دقة وكفاءة، مما يفيد تطبيقات متنوعة من المركبات الذاتية القيادة إلى المراقبة.

Un estudio reciente presenta un enfoque innovador para la fusión de características multispectrales en la detección de objetos, abordando desafíos significativos en el campo. Al tratar la dependencia excesiva de las características locales y el equilibrio entre el tamaño del campo receptivo y la complejidad computacional, este nuevo método promete mejorar el rendimiento de generalización y la escalabilidad. Este avance es crucial, ya que podría llevar a sistemas de detección de objetos más precisos y eficientes, beneficiando diversas aplicaciones desde vehículos autónomos hasta vigilancia.

Une étude récente présente une approche révolutionnaire de la fusion de caractéristiques multispectrales pour la détection d'objets, s'attaquant à des défis majeurs dans le domaine. En abordant la dépendance excessive aux caractéristiques locales et l'équilibre entre la taille du champ réceptif et la complexité computationnelle, cette nouvelle méthode promet d'améliorer les performances de généralisation et l'évolutivité. Cette avancée est cruciale car elle pourrait conduire à des systèmes de détection d'objets plus précis et efficaces, bénéficiant à diverses applications allant des véhicules autonomes à la surveillance.

A recent study introduces a groundbreaking approach to multispectral feature fusion for object detection, tackling significant challenges in the field. By addressing the over-reliance on local features and the balance between receptive field size and computational complexity, this novel method promises to enhance generalization performance and scalability. This advancement is crucial as it could lead to more accurate and efficient object detection systems, benefiting various applications from autonomous vehicles to surveillance.

Multispectral State-Space Feature Fusion: Bridging Shared and Cross-Parametric Interactions for Object Detection

arXiv:2511.19117v1 Announce Type: new 
Abstract: The miniaturization of thermal sensors for mobile platforms inherently limits their spatial resolution and textural fidelity, leading to blurry and less informative images. Existing thermal super-resolution (SR) methods can be grouped into single-image and RGB-guided approaches: the former struggles to recover fine structures from limited information, while the latter relies on accurate and laborious cross-camera calibration, which hinders practical deployment and robustness. Here, we propose 3M-TI, a calibration-free Multi-camera cross-Modality diffusion framework for Mobile Thermal Imaging. At its core, 3M-TI integrates a cross-modal self-attention module (CSM) into the diffusion UNet, replacing the original self-attention layers to adaptively align thermal and RGB features throughout the denoising process, without requiring explicit camera calibration. This design enables the diffusion network to leverage its generative prior to enhance spatial resolution, structural fidelity, and texture detail in the super-resolved thermal images. Extensive evaluations on real-world mobile thermal cameras and public benchmarks validate our superior performance, achieving state-of-the-art results in both visual quality and quantitative metrics. More importantly, the thermal images enhanced by 3M-TI lead to substantial gains in critical downstream tasks like object detection and segmentation, underscoring its practical value for robust mobile thermal perception systems. More materials: https://github.com/work-submit/3MTI.

إن إدخال 3M-TI، وهو إطار عمل متعدد الكاميرات بدون معايرة، يمثل تقدمًا كبيرًا في تكنولوجيا التصوير الحراري المحمول. تتضمن هذه الطريقة المبتكرة دمج وحدة انتباه متعددة الأبعاد في UNet الخاص بالانتشار، مما يسمح بمحاذاة الميزات الحرارية وRGB دون الحاجة إلى معايرة معقدة، مما يحسن جودة الصورة وتفاصيلها.

La introducción de 3M-TI, un marco de difusión multi-cámara sin calibración, marca un avance significativo en la tecnología de imágenes térmicas móviles. Este enfoque innovador integra un módulo de atención cruzada en el UNet de difusión, permitiendo la alineación de características térmicas y RGB sin necesidad de calibración compleja de cámaras, mejorando así la calidad y el detalle de las imágenes.

L'introduction de 3M-TI, un cadre de diffusion multi-caméras sans calibration, marque une avancée significative dans la technologie d'imagerie thermique mobile. Cette approche innovante intègre un module d'attention croisée dans le diffusion UNet, permettant l'alignement des caractéristiques thermiques et RGB sans nécessiter de calibration complexe des caméras, améliorant ainsi la qualité et le détail des images.

The introduction of 3M-TI, a calibration-free Multi-camera cross-Modality diffusion framework, marks a significant advancement in mobile thermal imaging technology. This innovative approach integrates a cross-modal self-attention module into the diffusion UNet, enabling the alignment of thermal and RGB features without the need for complex camera calibration, thus enhancing image quality and detail.

Multispectral State-Space Feature Fusion: Bridging Shared and Cross-Parametric Interactions for Object Detection

Was this article worth reading? Share it

Https

SwapAnything.io

Metaflow AI