arXiv:2511.10896v1 Announce Type: cross 
Abstract: Despite remarkable advancements in supervised pansharpening neural networks, these methods face domain adaptation challenges of resolution due to the intrinsic disparity between simulated reduced-resolution training data and real-world full-resolution scenarios.To bridge this gap, we propose an unsupervised pansharpening framework, CLIPPan, that enables model training at full resolution directly by taking CLIP, a visual-language model, as a supervisor. However, directly applying CLIP to supervise pansharpening remains challenging due to its inherent bias toward natural images and limited understanding of pansharpening tasks. Therefore, we first introduce a lightweight fine-tuning pipeline that adapts CLIP to recognize low-resolution multispectral, panchromatic, and high-resolution multispectral images, as well as to understand the pansharpening process. Then, building on the adapted CLIP, we formulate a novel \textit{loss integrating semantic language constraints}, which aligns image-level fusion transitions with protocol-aligned textual prompts (e.g., Wald's or Khan's descriptions), thus enabling CLIPPan to use language as a powerful supervisory signal and guide fusion learning without ground truth. Extensive experiments demonstrate that CLIPPan consistently improves spectral and spatial fidelity across various pansharpening backbones on real-world datasets, setting a new state of the art for unsupervised full-resolution pansharpening.

يقدم المقال CLIPPan، وهو إطار عمل غير خاضع للإشراف في عملية دمج الصور، يستخدم نموذج CLIP، وهو نموذج لغوي بصري، كمشرف. تتناول هذه الطريقة التحديات التي تواجهها طرق دمج الصور الخاضعة للإشراف، وخاصة مشكلات التكيف مع المجال الناتجة عن التباين بين بيانات التدريب منخفضة الدقة المحاكية والسيناريوهات الحقيقية عالية الدقة. تم تصميم الإطار لتحسين فهم عملية دمج الصور وزيادة قدرة النموذج على التعرف على أنواع مختلفة من الصور، مما يضع معيارًا جديدًا في دمج الصور غير الخاضع للإشراف بدقة كاملة.

El artículo presenta CLIPPan, un marco de pansharpening no supervisado que utiliza CLIP, un modelo de lenguaje visual, como supervisor. Este enfoque aborda los desafíos que enfrentan los métodos de pansharpening supervisados, particularmente los problemas de adaptación de dominio que surgen de la disparidad entre los datos de entrenamiento simulados de baja resolución y los escenarios del mundo real de alta resolución. El marco está diseñado para mejorar la comprensión del proceso de pansharpening y aumentar la capacidad del modelo para reconocer varios tipos de imágenes, estableciendo así un …

L'article présente CLIPPan, un cadre de pansharpening non supervisé qui utilise CLIP, un modèle visuel-linguistique, comme superviseur. Cette approche répond aux défis rencontrés par les méthodes de pansharpening supervisées, en particulier les problèmes d'adaptation de domaine dus à la disparité entre les données d'entraînement simulées à basse résolution et les scénarios réels à haute résolution. Le cadre est conçu pour améliorer la compréhension du processus de pansharpening et renforcer la capacité du modèle à reconnaître divers types d'images, établissant ainsi un nouvel état de l'art dan…

The article presents CLIPPan, an unsupervised pansharpening framework that utilizes CLIP, a visual-language model, as a supervisor. This approach addresses the challenges faced by supervised pansharpening methods, particularly the domain adaptation issues arising from the disparity between simulated low-resolution training data and real-world high-resolution scenarios. The framework is designed to improve the understanding of the pansharpening process and enhance the model's ability to recognize various image types, ultimately setting a new state of the art in unsupervised full-resolution pans…

CLIPPan: Adapting CLIP as A Supervisor for Unsupervised Pansharpening

Was this article worth reading? Share it

Ready to build your own newsroom?