arXiv:2504.12795v3 Announce Type: replace 
Abstract: Recent advances in natural-domain multi-modal large language models (MLLMs) have demonstrated effective spatial reasoning through visual and textual prompting. However, their direct transfer to remote sensing (RS) is hindered by heterogeneous sensing physics, diverse modalities, and unique spatial scales. Existing RS MLLMs are mainly limited to optical imagery and plain language interaction, preventing flexible and scalable real-world applications. In this article, EarthGPT-X is proposed, the first flexible spatial MLLM that unifies multi-source RS imagery comprehension and accomplishes both coarse-grained and fine-grained visual tasks under diverse visual prompts in a single framework. Distinct from prior models, EarthGPT-X introduces: 1) a dual-prompt mechanism combining text instructions with various visual prompts (i.e., point, box, and free-form) to mimic the versatility of referring in human life; 2) a comprehensive multi-source multi-level prompting dataset, the model advances beyond holistic image understanding to support hierarchical spatial reasoning, including scene-level understanding and fine-grained object attributes and relational analysis; 3) a cross-domain one-stage fusion training strategy, enabling efficient and consistent alignment across modalities and tasks. Extensive experiments demonstrate that EarthGPT-X substantially outperforms prior nature and RS MLLMs, establishing the first framework capable of multi-source, multi-task, and multi-level interpretation using visual prompting in RS scenarios.

EarthGPT-X هو تطوير ثوري في مجال الاستشعار عن بعد، حيث يستخدم نماذج اللغة الكبيرة متعددة الوسائط لتعزيز التفكير المكاني من خلال التحفيز البصري. تتناول هذه الابتكار التحديات التي تطرحها أنماط الاستشعار المتنوعة والمقاييس المكانية، والتي كانت قد حدت سابقًا من فعالية النماذج الحالية. من خلال تمكين تفاعلات أكثر مرونة وقابلية للتوسع مع صور الاستشعار عن بعد، يمتلك EarthGPT-X القدرة على تحسين تحليل البيانات واتخاذ القرارات في تطبيقات متنوعة، مما يجعله تقدمًا ملحوظًا في التكنولوجيا.

EarthGPT-X es un desarrollo innovador en el campo de la teledetección, que aprovecha modelos de lenguaje grande multimodal para mejorar el razonamiento espacial a través de la indicación visual. Esta innovación aborda los desafíos que presentan las diversas modalidades de detección y escalas espaciales, que anteriormente limitaron la efectividad de los modelos existentes. Al permitir interacciones más flexibles y escalables con las imágenes de teledetección, EarthGPT-X tiene el potencial de mejorar significativamente el análisis de datos y la toma de decisiones en diversas aplicaciones, lo que lo convierte en un avance notable en la tecnología.

EarthGPT-X est un développement révolutionnaire dans le domaine de la télédétection, utilisant des modèles de langage large multimodaux avancés pour améliorer le raisonnement spatial grâce à l'invite visuelle. Cette innovation répond aux défis posés par les modalités de détection diverses et les échelles spatiales, qui ont auparavant limité l'efficacité des modèles existants. En permettant des interactions plus flexibles et évolutives avec les images de télédétection, EarthGPT-X a le potentiel d'améliorer considérablement l'analyse des données et la prise de décision dans diverses applications, ce qui en fait une avancée remarquable dans la technologie.

EarthGPT-X is a groundbreaking development in the field of remote sensing, leveraging advanced multi-modal large language models to enhance spatial reasoning through visual prompting. This innovation addresses the challenges posed by diverse sensing modalities and spatial scales, which have previously limited the effectiveness of existing models. By enabling more flexible and scalable interactions with remote sensing imagery, EarthGPT-X has the potential to significantly improve data analysis and decision-making in various applications, making it a noteworthy advancement in technology.

EarthGPT-X: A Spatial MLLM for Multi-level Multi-Source Remote Sensing Imagery Understanding with Visual Prompting

arXiv:2511.17895v1 Announce Type: cross 
Abstract: Spectral super-resolution (SSR) aims to reconstruct hyperspectral images (HSIs) from multispectral observations, with broad applications in remote sensing. Data-driven methods are widely used, but they often overlook physical principles, leading to unrealistic spectra, particularly in atmosphere-affected bands. To address this challenge, we propose the Spectral Super-Resolution Neural Operator (SSRNO), which incorporates atmospheric radiative transfer (ART) prior into the data-driven procedure, yielding more physically consistent predictions. The proposed SSRNO framework consists of three stages: upsampling, reconstruction, and refinement. In the upsampling stage, we leverage prior information to expand the input multispectral image, producing a physically plausible hyperspectral estimate. Subsequently, we utilize a neural operator in the reconstruction stage to learn a continuous mapping across the spectral domain. Finally, the refinement stage imposes a hard constraint on the output HSI to eliminate color distortion. The upsampling and refinement stages are implemented via the proposed guidance matrix projection (GMP) method, and the reconstruction neural operator adopts U-shaped spectral-aware convolution (SAC) layers to capture multi-scale features. Moreover, we theoretically demonstrate the optimality of the GMP method. With the neural operator and ART priors, SSRNO also achieves continuous spectral reconstruction and zero-shot extrapolation. Various experiments validate the effectiveness and generalization ability of the proposed approach.

تم تقديم مشغل الاستعادة الطيفية الفائق (SSRNO) لتحسين إعادة بناء الصور الطيفية الفائقة من الملاحظات متعددة الطيف، مما يعالج قيود الطرق المعتمدة على البيانات التي غالبًا ما تتجاهل المبادئ الفيزيائية، خاصة في النطاقات المتأثرة بالغلاف الجوي. يتضمن هذا الإطار المبتكر سابقة نقل الإشعاع الجوي، مما يضمن توقعات أكثر دقة وتوافقًا مع الفيزياء.

Se ha presentado el Spectral Super-Resolution Neural Operator (SSRNO) para mejorar la reconstrucción de imágenes hiperespectrales a partir de observaciones multispectrales, abordando las limitaciones de los métodos basados en datos que a menudo ignoran principios físicos, especialmente en bandas afectadas por la atmósfera. Este marco innovador incorpora un antecedente de transferencia radiativa atmosférica, asegurando predicciones más precisas y físicamente consistentes.

Le Spectral Super-Resolution Neural Operator (SSRNO) a été introduit pour améliorer la reconstruction d'images hyperspectrales à partir d'observations multispectrales, en répondant aux limites des méthodes basées sur les données qui négligent souvent les principes physiques, en particulier dans les bandes affectées par l'atmosphère. Ce cadre innovant intègre un antécédent de transfert radiatif atmosphérique, garantissant des prédictions plus précises et physiquement cohérentes.

The Spectral Super-Resolution Neural Operator (SSRNO) has been introduced to enhance the reconstruction of hyperspectral images from multispectral observations, addressing the limitations of existing data-driven methods that often neglect physical principles, particularly in atmosphere-affected bands. This innovative framework incorporates atmospheric radiative transfer prior, ensuring more accurate and physically consistent predictions.

EarthGPT-X: A Spatial MLLM for Multi-level Multi-Source Remote Sensing Imagery Understanding with Visual Prompting

Was this article worth reading? Share it

Augmeta

Supametas.AI

Agentcloud