arXiv:2509.21670v3 Announce Type: replace 
Abstract: We introduce MORPH, a modality-agnostic, autoregressive foundation model for partial differential equations (PDEs). MORPH is built on a convolutional vision transformer backbone that seamlessly handles heterogeneous spatiotemporal datasets of varying data modality (1D--3D) at different resolutions, and multiple fields with mixed scalar and vector components. The architecture combines (i) component-wise convolution, which jointly processes scalar and vector channels to capture local interactions, (ii) inter-field cross-attention, which models and selectively propagates information between different physical fields, (iii) axial attentions, which factorize full spatiotemporal self-attention along individual spatial and temporal axes to reduce computational burden while retaining expressivity. We pretrain multiple model variants on a diverse collection of heterogeneous PDE datasets and evaluate transfer to a range of downstream prediction tasks. Using both full-model fine-tuning and parameter-efficient low-rank adapters (LoRA), MORPH outperforms models trained from scratch. Across extensive evaluations, MORPH matches or surpasses strong baselines and recent state-of-the-art models. Collectively, these capabilities present a flexible and powerful backbone for learning from the heterogeneous and multimodal nature of scientific observations, charting a path toward scalable and data-efficient scientific machine learning. The source code, datasets, and models are publicly available at https://github.com/lanl/MORPH.

تم تقديم MORPH كنموذج أساسي ذاتي التكرار غير معتمد على النمط، مصمم للمعادلات التفاضلية الجزئية (PDEs)، باستخدام هيكل محول رؤية تلافيفي لإدارة مجموعات بيانات زمنية مكانية متنوعة عبر دقة وأنماط بيانات مختلفة. يتضمن النموذج تقنيات متقدمة مثل الالتفاف حسب المكونات والانتباه المتقاطع بين الحقول لتعزيز قدراته التنبؤية.

Se ha presentado MORPH como un modelo de fundación autoregresivo agnóstico a la modalidad, diseñado para ecuaciones en derivadas parciales (EDP), utilizando una arquitectura de transformador de visión convolucional para gestionar conjuntos de datos espaciotemporales diversos a diferentes resoluciones y modalidades de datos. El modelo incorpora técnicas avanzadas como la convolución por componentes y la atención cruzada entre campos para mejorar sus capacidades predictivas.

MORPH a été introduit comme un modèle de fondation autoregressif agnostique aux modalités, conçu pour les équations aux dérivées partielles (EDP), utilisant une architecture de transformateur de vision convolutionnelle pour gérer des ensembles de données spatiotemporelles divers à différentes résolutions et modalités de données. Le modèle intègre des techniques avancées telles que la convolution par composants et l'attention croisée entre champs pour améliorer ses capacités prédictives.

MORPH has been introduced as a modality-agnostic, autoregressive foundation model designed for partial differential equations (PDEs), utilizing a convolutional vision transformer backbone to manage diverse spatiotemporal datasets across various resolutions and data modalities. The model incorporates advanced techniques such as component-wise convolution and inter-field cross-attention to enhance its predictive capabilities.

MORPH: PDE Foundation Models with Arbitrary Data Modality

Was this article worth reading? Share it

LucidQuery AI

Airparser

Deptho.ai