Context-Aware Multimodal Representation Learning for Spatio-Temporally Explicit Environmental Modelling

The accurate retrieval of vegetation biophysical variables from satellite imagery is essential for ecosystem monitoring and agricultural management. This study introduces a physics-informed Transformer-VAE architecture designed to invert the PROSAIL radiative transfer model, enabling the simultaneous estimation of key canopy parameters from Sentinel-2 data. The model, trained solely on simulated data, demonstrates performance comparable to state-of-the-art methods that utilize real imagery, successfully retrieving leaf area index and canopy chlorophyll content from real-world datasets.

OlmoEarth is a new multimodal, spatio-temporal foundation model designed for Earth observation data, which combines the characteristics of images, videos, and text. Utilizing a self-supervised learning approach, it incorporates innovative masking strategies and loss functions tailored for this domain. OlmoEarth outperforms 12 other foundation models across various benchmarks, achieving top results in multiple tasks. It serves as the backbone for a comprehensive platform aimed at enhancing data collection, labeling, training, and inference for Earth observation applications.

Monitoring the spatiotemporal dynamics of forest CO2 uptake, known as Gross Primary Production (GPP), poses significant challenges in terrestrial ecosystem research. While Eddy Covariance towers provide high-frequency estimates, their spatial limitations hinder large-scale assessments. Remote sensing offers a scalable alternative, yet many methods rely on single-sensor spectral indices and statistical models that struggle to capture GPP's complex temporal dynamics. This study evaluates the performance of GPT-2, a transformer model, against LSTM, a recurrent neural network, for GPP prediction u…