Skip to main content

ClimaX: A foundation model for weather and climate

LIDA: Environment
Machine Learning
Friday 17 March 2023, 3-4pm
LIDA room 11.87, Worsley Building / MS Teams (hybrid)

Speaker – Tung Nguyen, PhD student, UCL


Recent data-driven approaches based on machine learning aim to directly solve a downstream forecasting or projection task by learning a data-driven functional mapping using deep neural networks. However, these networks are trained using curated and homogeneous climate datasets for specific spatiotemporal tasks, and thus lack the generality of currently used computationally intensive physics-informed numerical models for weather and climate modeling. We develop and demonstrate ClimaX, a flexible and generalizable deep learning model for weather and climate science that can be trained using heterogeneous datasets spanning different variables, spatio-temporal coverage, and physical groundings. ClimaX extends the Transformer architecture with novel encoding and aggregation blocks that allow effective use of available compute and data while maintaining general utility. ClimaX is pretrained with a self-supervised learning objective on climate datasets derived from CMIP6. The pretrained ClimaX can then be fine-tuned to address a breadth of climate and weather tasks, including those that involve atmospheric variables and spatio-temporal scales unseen during pretraining. Compared to existing data-driven baselines, we show that this generality in ClimaX results in superior performance on benchmarks for weather forecasting and climate projections, even when pretrained at lower resolutions and compute budgets.


Tung Nguyen is a second-year PhD student in Computer Science at UCLA. His research interests lie in the intersection of decision making, sequence modeling, and uncertainty quantification. He is also interested in grounding his research in applications to sustainability, especially climate change.