Clay Foundation Model

An open source AI model for Earth

Clay is a foundational model of Earth. It uses a Vision Transformer architecture adapted to understand geospatial and temporal relations on Earth Observation data. The model is trained via Self-supervised learning (SSL) using a Masked Autoencoder (MAE) method.

The Clay model can be used in three main ways:

• Generate semantic embeddings for any location and time.

• Fine-tune the model for downstream tasks such as classification, regression, and generative tasks.

• Use the model as a backbone for other models.

Where is what

• Our website is madewithclay.org.

• The Clay model code lives on Github. License: Apache-w.0. The latest release is v0.0.1

• The Clay model weights on Hugging Face. License: OpenRAIL-M.

• The Clay documentation lives on this site. License: CC-BY.

• We release the embeddings of the used trainning data on Source Cooperative. License: ODC-BY.

CLAY is a fiscal sponsored project of the 501c3 non-profit Radiant Earth.

---

Table of Contents

Release notes

• Software release notes
• Model release notes

Getting Started

• Installation
• Basic Use

Data Preparation

• Creating datacubes
• Benchmark dataset labels
• Data sampling strategy

Running the model

• Run over a region
• Generating embeddings
• Finetuning
• Embeddings for Contiguous US

Tutorials

• Generative AI for pixel reconstruction
• Create location embeddings
• Interpolating images in embedding space
• Generate embeddings from partial inputs
• Generate and use patch level embeddings from partial input
• Patch level cloud coverage
• Plot similar patches

About Clay

• GitHub
• LinkedIn
• Website