Page Deprecated

Information you find on this page may be out of date and no longer accurate.

Synda

Tool deprecation

Synda has been deprecated and replaced by ESGF Downloader

What is synda?

Synda is a command line tool to search and download files from the Earth System Grid Federation (ESGF) archive. Synda is a useful tool if you need to download Climate Model Intercomparison Project Phase 6 (CMIP6) data in particular. Synda supports different download protocols (e.g. HTTPS) and can download files in parallel.

How to install synda

1. Load Anaconda3

module load Anaconda3

2. Create a conda environment and activate the environment

conda create --prefix=<SYNDA_ENV>

where <SYNDA_ENV> is a directory of your choice. Activate your new environment

conda activate <SYNDA_ENV>

3. Install synda (currently version 3.10)

conda install -c IPSL synda

4. Configure synda

Set the ST_HOME environment variable and populate $ST_HOME, for instance

export ST_HOME=/nesi/nobackup/<YOUR PROJECT>/synda_home

Note: you may want to add the above "export ST_HOME=<...>" somewhere near the bottom of your ~/.bashrc file.

To search and download climate model data you will in addition need to create an account on one or more of the ESGF nodes, e.g. https://esgf-node.llnl.gov/projects/esgf-llnl/. This will require you to provide a user name (USER_NAME) and a password - you will receive an openID in return. Copy your openID string as you will need later.

To configure and store your ESGF credentials, type

synda -h
synda check-env

Paste in the openID you received when creating your ESGF account. If you created an account on esgf-node.llnl.gov then your openID will be:

openID url: https://esgf-node.llnl.gov/esgf-idp/openid/USER_NAME

(USER_NAME is the user name you chose when creating the account). You can change your default configuration by editing $ST_HOME/conf/sdt.conf. I have

indexes = esgf-node.llnl.gov
default_index = esgf-node.llnl.gov

Example

Find all the datasets for given ocean surface temperature ("tos"), variant label ("r1i1p1f1") produced by institution "NOAA-GFDL" for historical data and table Id "Omon":

synda search project=CMIP6 realm=ocean variable=tos variant_label=r1i1p1f1 institution_id=NOAA-GFDL table_id=Omon experiment_id=historical

This will return

new  CMIP6.CMIP.NOAA-GFDL.GFDL-CM4.historical.r1i1p1f1.Omon.tos.gn.v20180701
new  CMIP6.CMIP.NOAA-GFDL.GFDL-CM4.historical.r1i1p1f1.Omon.tos.gr.v20180701
new  CMIP6.CMIP.NOAA-GFDL.GFDL-ESM4.historical.r1i1p1f1.Omon.tos.gn.v20190726
new  CMIP6.CMIP.NOAA-GFDL.GFDL-ESM4.historical.r1i1p1f1.Omon.tos.gr.v20190726

Choose one of the datasets. To find out how big the dataset is, type: 

synda stat CMIP6.CMIP.NOAA-GFDL.GFDL-ESM4.historical.r1i1p1f1.Omon.tos.gr.v20190726

which returns

Total files count: 9
New files count: 9
Total size: 262.8 MB
New files size: 262.8 MB

``To download the dataset, type

synda get CMIP6.CMIP.NOAA-GFDL.GFDL-CM4.historical.r1i1p1f1.Omon.tos.gn.v2018070

The datasets (NetCDF files) will be downloaded locally in your directory.