How to download multiple files from THREDDS and similar

Many of the datasets served accessible through the ADC catalogue are served through THREDDS or similar (e.g. HYRAX). Downloading multiple files from these are slightly more complicated than from e.g. a FTP server. Below two approaches are described.

• Use wget and avoid downloading the catalog.html files. An example formulation is provded below.

  • wget -e robots=off -nH --cut-dirs 4 -nc -r -l5 -A '*.nc' -R 'catalog*' -I /thredds/fileServer/,/thredds/catalog/ 'https://thredds.met.no/thredds/catalog/arcticdata/S2S_drift_TCD/SIDRIFT_DM_NH/catalog.html'

• Use a Python script traversing THREDDS catalogues. An example is provided at.
  • https://github.com/steingod/downloadfromthredds