Downloading large files from Dryad

Rich Pang
2025-09-27

Dryad is one of the major online repositories for hosting biological research data, and which makes it easy to download files just by clicking the download link. However, downloading large files (e.g. 100Gb) can be tricky, since limited bandwidth can yield hours-long downloads, which can easily get canceled if there is any interruption in network connectivity.

To avoid these problems, do the following.

Get the actual file URL

If you right-click on a Dryad file and select "Copy Link" you will probably receive something like https://datadryad.org/downloads/file_stream/4111789. Unfortunately this is not the proper URL of the actual data file, but rather a link that triggers a series of redirects to the actual file (hosted on Amazon S3). Thus, you will get an error if you try to paste it into a command-line downloading utility.

To get the correct link:

  1. Start the download by clicking on the data file as usual.
  2. Go to your download manager and cancel the download.
  3. Right-click on the download in the download manager and click "Copy Download Link". You should get something long and comlicated, e.g. of the form https://dryad-assetstore-merritt-west.s3.uw-west.....623514842c.
  4. Copy this link into your clipboard.

Download the file using the command line

There are various utilities you can use to download the file. You want something that will resume if the network connection is interrupted.

Download using curl

curl is a built-in command-line tool for downloading files from the internet. To download your data file with curl, use the following command (one line):

curl -L -C - "https://dryadassetstore-very-long-url" -o local_file_name_to_save_to.zip

Note: make sure to include the -L -C - flags to ensure the download will resume after network connectivity interruptions.

Download using aria2

If you do not have aria2 installed you can install it with sudo apt install aria2 on linux, or brew install aria2 on Mac (note that you will first need to install brew, an auxiliary package manager for Mac). After aria2 has been installed, run the command:

aria2c -x 16 -s 16 -k 1M "https://dryadassetstore-very-long-url" -o local_file_name_to_save_to.zip

The flags here help decompose and parallelize the download for faster download speeds. More information about using aria2 is available here.

Both of these commands will download the full data file. It may still take a while due to bandwidth limitations on Dryad downloads, so leave your local machine powered on until the download has finished. However, these commands will ensure that the download continues through network interruptions.

Note that if the connection is interrupted for a long time (several minutes), this might end the downloading process. However, the partial download and metadata for continuing it are saved by aria, so just run the exact same command again to pick up the download where it left off.