What Is a Run?

The Basics

Runs are the foundation of Spell. Creating a run is how you execute your code on Spell's computing infrastructure.

The run command is intended to be simple. Any code, software, binaries, etc., that you can run locally on your computer can be run on Spell — you simply put spell run in front of the same commands you would use locally and the commands will run remotely on Spell.

Anatomy of a Run Command

The spell run command is used to create runs and is likely the command you'll use most while using Spell.


Run commands consist of two main parts: 1. Spell command which includes information such as the machine type you want to use, and files you want to mount, and 2. Your command.

One of the simplest commands you can run is spell run "echo hello world", which will print hello world to the screen. In this case, only spell run is related to the Spell command, while echo hello world is your regular command:

$ spell run "echo hello world"
✨ Casting spell #1…
✨ Stop viewing logs with ^C
✨ Run is running
hello world

A more complicated run example is the command to train a style transfer neural network. (You can find the full example at learn.spell.run/transferring_style.)

In this example, we've mounted a file using the spell --mount flag, specified that we want a K80 GPU with the --machine-type flag, and added both an apt and a pip dependency. These flags add the correct files and dependencies to your spell environment.

Then everything in the quotation marks is the regular command from the fast style transfer repository.

$ spell run --mount runs/1/data:datasets \
            --machine-type K80 \
            --apt ffmpeg \
            --pip moviepy \
  "python style.py \
            --checkpoint-dir ckpt \
            --style images/style/my-style-image.jpg \
            --style-weight 1.5e2 \
            --train-path datasets/train2014 \
            --vgg-path datasets/imagenet-vgg-verydeep-19.mat"

Customizing Your Run Environment

Many commands can be run in Spell's default environment. Spell's default environment has the Ubuntu 18.04 operating system, Python versions 2.7 and 3.6 installed, Pytorch, TensorFlow 1, Keras, Anaconda, XGBoost, and Scikit-learn. If a machine type is specified that has a GPU, then CUDA and cuDNN are also included.

However, if further customization is needed, Spell offers several options for customizing your run environment.

Adding Dependencies

Most commands will require you to add dependencies, such as pip and apt to your environment. The run example in the Anatomy of a Run Command section uses both the apt and pip flags: --apt ffmpeg and --pip moviepy

For more examples of adding dependencies, including adding conda files and Docker Hub public images, go to the Dependencies section of our docs.

Early Stopping Conditions

Sometimes if a run doesn't meet a certain condition by a certain point you would rather stop the run automatically. For example, if a run doesn't reach a validation accuracy of 50 percent within 10 epochs, maybe it's not worth proceeding. Early stop conditions can be specified via the --stop-condition flag. For example, --stop-condition "keras/val_acc < 0.5 : 10". The value after the colon is the minimum index for which the condition is valid.

Run IDs

All runs get a unique run ID. The run ID can be found in the ✨ Casting spell #<ID>… message that appears after a run is created.

Run IDs are useful if you want to:

  • Refer back to all the information from your run. You can find this information on the web on your run details page by going to web.spell.run/runs/

  • Mount the outputs from your run into another run. For example, this flag mounts the outputs from run 1 into our current run: --mount runs/1/data:datasets. (See the second example in the above section Anatomy of a Run Command.)

More on Runs

Runs and Git

A run is typically initiated from a Git repository on the machine where you are using Spell, but it does not have to be.

When initiated from a Git repository, Spell uses Git to safely sync the code in your current directory to the remote machine. Turning any directory into a Git repository is very easy - just run git init. Read more about versioning and Git here.

If the run is initiated from a Git repository, the run will automatically sync code from the repository and use it within the run. If the current repo has uncommitted changes, these changes will also be synced and used within the run, and can be downloaded as a separate patch via the web console. If there are any untracked files, the run will provide a warning. This is to avoid any ambiguity about the state of your code being transferred to Spell.


  • Spell uses SSH to sync your code. If you require special SSH configuration (e.g. proxy commands for network firewalls) update the ssh_config file in your Spell config directory. This will likely be in ~/.spell on macOS or Linux and your user's AppData/Roaming/spell directory on Windows.

  • Spell does not support git-lfs. If you have large files inside your repository, we recommend using the spell upload command. Git submodules are not yet supported either, as a workaround you can directly commit the submodules into the parent repo.

  • It is possible to configure Spell to error if uncommitted changes are detected. This can help enforce tracking code changes and improve reproducibility. Configure this in the config file in your Spell config directory.

Everything in the commit that is currently checked out (unless the --commit-ref option is provided to specify a different commit) will be available in the run.

For example, if you go through the steps below, you will see running on Spell print out during your run, showing that the file in your current Git commit was transferred to Spell for the run:

$ mkdir project
$ cd project
$ echo 'running on Spell!' > file
$ git add file && git commit -m 'first commit'
$ spell run cat file

Where Your Run Is Executed

The run will execute in whatever the local current working directory is. For example, if you are working in a nested directory within your Git repository, you can run commands there without worrying about the path relative to the Git repository root.

Run States

Runs transition through the following states:

  • Requested: the run has been created, and is queued for execution. Typically, a run will transition through this state very quickly. However, a run could stay Requested for an extended period of time for two reasons:
    • You already have the maximum number of concurrent runs executing that is allowed per your plan. If this is the case, your run will transition to Building when the number of concurrent runs you have falls below the limit.
    • Spell received a number of runs at the same time and is starting up more machines to execute your run. If this is the case, your run will transition to Building as soon as a machine is ready.
  • Building: the environment for your run is created. This includes installing any dependencies that are specified (e.g. --pip or --apt parameters provided on the command line to create the run) and copying the code from your Git repository into the run. First we check to see if the run uses an environment from a previous run, and if so we used the cached environment to minimize build time in lieu of building the environment again.
  • Mounting: the resources (if any) that were specified are mounted into the run. (See the Resources section for more information.
  • Running: your command is executing!
  • Saving: any new or modified files from your command are saved as a resource into runs/
  • Pushing: your environment is pushed to our cache of environments to expedite a future run with the same environment. Usually, your run will transition through this step very quickly since this push happens asynchronously in the background throughout every other state. The push is usually completed by the time this state is reached unless your run is very quick.

All runs eventually transition to one of the following final states:

  • Complete: this is the normal state for a run to transition to upon completion.
  • Killed: the run was killed using the spell kill command. When a run is killed, it does not transition through any subsequent steps and is immediately terminated.
  • Stopped: the run was stopped using the spell stop command. A stopped run still transitions through the Saving and Pushing states, however its Running state is immediately exited whenever the spell stop command is issued.
  • Failed: the run did not complete due to an error in some other state.