SystemsLab Step Reference

This page describes how to use steps when writing your own SystemsLab experiments: how to use them in your experiment jobs, and what steps are available by default.

In SystemsLab, steps are individual units of work that you can combine to define what a job will do. The do tasks like running a bash script, uploading and artifact, or waiting on a barrier. Using a step in your job looks like this:

- uses: shell
  with:
    shell: bash
    run: echo "test"

or, if you are writing your job in jsonnet:

local systemslab = import "systemslab.libsonnet";

systemslab.bash('echo "test"')

Built-in Steps

These are step definitions that are included with a SystemsLab installation by default. All builtin step definitions are available at both <builtin> and systemslab/<builtin>.

systemslab/shell

The shell step allows you to run a script using a system shell (e.g. bash, sh, zsh, etc.). It has two parameters:

• run - The shell script to run. This will be passed to the standard input of the shell program.
• shell - The shell to use. This will be looked up in the runner's $PATH. The default is bash.

Like all builtin steps, the shell step is available at both shell and systemslab/shell.

Examples

The simplest use of this step is to use it to just run some commands

- uses: systemslab/shell
  with:
    run: cp some-file

It is also fine to run multiple bash commands in a single step

- uses: systemslab/shell
  with:
    run: |
      echo "one thing"
      cat some-file.txt

By default, the shell step runs the commands using bash. However, you can use an alternate shell as long it as available on the $PATH of the runner that is running the job. If the shell is only available on a subset of hosts then you can use tag constraints to ensure your job is only scheduled to those hosts.

- uses: systemslab/shell
  with:
    shell: sh
    run: echo "test"

You can also use programs that are not traditional shells. As long as they accept a script provided on stdin then it can be used by the shell step.

- uses: systemslab/shell
  with:
    shell: python3
    run: print("hello from python!")

You can also use this for programs that don't actually interpret their input as a script. For example, you can use cat to print text into the logs

- uses: systemslab/shell
  with:
    shell: cat
    run: Here's my very own log message!

systemslab/barrier

The barrier step allows you to synchronize multiple threads. It works exactly like a barrier would when using multiple threads: jobs queue up at the barrier until all of them have reached it and then they all get released at once. Its parameters are

• name - A unique name for this barrier. This must be unique within the experiment.
• timeout - An optional timeout in seconds. If the barrier does not become ready within this timeout then the action will fail. By default this is 3600s.

Note that the barrier step is handled specially by the SystemsLab server. The server creates the barriers in advance by scanning the experiment specification to figure out which barriers are needed. Attempting to directly use the barrier APIs to replicate the barrier will not work.

Like all builtin steps, the barrier step is available as both barrier and systemslab/barrier.

Examples

At its most basic, the barrier step allows you to synchronize two jobs

jobs:
  job1:
    steps:
      - uses: shell
        with:
          run: echo "This runs first"
      - uses: barrier
        with:
          name: job1-done
  job2:
    steps:
      - uses: barrier
        with:
          name: job1-done
      - uses: shell
        with:
          run: echo "This runs second"

You can also use barriers to synchronize a subset of jobs. It is not necessary that all jobs block on each and every barrier. Here job1 and job2 are synchronized while job3 runs independently.

jobs:
  job1:
    steps:
      - uses: shell
        with:
          run: sleep 10
      - uses: barrier
        with:
          name: sync
  job2:
    steps:
      - uses: barrier
        with:
          name: sync
  job3:
    steps:
      - uses: shell
        with:
          run: echo "I stop for nobody!"

One thing to be aware of is that barriers can cause deadlocks if they are used improperly. To avoid deadlocks make sure to wait on the barriers in the same order in each job. The job below will eventually fail once the barriers time out but it will be blocked until then.

jobs:
  job1:
    - uses: barrier
      with:
        name: b1
    - uses: barrier
      with:
        name: b2
  job2:
    - uses: barrier
      with:
        name: b2
    - uses: barrier
      with:
        name: b1

systemslab/upload-artifact

This step allows up to upload an artifact that is saved by the SystemsLab server for later use. Uploading artifacts is the main way to store the results generated by running an experiment. Its parameters are

• path - The path to the file that should be uploaded.
• name (optional) - The name of the artifact to be uploaded. If not provided then the last component of path will be used instead.
• tags - A set of tags that are used to document what this artifact is. These are available for use by SystemsLab and other systems to determine what sort of post-processing should be done on an artifact or to find artifacts that match a condition. This can be either a string or an array of strings.

Like all builtin steps, the upload-artifact step is available as both upload-artifact and systemslab/upload-artifact.

systemslab/write-file

This step allows you to easily write text out to a file. This was originally added to allow more easily writing out config files but can also be used to write out binary files as well, although that is likely to only be practical if the experiment specification is written in jsonnet. It takes two parameters

• path - The path at which to write the file.
• content - The contents of the file. This may either be a string or an array of bytes.

Like all builtin steps, this one is available as both write-file and systemslab/write-file.