Steps and chaining

Overview

A single-step protocol’s one step reads workflow.inputFile directly. A multi-step protocol chains steps together by having each step past the first consume a prior step’s output instead — this page covers that rule and walks through a real four-step example.

The chaining rule

• The first step in a chain takes input_file_pointer: $( workflow.inputFile ).

• A later step chains .output from whichever prior step actually produced the version of the file it needs — usually, but not always, the step immediately before it. A step that only annotates or labels a file without changing its content doesn’t produce a new file version for the next step to consume; a step that transforms the file’s content does.

• The task-script function on the producing side must actually return the file pointer the next step expects — the expression layer has no way to verify that for you.

The most common multi-step wiring bug is the reverse mistake: leaving a later step pointed at workflow.inputFile when it should be reading a prior step’s .output instead.

Worked example: decorate → harmonize → enrich → push

This example wires four functions from a single task script (sspdemo-taskscript, v5.0.0) into one protocol: decorate adds labels to the raw file, harmonize parses it into IDS format, enrich pulls in data from elsewhere in the platform, and push sends the result to a third-party application.

protocolSchema: "v3"
name: "Multistep - v3 protocol"
description: "Protocol that pushes data to a third-party application."

config:
  labels_json:
    label: "Labels that can be added to file."
    description: "A json of labels that can be added to a file"
    type: "object"
    required: false
  kaggle_username:
    label: "Kaggle Username."
    description: "Kaggle Username to use for pushing data."
    type: "string"
    required: true
  kaggle_api_key:
    label: "Kaggle API Key."
    description: "Kaggle API Key to use for pushing data."
    type: "secret"
    required: true

steps:
  - id: decorate-input-file-step
    task:
      namespace: private-example-org
      slug: sspdemo-taskscript
      version: v5.0.0
      function: decorate-input-file
    input:
      input_file_pointer: $( workflow.inputFile )
      labels_json: $( config.labels_json )
  - id: harmonize-input-file-step
    task:
      namespace: private-example-org
      slug: sspdemo-taskscript
      version: v5.0.0
      function: harmonize-input-file
    input:
      input_file_pointer: $( workflow.inputFile )
  - id: enrich-input-file-step
    task:
      namespace: private-example-org
      slug: sspdemo-taskscript
      version: v5.0.0
      function: enrich-input-file
    input:
      input_file_pointer: $( steps["harmonize-input-file-step"].output )
  - id: push-step
    task:
      namespace: private-example-org
      slug: sspdemo-taskscript
      version: v5.0.0
      function: push-data
    input:
      input_file_pointer: $( steps["enrich-input-file-step"].output )
      kaggle_username: $( config.kaggle_username )
      kaggle_api_key: $( config.kaggle_api_key )

Note the required configs (kaggle_username, kaggle_api_key) are listed before the optional one (labels_json) — see the ordering rule in Config and constants conventions.

harmonize-input-file-step is not a chaining bug. It deliberately re-reads workflow.inputFile instead of decorate-input-file-step’s output, because “decorate” only adds labels to the raw file rather than producing a new derived file — “harmonize” needs the original raw content. The two steps after it, enrich-input-file-step and push-step, each correctly consume the immediately preceding step’s .output, since those steps do transform the file’s content. This is the general rule from above: chain .output from whichever prior step actually produced the file this step needs — not necessarily the step immediately before it, and not necessarily every step past the first.

Deploying a protocol

Once protocol.yml and manifest.json are ready, publish the protocol with the TetraScience CLI:

ts-cli publish --type protocol --namespace private-{org} --slug multistep --version v1.0.0 {protocol-folder} -c {auth-folder}/auth.json

Replace {org} with the target organization’s slug, {protocol-folder} with the local folder containing the protocol code, and {auth-folder} with the folder containing authentication information. Published protocol versions are immutable (see Versioning) — bump --version rather than re-publishing an existing one.