Working with programmatic IDS instances

Usage in task scripts

Consider the example programmatic IDS:

from ts_ids_core.annotations import Nullable, NullableString, Required
from ts_ids_core.base.ids_element import IdsElement
from ts_ids_core.base.ids_field import IdsField
from ts_ids_core.schema import IdsSchema, SchemaExtraMetadataType


class DemoRun(IdsElement):
    instrument: Required[NullableString]
    protocol: Nullable[int]


class DemoIdsSchema(IdsSchema):
    schema_extra_metadata: ClassVar[SchemaExtraMetadataType] = {
        "$id": "https://ids.tetrascience.com/my_namespace/my_ids_slug/v1.0.0/schema.json",
        "$schema": "http://json-schema.org/draft-07/schema#",
    }
    ids_namespace: Required[Literal["my_namespace"]] = IdsField(
        default="my_namespace", alias="@idsNamespace"
    )
    ids_type: Required[Literal["my-ids-slug"]] = IdsField(
        default="my-ids-slug", alias="@idsType"
    )
    ids_version: Required[Literal["v1.0.0"]] = IdsField(
        default="v1.0.0", alias="@idsVersion"
    )

    runs: List[DemoRun]

A recommended workflow, which is used by TetraScience, is to package your programmatic IDSs so they are installable within task-scripts and other Python based projects. You can of course not package the IDS, but using the IDS within a task-script would require copying the IDS definition within the task-script code. In the following sections, we will consider the above demo programmatic IDS as a Python package named ts-ids-demo-schema.

Installing the IDS

Within your task-script project, install your packaged programmatic IDS:

pip install ts-ids-demo-schema

Once installed you can then import and populate your programmatic IDS classes within your task script. Assuming the packaged programmatic IDS defines the classes in the ts_ids_demo_schema.demo module:

from ts_ids_demo_schema.demo import DemoIdsSchema, DemoRun

Using the schema classes in a task script

Consider the following list as raw data produced by plate reader control software.

raw_data = [
    {"instrument": "Plate Reader A", "protocol": "1"},
    {"instrument": "Plate Reader B", "protocol": "2"},
]

In the task script we can populate the programmatic IDS classes with the raw data as follows:

runs = [
    DemoRun(instrument=run.get("instrument"), protocol=run.get("protocol"))
    for run in raw_data
]

demo_ids = DemoIdsSchema(runs=runs)

demo_ids_json = demo_ids.model_dump_json(indent=2)

print(demo_ids_json)

{
  "@idsType": "my-ids-slug",
  "@idsVersion": "v1.0.0",
  "@idsNamespace": "my_namespace",
  "runs": [
    {
      "instrument": "Plate Reader A",
      "protocol": 1
    },
    {
      "instrument": "Plate Reader B",
      "protocol": 2
    }
  ]
}

Notice the protocol fields contain integers whereas the input file defined them as strings. This automatic type conversion is a great benefit of using the previously defined programmatic IDS. Task-scripts typically will not have to consider data type conversion since it is built into the programmatic IDS definition.

Alternatively, we can use built-in Pydantic features to simplify our parsing of the data. In this case we can use the TypeAdapter class.

from pydantic import TypeAdapter

from typing import List

adapter = TypeAdapter(List[DemoRun])

demo_ids = DemoIdsSchema(runs=adapter.validate_python(raw_data))

Note that this requires:

  • The dictionary keys to match the field names within the programmatic IDS class

    • See Pydantic’s AliasPath and AliasChoices for a convenient way to handle when the raw data attributes to not match the IDS field names

  • The dictionaries do not contain any keys which do not appear in the DemoRun class.

Generating the IDS JSON

As shown above, once the programmatic IDS instances have been created, you can serialize the classes to a JSON instance using ts_ids_core.base.ids_element.IdsElement.model_dump_json. This JSON instance is what can be written to the TetraScience platform using Context.write_file from the Context object that is passed in the task-scripts via a protocol.

Create the IDS JSON:

demo_ids.model_dump_json()

Populating Non-Required Fields

Recall our previous definition of DemoRun:

class DemoRun(IdsElement):
    instrument: Required[NullableString]
    protocol: Nullable[int]

The following examples show different ways of populating values and omitting fields when instantiating this IDS element.

Passing a value

Both fields can have non-null values passed to them when instantiating the IDS element. These values are included in the output of model_dump_json.

demo = DemoRun(instrument="Plate Reader", protocol=1.0)

demo_json = demo.model_dump_json()
print(demo_json)

Output:

{"instrument":"Plate Reader","protocol":1}

Passing None

Both fields can be instantiated with a value of None because they are both Nullable.

demo = DemoRun(instrument=None, protocol=None)

demo_json = demo.model_dump_json()
print(demo_json)

Output:

{"instrument":null,"protocol":null}

Not including the non-required field

DemoRun.instrument is a required field so we must populate it with something. However, DemoRun.protocol_number is not a required field, so we can omit it when creating an instance of the class.

demo = DemoRun(instrument="Plate Reader")

demo_json = demo.model_dump_json()
print(demo_json)

Output:

{"instrument":"Plate Reader"}

In IdsElement instances, a value of IDS_UNDEFINED is assigned to fields which are not populated, to distinguish them from other fields. That means demo.protocol_number is defined with a value of IDS_UNDEFINED in the above example. The outcome is that protocol_number is not populated when converting the instance to a dict or JSON.

Passing IDS_UNDEFINED

In some cases it can be useful to use IDS_UNDEFINED in task-script code such as explicitly instantiating a field which shouldn’t be populated in the output IDS instance.

from ts_ids_core.base.ids_undefined_type import IDS_UNDEFINED

raw_data = {"foo": "bar"}  # does not contain "protocol" key

# omit "protocol" if it does not exist in the raw data
demo = DemoRun(instrument="Plate Reader", protocol=raw_data.get("protocol", IDS_UNDEFINED))

demo_json = demo.model_dump_json()
print(demo_json)

Output:

{"instrument":"Plate Reader"}