Mapping the primary model to the IDS

Overview

This page outlines our guidelines for mapping the primary model to the IDS.

Guidelines

The mapping from the primary model to the IDS should be done in a function named map_to_ids which takes the primary model as input and returns an instance of the IDS model.

There should be one function per top level array in the IDS, which takes the necessary parts of the primary model as input and returns an instance of the object for that array. For example, if the IDS has a top level array named samples, there should be a function named map_to_samples which takes the necessary parts of the primary model as input and returns a list of ts_ids_core.schema.Sample objects.

The map_to_ids function should then invoke these helper functions and assign the results to the top level arrays in the IDS. The “dimensions” (i.e. objects containing primary keys pk) should be mapped first, followed by the “fact” (i.e. objects containing foreign keys fk_...) which reference the dimensions.

During this step the raw values from the input file should be transformed to the desired formats for the IDS. This includes:

  • Casting to the correct data types, e.g. a str to an int or float

  • Formatting datetimes to ISO-8601 using task_script_utils datetime_parser.DatetimeParser.to_tsformat()

  • Converting the raw unit strings to the TetraScience standard units via ts_lib_units UnitParser.parse()