Top-level data blocks

Every IDS instance is organized into the same handful of top-level data blocks. This page is the map: the two conceptual domains the blocks draw from, the shape they form, a one-line reference for each block, and how to decide where a given piece of raw data belongs. For field-level conventions, see Design Guidelines.

1. Two domains behind the blocks

The blocks draw on two conceptual domains. A single IDS mixes the two, but the distinction is worth holding in mind — it explains why some blocks group work and others measure it:

• Organizational context — why and for whom. The business framing of the work: programs, projects, and studies, plus the protocols and organizations (sponsor, CRO) behind them. These group related activity; they do not themselves measure anything.

• Scientific measurement — what was measured, and how. The experimental loop: the assay that defines what to measure, the experiment that applies it, the run that executes it on an instrument, and the samples, methods, results, and signals involved.

IDS top-level blocks span both domains, but IDS is deliberately narrower than a full enterprise data model: a single instance captures roughly one experiment’s worth of data, so a few concepts from these domains are not modeled as their own blocks (see Concepts IDS does not model as blocks). Where an omission affects where data lands, this page says so.

These two domains are a conceptual lens — they explain why a block exists (to group work, or to measure it). They are not the same as the metadata / input / output buckets used later in Section 4 and Section 5, which describe where a given piece of raw data lands. The two cut across each other, so don’t expect one to predict the other: for example, the sponsor or CRO behind the work is organizational context (carried on studies), while the individual operator or analyst who executed a step lands in users — a metadata block.

2. The shape of an IDS

An IDS roughly follows how the science is organized, from the broadest organizational frame down to a single measured sample. Read the ordering below as a guide to what each block means, not a strict nesting — how the blocks relate below explains the actual relationships:

Organizational context — why the work exists and how it is grouped

• Project → projects — the research objective that groups related work.

• Study → studies — one planned investigation.

Scientific measurement — what is measured and how it is executed

• Assay → assays — the reusable measurement definition.

• Experiment → experiments — one application of an assay within a study.

• Run → runs — one execution of an experiment on an instrument (e.g. a single plate read).

• Method → methods — the reusable procedure a run executes: the instrument setup, acquisition, and processing steps and their parameters, defined in the abstract. A run references the method it ran.

Measured — the materials and the outputs

• Sample → samples — the material measured (e.g. one entry per well). Often prepared and plated before a run executes, so samples stand on their own rather than nesting inside a run.

• Results & signals → results, datacubes — what each run measured or calculated.

2.1 How the high-level blocks relate

These are the relationships between the high-level blocks — a mental framework, not a required hierarchy or rigid structure. A useful way to hold them in mind is a star schema: a couple of measurement blocks hold the values a consumer queries, and every other block describes that measurement — the who, why, what, and how around it.

Start from the measurement, then read outward to the blocks that describe it:

  MEASUREMENT   the values a consumer ultimately queries
      results · datacubes            (measured · calculated)

  DESCRIBED BY
      organizational     projects → studies
      scientific intent  assays   → experiments
      procedure / run    methods  → runs
      material           samples

• results and datacubes are the measurement — the outputs everything else exists to explain. Values in either block may be measured directly or calculated from other data; don’t assume datacubes are always measured and results always derived. The split between the two blocks is by shape — scalar, queryable values go in results; multi-dimensional signals go in datacubes — not by whether a value was measured or calculated.

• Every other block describes that measurement — who and why (projects, studies), what was intended (assays, experiments), how it was done (methods, runs), and what material was measured (samples). Section 4 sorts these describing blocks into metadata and input (with the measurement itself as output); here they are grouped only by what they describe.

• Read the relationships from the measurement outward. Nothing nests strictly. The clearest way to state how the blocks relate is to start from a result and read out to what describes it: a result was collected on specific samples, via a specific method, as part of a specific run. The arrows above show the usual relationships, not a containment order.

Here are some of the relationships between the high-level blocks:

• A method defines; a run executes. A method is a reusable procedure described in the abstract, with its variables left open; a run is one execution of a method with those variables fixed (the instrument, the date/time, the operator). A single method can describe many results, covering how the instrument was set up, how the data was acquired, and how the results were processed.

• Samples are independent. The material is often prepared and plated before any run executes, so samples is its own block. results may contain foreign keys to samples and another block like runs - this is how samples gets linked to other blocks without needing its own explicit links.

Because the blocks relate loosely rather than nest rigidly, any record can still be traced back to the blocks that describe it — a result to its run, method, and sample; a run to its experiment, study, and project.

2.2 studies, experiments, and assays

These three blocks capture the scientific framing distinctly, so a consumer can tell what was investigated, what was done, and how it is measured apart:

• studies — a planned scientific investigation conducted under a protocol to generate data toward a research objective. A study is the container that holds experiments (and their samples and results); its identity comes from its protocol and objective, not from any single instrument file. A study often carries both an internal (sponsor) and an external (CRO) identifier.

• experiments — one application of an assay to one or more test articles within a study: the scientific-intent event that sits between the reusable assay definition and the individual result rows. An experiment records the decision to run a given assay against given samples; its execution on an instrument is a runs[*] entry.

• assays — a reusable measurement definition: what is measured, against which target, in which biological system, by which method. The same assay recurs across many experiments, so an assay is distinct from its applications — its identity is the readouts it produces plus its target and method, not any single run or result.

One IDS usually covers a single study and experiment — but often several runs. Multiple microplates read together, or a sequence of injections, belong in one file: each execution is its own runs[*] entry, and that run’s results and datacubes link back with fk_run. The studies and experiments context then spans every run in the file. Split runs into separate IDS files only when they are genuinely unrelated — don’t force one-run-per-file onto data the source delivered together.

3. Concepts IDS does not model as blocks

IDS-core keeps a small, stable set of top-level blocks, so a few concepts that richer data models treat as first-class have no block of their own in IDS. They still belong in the data — carried as descriptive context or attributes on a related block — and knowing where they land helps you place data correctly:

• Program — the portfolio layer above project (a therapeutic area, modality strategy, or platform mandate spanning many projects). No program block; record program-level identity as context within projects when a file needs it.

• Protocol — the governing document that specifies how a study is designed and conducted (one protocol may govern many studies, and a study is only interpretable in the context of its protocol). This is distinct from the instrument method in methods: the protocol governs the study; the method configures the run. IDS has no protocol block; carry the protocol’s identity and version as context on studies.

• Subject — the specific individual organism (a given animal or patient) that samples are taken from and treatments are given to. Do not confuse this with the biological system the assays block already describes (its organism and target): the assay says mouse; a subject is this mouse, with its own strain, sex, age, and baseline values. The assay captures the biology at the template level (one value, same for every run); a subject is instance-level (many, each different). For in-vitro work there are no subjects — the test article in a well is a sample; for in-vivo and clinical work, per-individual context is usually carried on samples (which organism each sample came from). Promote it to its own structure only when the science centers on the individual subject.

• Organization — an external party such as a CRO, vendor, sponsor, or testing lab. Recorded as an attribute where it applies — the CRO on studies, the instrument vendor on systems — not as its own block.

• Test article (compound / batch) — the substance being tested (the perturbagen an experiment is run against). Typically captured within samples — the compound and batch a given well or vial contains — rather than as a separate substance block.

These omissions are intentional simplifications, not gaps to backfill: add a dedicated structure only when a use case needs it, following the phased approach.

4. The data blocks at a glance

Grouped by where raw data lands — metadata · input · output (the same three buckets used in Section 5):

Category	Data block	What maps here
metadata	projects	The program / project the data belongs to — for grouping related datasets.
	studies	The planned investigation the data was generated under.
	experiments	The application of an assay that produced the runs — the scientific intent.
	assays	The reusable measurement definition applied — what is measured, target, method.
	systems	The hardware that produced the data — instrument and its components.
	users	People associated with a step — operator, analyst, reviewer.
	runs	One execution of an experiment on the instrument (an IDS may hold several).
input	methods	How a run was configured — method, acquisition / processing parameters.
	samples	The material measured — e.g. one entry per well: compound + concentration.
output	results	Scalar, queryable per-record measurements and calculated values.
	datacubes	Multi-dimensional signals (chromatograms, spectra, traces).

Each data block’s full component definition lives in Components.

5. Placing your data

Roughly bucket the raw data into three categories — metadata, input, output — then pick the data block within it.

Metadata — describes the workflow.

• projects (see the ProjectAttributes component) — the organizational context (program / project); typical of ELN and LIMS, used to associate related datasets.

• studies — one entry per planned investigation the file’s data was generated under; carries the protocol/objective identity and the internal (sponsor) and external (CRO) identifiers.

• experiments — one entry per application of an assay within a study; links the runs that executed it back to the study and the assay it applies.

• assays — one entry per reusable measurement definition applied (what is measured, target, biological system, method). The assay’s scientific subject (target, organism, readout) is captured here.

• systems (see the System component) — the hardware (instrument, columns, detectors, vendor). Record the components used, not how they were configured for the run (that is methods). Something that only holds samples (e.g. a pipettor) is a container → samples[*].holder / location, not a system.

• users (see the User component) — one entry per person associated with a step (an instrument run, a software login, a sign-off).

• runs — one entry per execution of an experiment; per-run output links back with fk_run (see “The shape of an IDS”).

Input — the parameters and materials that define how a run executes and what is tested.

• methods — freeform, typically tied to an instrument family (predefined method components live in ts-ids-components). Holds the protocol and the run’s parameters.

• samples (see the Sample component) — the material input; one entry per sample tested (for a plate assay, one per well: the compound and concentration). Each needs a unique name / ID within its sample type; samples may be shared across assays and searched by compound / batch / experiment / project id.

Output — what the run measured or calculated.

• results — freeform, typically tied to an instrument family (predefined components in ts-ids-components); one record per scalar measurement / value. Not for n-dimensional data.

• datacubes (see DataCube) — n-dimensional matrices (traces, spectra), stored as aligned dimensions + measures. Handled specially by the platform; see the product documentation.

results vs. datacubes — a scalar and its underlying trace often coexist (keep both: the value in results, the signal in datacubes):

• Search individual values in Elasticsearch? → results

• Data is multidimensional? → datacubes

• Meaningful only when analyzed all together? → datacubes

• Individual value meaningful by itself? → results

Whatever the data block, populate only what the raw file carries — never fabricate project / study / experiment / assay structure it doesn’t provide.