Large Numerical Data

Represent large numerical data as datacubes or file pointers, not inline scalars.

Chromatograms, spectra, and raw traces are large numeric arrays, not scalars. Two patterns handle them:

• Datacube — when the array is parseable and needed for analysis (plotting, re-integration), model it as aligned arrays of the dimension (time, m/z) and measure (intensity), each with its unit. This keeps the signal queryable and overlay-able.

• File pointer — when the data is a large binary the pipeline cannot or should not parse (a vendor .wiff.scan, a raw blob), reference it with a file pointer so the IDS stays lightweight and the original is preserved.

Structure what will be queried; preserve the rest by reference. Never flatten a chromatogram to a single scalar (e.g. only its peak area) if a consumer might need the trace — that is silent, irreversible data loss. For the components — datacubes and FilePointer / RelatedFile — see Components.

Example — an MRM chromatogram. The integrated peak area is a scalar and belongs in the result record; the intensity-vs-time trace is a datacube; the multi-gigabyte .wiff.scan it came from is a file pointer. All three coexist: the scalar for fast query, the datacube for re-analysis, the pointer for provenance.