Text based files¶
Overview¶
This page outlines our guidelines for parsing data from text based files to a primary model.
Strategy¶
Text based files usually contain some amount of structure, typically a mix between key-value sections and tabular data.
Use ts-lib-parsy to create a parsy parser which:
Extracts data from the non-tabular sections of the file, validating the structure & content via a
pydantic.BaseModelprimary model.Extracts tabular data into a
pandas.DataFrame, validating the structure & content via apandera.DataFrameModelprimary model.