ts_lib_parsy package#

Submodules#

ts_lib_parsy.error module

Module contents#

class SupportsAdd(*args, **kwargs)[source]#: Bases: Protocol[_T_contra, _T_co]

noop(val: T) → T[source]#

class Result(status: 'bool', index: 'int', value: 'OUT_co', furthest: 'int', expected: 'FrozenSet[str]')[source]#

Bases: Generic[OUT_co]

status: bool#

index: int#

value: OUT_co#

furthest: int#

expected: FrozenSet[str]#

static success(index: int, value: OUT) → Result[OUT][source]#

static failure(index: int, expected: str) → Result[Any][source]#

aggregate(other: Result[Any] | None) → Result[OUT][source]#

class Parser(wrapped_fn: Callable[[str, int], Result[OUT_co]])[source]#

Bases: Generic[OUT_co]

A Parser is an object that wraps a function whose arguments are a string to be parsed and the index on which to begin parsing. The function should return either Result.success(next_index, value), where the next index is where to continue the parse and the value is the yielded value, or Result.failure(index, expected), where expected is a string indicating what was expected, and the index is the index of the failure.

parse(stream: str) → OUT_co[source]#: Attempts to parse the given string (or list). If the parse is successful and consumes the entire string, the result is returned - otherwise, a ParseError is raised.

parse_partial(stream: str) → Tuple[OUT_co, str][source]#: Parse the longest possible prefix of a given string. Return a tuple of the result and the rest of the string, or raise a ParseError.

bind(bind_fn: Callable[[OUT1], Parser[OUT2]]) → Parser[OUT2][source]#: Returns a parser which, if the initial parser is successful, passes the result to bind_fn, and continues with the parser returned from bind_fn. This is the monadic binding operation.

map(map_fn: Callable[[OUT1], OUT2]) → Parser[OUT2][source]#

Returns a parser that transforms the produced value of the initial parser with map_fn.

>>> regex(r'[0-9]+').map(int).parse('1234')
1234

This is the simplest way to convert parsed strings into the data types that you need.

concat() → Parser[str][source]#

Returns a parser that concatenates together (as a string) the previously produced values. Usually used after many() and similar methods that produce multiple values.

>>> letter.at_least(1).parse("hello")
['h', 'e', 'l', 'l', 'o']
>>> letter.at_least(1).concat().parse("hello")
'hello'

then(other: Parser[OUT2]) → Parser[OUT2][source]#

Returns a parser which, if the initial parser succeeds, will continue parsing with other_parser. This will produce the value produced by other_parser.

>>> string('x').then(string('y')).parse('xy')
'y'

skip(other: Parser[Any]) → Parser[OUT1][source]#

Similar to Parser.then(), except the resulting parser will use the value produced by the first parser.

>>> string('x').skip(string('y')).parse('xy')
'x'

result(res: OUT2) → Parser[OUT2][source]#

Returns a parser that, if the initial parser succeeds, always produces val.

>>> string('foo').result(42).parse('foo')
42

many() → Parser[List[OUT_co]][source]#

Returns a parser that expects the initial parser 0 or more times, and produces a list of the results. Note that this parser does not fail if nothing matches, but instead consumes nothing and produces an empty list.

>>> parser = regex(r'[a-z]').many()
>>> parser.parse('')
[]
>>> parser.parse('abc')
['a', 'b', 'c']

times(min: int, max: int | float | None = None) → Parser[List[OUT_co]][source]#: Returns a parser that expects the initial parser at least min times, and at most max times, and produces a list of the results. If only one argument is given, the parser is expected exactly that number of times.

at_most(n: int) → Parser[List[OUT_co]][source]#: Returns a parser that expects the initial parser at most n times, and produces a list of the results.

at_least(n: int) → Parser[List[OUT_co]][source]#: Returns a parser that expects the initial parser at least n times, and produces a list of the results.

optional(default: OUT2 = None) → Parser[OUT1 | OUT2][source]#

Returns a parser that expects the initial parser zero or once, and maps the result to a given default value in the case of no match. If no default value is given, None is used.

>>> string('A').optional().parse('A')
'A'
>>> string('A').optional().parse('')
None
>>> string('A').optional('Oops').parse('')
'Oops'

until(other: Parser[Any], min: int = 0, max: int | float = inf) → Parser[List[OUT_co]][source]#

Returns a parser that expects the initial parser followed by other. The initial parser is expected at least min times and at most max times.

>>> seq(string('A').until(string('B')), string('BC')).parse('AAABC')
[['A','A','A'], 'BC']
>>> string('A').until(string('B')).then(string('BC')).parse('AAABC')
'BC'

sep_by(sep: Parser[Any], *, min: int = 0, max: int | float = inf) → Parser[List[OUT]][source]#

Like Parser.times(), this returns a new parser that repeats the initial parser and collects the results in a list, but in this case separated by the parser sep (whose return value is discarded). By default it repeats with no limit, but minimum and maximum values can be supplied.

>>> csv = letter.at_least(1).concat().sep_by(string(","))
>>> csv.parse("abc,def")
['abc', 'def']

desc(description: str) → Parser[OUT_co][source]#

Adds a description to the parser, which is used in the error message if parsing fails.

>>> year = regex(r'[0-9]{4}').desc('4 digit year')
>>> year.parse('123')
ParseError: expected 4 digit year at 0:0

mark() → Parser[Tuple[Tuple[int, int], OUT_co, Tuple[int, int]]][source]#

Returns a parser that wraps the initial parser’s result in a value containing column and line information of the match, as well as the original value. The new value is a 3-tuple:

((start_row, start_column),
original_value,
(end_row, end_column))

This is useful for being able to report problems with parsing more accurately.

tag(name: str) → Parser[Tuple[str, OUT]][source]#

Returns a parser that wraps the produced value of the initial parser in a 2 tuple containing (name, value). This provides a very simple way to label parsed components. e.g.:

>>> day = regex(r'[0-9]+').map(int)
>>> month = string_from("January", "February", "March", "April", "May",
...                     "June", "July", "August", "September", "October",
...                     "November", "December")
>>> day.parse("10")
10
>>> day.tag("day").parse("10")
('day', 10)

>>> seq(day.tag("day") << whitespace,
...     month.tag("month")
...     ).parse("10 September")
[('day', 10), ('month', 'September')]

It also works well when combined with .map(dict) to get a dictionary of values:

>>> seq(day.tag("name") << whitespace,
...     month.tag("month")
...     ).map(dict).parse("10 September")
{'day': 10, 'month': 'September'}

should_fail(description: str) → Parser[Result[OUT]][source]#

Returns a parser that fails when the initial parser succeeds, and succeeds when the initial parser fails (consuming no input). A description must be passed which is used in parse failure messages.

This is essentially a negative lookahead:

>>> p = letter << string(" ").should_fail("not space")
>>> p.parse('A')
'A'
>>> p.parse('A ')
ParseError: expected 'not space' at 0:1

join(other: Parser[OUT2]) → Parser[Tuple[OUT1, OUT2]][source]#: An alternative name for the & parser (Parser.__and__).

as_tuple() → Parser[Tuple[OUT]][source]#

Modify the result of a parser, wrapping it in a tuple.

Several combinators work on parsers which return tuples, like append and combine, so this parser can be used before calling other parsers which expect a tuple result type.

>>> parser = string('x').as_tuple()
>>> parser.parse('x')
('x',)
>>> parser.append(string('y')).parse('xy')
('x', 'y')

append(other: Parser[OUT2]) → Parser[Tuple[Unpack[OUT_T], OUT2]][source]#

Take a parser whose result is a tuple, and add the result of other to the end of that tuple.

>>> parser = string('x').as_tuple()
>>> parser.parse('x')
('x',)
>>> parser.append(string('y')).parse('xy')
('x', 'y')

combine(combine_fn: Callable[[Unpack[OUT_T]], OUT2]) → Parser[OUT2][source]#

Returns a parser that transforms the produced values of the initial parser with combine_fn, passing the arguments using *args syntax.

Where the current parser produces an iterable of values, this can be a more convenient way to combine them than map().

Example 1 - the argument order of our callable already matches:

>>> from datetime import date
>>> yyyymmdd = seq(regex(r'[0-9]{4}').map(int),
...                regex(r'[0-9]{2}').map(int),
...                regex(r'[0-9]{2}').map(int)).combine(date)
>>> yyyymmdd.parse('20140506')
datetime.date(2014, 5, 6)

generate(fn: Callable[[], Generator[Parser[Any], Any, OUT]]) → Parser[OUT][source]#: generate converts a generator function (one that uses the yield keyword) into a parser. The generator function must yield parsers. These parsers are applied successively and their results are sent back to the generator using the .send() protocol. The generator function should return the final result of the parsing. Alternatively it can return another parser, which is equivalent to applying it and returning its result.

index = <ts_lib_parsy.Parser object>#: A parser that consumes no input and always just returns the current stream index. This is normally useful when wanting to build more debugging information into parse failure error messages.

line_info_at(stream: str, index: int) → Tuple[int, int][source]#: Given text and an index, return the line and column indices, zero-indexed.

line_info = <ts_lib_parsy.Parser object>#: A parser that consumes no input and always just returns the current line information, a tuple of (line, column), zero-indexed, where lines are terminated by \n. This is normally useful when wanting to build more debugging information into parse failure error messages.

success(val: OUT) → Parser[OUT][source]#: Returns a parser that does not consume any of the stream, but produces val.

fail(expected: str) → Parser[None][source]#: Returns a parser that always fails with the provided error message.

string(s: str, transform: ~typing.Callable[[str], str] = <function noop>) → Parser[str][source]#

Returns a parser that expects the expected_string and produces that string value.

Optionally, a transform function can be passed, which will be used on both the expected string and tested string. This allows things like case insensitive matches to be done. This function must not change the length of the string (as determined by len). The returned value of the parser will always be expected_string in its un-transformed state.

>>> parser = string("Hello", transform=lambda s: s.upper())
>>> parser.parse("Hello")
'Hello'
>>> parser.parse("hello")
'Hello'
>>> parser.parse("HELLO")
'Hello'

regex(pattern: str | Pattern[str], *, flags: RegexFlag = re.RegexFlag(0), group: Literal[0] = 0) → Parser[str][source]#

regex(pattern: str | Pattern[str], *, flags: RegexFlag = re.RegexFlag(0), group: str | int) → Parser[str]

regex(pattern: str | Pattern[str], *, flags: RegexFlag = re.RegexFlag(0), group: Tuple[str | int]) → Parser[Tuple[str]]

regex(pattern: str | Pattern[str], *, flags: RegexFlag = re.RegexFlag(0), group: Tuple[str | int, str | int]) → Parser[Tuple[str, str]]

regex(pattern: str | Pattern[str], *, flags: RegexFlag = re.RegexFlag(0), group: Tuple[str | int, str | int, str | int]) → Parser[Tuple[str, str, str]]

Returns a parser that expects the given exp, and produces the matched string. exp can be a compiled regular expression, or a string which will be compiled with the given flags.

Optionally, accepts group, which is passed to re.Match.group to return the text from a capturing group in the regex instead of the entire match.

Using a regex parser for small building blocks, instead of building up parsers from primitives like string(), test_char() and Parser.times() combinators etc., can have several advantages, including:

It can be more succinct e.g. compare:

>>> (string('a') | string('b')).times(1, 4)
>>> regex(r'[ab]{1,4}')

It can return the entire matched string as a single item,
so you don’t need to use Parser.concat().
It can return a part of the matched string using a capturing group
from the regex, so you don’t need to split the string yourself.

You can use named or numbered groups, just like with re.Match.group. Tuples also work, and return the captured text from multiple groups.
>>> regex(r'([0-9]{4})-([0-9]{2})', group=1).parse('2020-03') '2020' >>> regex(r'(?P<year>[0-9]{4})-(?P<month>[0-9]{2})', group='month').parse('2020-03') '03' >>> regex(r'([0-9]{4})-([0-9]{2})', group=(1,2)).parse('2020-03') ('2020', '03')
It can be much faster.

seq(__parser_1: Parser[OUT1], __parser_2: Parser[OUT2], __parser_3: Parser[OUT3], __parser_4: Parser[OUT4], __parser_5: Parser[OUT5], __parser_6: Parser[OUT6]) → Parser[Tuple[OUT1, OUT2, OUT3, OUT4, OUT5, OUT6]][source]#

seq(__parser_1: Parser[OUT1], __parser_2: Parser[OUT2], __parser_3: Parser[OUT3], __parser_4: Parser[OUT4], __parser_5: Parser[OUT5]) → Parser[Tuple[OUT1, OUT2, OUT3, OUT4, OUT5]]

seq(__parser_1: Parser[OUT1], __parser_2: Parser[OUT2], __parser_3: Parser[OUT3], __parser_4: Parser[OUT4]) → Parser[Tuple[OUT1, OUT2, OUT3, OUT4]]

seq(__parser_1: Parser[OUT1], __parser_2: Parser[OUT2], __parser_3: Parser[OUT3]) → Parser[Tuple[OUT1, OUT2, OUT3]]

seq(__parser_1: Parser[OUT1], __parser_2: Parser[OUT2]) → Parser[Tuple[OUT1, OUT2]]

seq(__parser_1: Parser[OUT1]) → Parser[Tuple[OUT1]]

seq(*parsers: Parser[Any]) → Parser[Tuple[Any, ...]]

Creates a parser that applies a sequence of parsers in sequence and combines their results into a tuple.

>>> x_bottles_of_y_on_the_z =         ...    seq(regex(r"[0-9]+").map(int) << string(" bottles of "),
...        regex(r"\S+") << string(" on the "),
...        regex(r"\S+")
...        )
>>> x_bottles_of_y_on_the_z.parse("99 bottles of beer on the wall")
(99, 'beer', 'wall')

test_char(func: Callable[[str], bool], description: str) → Parser[str][source]#

Returns a parser that tests a single character with the callable func. If func returns True, the parse succeeds, otherwise the parse fails with the description description.

>>> ascii = test_char(lambda c: ord(c) < 128,
...                   'ascii character')
>>> ascii.parse('A')
'A'

match_char(char: str, description: str | None = None) → Parser[str][source]#

string_from(*strings: str, transform: ~typing.Callable[[str], str] = <function noop>) → Parser[str][source]#

Accepts a sequence of strings as positional arguments, and returns a parser that matches and returns one string from the list. The list is first sorted in descending length order, so that overlapping strings are handled correctly by checking the longest one first.

>>> string_from('y', 'yes').parse('yes')
'yes'

Optionally accepts transform, which is passed to string() (see the documentation there).

char_from(string: str) → Parser[str][source]#

Accepts a string and returns a parser that matches and returns one character from the string.

>>> char_from('abc').parse('a')
'a'

peek(parser: Parser[OUT]) → Parser[OUT][source]#: Returns a lookahead parser that parse the input stream without consuming chars.

any_char = <ts_lib_parsy.Parser object>#: A parser that matches any single character.

whitespace = <ts_lib_parsy.Parser object>#: A parser that matches and returns one or more whitespace characters.

letter = <ts_lib_parsy.Parser object>#: A parser that matches and returns a single letter, as defined by str.isalpha.

digit = <ts_lib_parsy.Parser object>#: A parser that matches and returns a single digit, as defined by str.isdigit. Note that this includes various unicode characters outside of the normal 0-9 range, such as ¹²³.

decimal_digit = <ts_lib_parsy.Parser object>#: A parser that matches and returns a single decimal digit, one of “0123456789”.

from_enum(enum_cls: type[~ts_lib_parsy.E], transform: ~typing.Callable[[str], str] = <function noop>) → Parser[E][source]#

Given a class that is an enum.Enum class, returns a parser that will parse the values (or the string representations of the values) and return the corresponding enum item.

>>> from enum import Enum
>>> class Pet(Enum):
...     CAT = "cat"
...     DOG = "dog"
>>> pet = from_enum(Pet)
>>> pet.parse("cat")
<Pet.CAT: 'cat'>

str is first run on the values (for the case of values that are integers etc.) to create the strings which are turned into parsers using string().

If transform is provided, it is passed to string() when creating the parser (allowing for things like case insensitive parsing).

parser_field(parser: Parser[OUT], *, init: bool = True, repr: bool = True, hash: bool | None = None, compare: bool = True, metadata: Mapping[Any, Any] | None = None) → OUT[source]#: A field descriptor for dataclass fields which adds a parser to the metadata of the field. This parser can later be used by dataclass_parser to convert the dataclass into a parser whose result is the dataclass.

class DataClassProtocol(*args, **kwargs)[source]#

Bases: Protocol

Protocol used to annotate a generic dataclass type.

dataclass_parser(datatype: Type[OUT_D]) → Parser[OUT_D][source]#: Take a dataclass with field descriptors defined using parser_field, and call each field’s parser sequentially. Take the results of those parsers and use them to create an instance of the datatype dataclass, resulting in a parser with a type of Parser[datatype].