Parse

Everything to do with parsing user input.

Use provided classes to construct parsers and add validation:

>>> # Parses a string that matches the given regex.
>>> ident = Str().regex(r'^[a-zA-Z_][a-zA-Z0-9_]*$')

>>> # Parses a non-empty list of strings.
>>> idents = List(ident).len_ge(1)

Pass a parser to other yuio functions:

>>> yuio.io.ask('List of modules to reformat', parser=idents) 

Or parse strings yourself:

>>> idents.parse('sys os enum dataclasses')
['sys', 'os', 'enum', 'dataclasses']

Build a parser from type hints:

>>> from_type_hint(list[int] | None)
Optional(List(Int))

Parser basics

All parsers are derived from the same base class Parser, which describes parsing API.

class yuio.parse.Parser

Base class for parsers.

abstract parse(value: str, /) → T_co

Parse user input, raise ParsingError on failure.

abstract parse_many(value: Sequence[str], /) → T_co

For collection parsers, parse and validate collection by parsing its items one-by-one.

Example:

>>> # Let's say we're parsing a set of ints.
>>> parser = Set(Int())
>>>
>>> # And the user enters collection items one-by-one.
>>> user_input = ['1', '2', '3']
>>>
>>> # We can parse collection from its items:
>>> parser.parse_many(user_input)
{1, 2, 3}

abstract supports_parse_many() → bool

Return True if this parser returns a collection and so supports parse_many().

abstract parse_config(value: object, /) → T_co

Parse value from a config, raise ParsingError on failure.

This method accepts python values that would result from parsing json, yaml, and similar formats.

Example:

>>> # Let's say we're parsing a set of ints.
>>> parser = Set(Int())

>>> # And we're loading it from json.
>>> import json
>>> user_config = json.loads('[1, 2, 3]')

>>> # We can process parsed json:
>>> parser.parse_config(user_config)
{1, 2, 3}

class yuio.parse.ParsingError

Raised when parsing or validation fails.

This exception is derived from both ValueError and argparse.ArgumentTypeError to ensure that error messages are displayed nicely with argparse, and handled correctly in other places.

Building a parser

Parser includes several convenience methods that make building complex verification easier.

class yuio.parse.Parser

bound(*, lower: Cmp | None = None, lower_inclusive: Cmp | None = None, upper: Cmp | None = None, upper_inclusive: Cmp | None = None) → Bound[Cmp]

Check that value is upper- or lower-bound by some constraints.

Example:

>>> # Int in range `0 < x <= 1`:
>>> Int().bound(lower=0, upper_inclusive=1)
Bound(Int, 0 < x <= 1)

Parameters:

• lower – set lower bound for value, so we require that value > lower. Can’t be given if lower_inclusive is also given.

• lower_inclusive – set lower bound for value, so we require that value >= lower. Can’t be given if lower is also given.

• upper – set upper bound for value, so we require that value < upper. Can’t be given if upper_inclusive is also given.

• upper_inclusive – set upper bound for value, so we require that value <= upper. Can’t be given if upper is also given.

gt(bound: Cmp, /) → Bound[Cmp]

Check that value is greater than the given bound.

See bound() for more info.

ge(bound: Cmp, /) → Bound[Cmp]

Check that value is greater than or equal to the given bound.

See bound() for more info.

lt(bound: Cmp, /) → Bound[Cmp]

Check that value is lesser than the given bound.

See bound() for more info.

le(bound: Cmp, /) → Bound[Cmp]

Check that value is lesser than or equal to the given bound.

See bound() for more info.

len_bound(*, lower: int | None = None, lower_inclusive: int | None = None, upper: int | None = None, upper_inclusive: int | None = None) → LenBound[Sz]

Check that length of a value is upper- or lower-bound by some constraints.

The signature is the same as of the bound() method.

Example:

>>> # List of up to five ints:
>>> List(Int()).len_bound(upper_inclusive=5)
LenBound(List(Int), len <= 5)

len_between(*args: int) → LenBound[Sz]

Check that length of the value is within the given range.

Example:

>>> # List of up to five ints:
>>> List(Int()).len_between(6)
LenBound(List(Int), len < 6)

>>> # List of one, two, or three ints:
>>> List(Int()).len_between(1, 4)
LenBound(List(Int), 1 <= len < 4)

See len_bound() for more info.

Parameters:

• lower – lower length bound, inclusive.

• upper – upper length bound, not inclusive.

len_gt(bound: int, /) → LenBound[Sz]

Check that length of the value is greater than the given bound.

See len_bound() for more info.

len_ge(bound: int, /) → LenBound[Sz]

Check that length of the value is greater than or equal to the given bound.

See len_bound() for more info.

len_lt(bound: int, /) → LenBound[Sz]

Check that length of the value is lesser than the given bound.

See len_bound() for more info.

len_le(bound: int, /) → LenBound[Sz]

Check that length of the value is lesser than or equal to the given bound.

See len_bound() for more info.

len_eq(bound: int, /) → LenBound[Sz]

Check that length of the value is equal to the given bound.

See len_bound() for more info.

one_of(values: Collection[T], /) → OneOf[T]

Check that the parsed value is one of the given set of values.

Example:

>>> # Accepts only strings 'A', 'B', or 'C':
>>> Str().one_of(['A', 'B', 'C'])
OneOf(Str)

Value parsers

class yuio.parse.Str(*modifiers: Callable[[str], str])

Parser for str values.

Applies modifiers to the value, in order they are given.

Example:

>>> parser = Str().strip().lower()
>>> parser.parse('  SOME STRING  ')
'some string'

lower() → Str

Return a parser that applies str.lower() to all parsed strings.

upper() → Str

Return a parser that applies str.upper() to all parsed strings.

strip(char: str | None = None, /) → Str

Return a parser that applies str.strip() to all parsed strings.

lstrip(char: str | None = None, /) → Str

Return a parser that applies str.lstrip() to all parsed strings.

rstrip(char: str | None = None, /) → Str

Return a parser that applies str.rstrip() to all parsed strings.

regex(regex: str | Pattern[str], /, group: int | str = 0) → Str

Return a parser that matches the parsed string with the given regular expression.

If regex has capturing groups, parser can return contents of a group.

class yuio.parse.Int

Parser for int values.

class yuio.parse.Float

Parser for float values.

class yuio.parse.Bool

Parser for bool values, such as ‘yes’ or ‘no’.

class yuio.parse.Enum(enum_type: Type[E], /, *, by_name: bool = False)

Parser for enums, as defined in the standard enum module.

class yuio.parse.Decimal

Parser for decimal.Decimal.

class yuio.parse.Fraction

Parser for fractions.Fraction.

class yuio.parse.List(inner: Parser[T], /, *, delimiter: str | None = None)

Parser for lists.

Will split a string by the given delimiter, and parse each item using a subparser.

Parameters:

• inner – inner parser that will be used to parse list items.

• delimiter – delimiter that will be passed to str.split().

class yuio.parse.Set(inner: Parser[T], /, *, delimiter: str | None = None)

Parser for sets.

Will split a string by the given delimiter, and parse each item using a subparser.

Parameters:

• inner – inner parser that will be used to parse set items.

• delimiter – delimiter that will be passed to str.split().

class yuio.parse.FrozenSet(inner: Parser[T], /, *, delimiter: str | None = None)

Parser for frozen sets.

Will split a string by the given delimiter, and parse each item using a subparser.

Parameters:

• inner – inner parser that will be used to parse set items.

• delimiter – delimiter that will be passed to str.split().

class yuio.parse.Dict(key: Parser[K], value: Parser[V], /, *, delimiter: str | None = None, pair_delimiter: str = ':')

Parser for dicts.

Will split a string by the given delimiter, and parse each item using a Tuple parser.

Parameters:

• key – inner parser that will be used to parse dict keys.

• value – inner parser that will be used to parse dict values.

• delimiter – delimiter that will be passed to str.split().

• pair_delimiter – delimiter that will be used to split key-value elements.

class yuio.parse.Tuple(*parsers: Parser[...], delimiter: Optional[str] = None)

Parser for tuples of fixed lengths.

Parameters:

• parsers – parsers for each tuple element.

• delimiter – delimiter that will be passed to str.split().

File path parsers

class yuio.parse.Path(extensions: str | Collection[str] | None = None)

Parse a file system path, return a pathlib.Path.

Parameters:

extensions – list of allowed file extensions.

class yuio.parse.NonExistentPath(extensions: str | Collection[str] | None = None)

Parse a file system path and verify that it doesn’t exist.

class yuio.parse.ExistingPath(extensions: str | Collection[str] | None = None)

Parse a file system path and verify that it exists.

class yuio.parse.File(extensions: str | Collection[str] | None = None)

Parse path to a file.

class yuio.parse.Dir

Parse path to a directory.

class yuio.parse.GitRepo

Parse path to a git repository.

This parser just checks that the given directory has a subdirectory named .git.

Validators

class yuio.parse.Bound(inner: Parser[Cmp], /, *, lower: Cmp | None = None, lower_inclusive: Cmp | None = None, upper: Cmp | None = None, upper_inclusive: Cmp | None = None)

Check that value is upper- or lower-bound by some constraints.

See Parser.bound() for more info.

class yuio.parse.LenBound(inner: Parser[Sz], /, *, lower: int | None = None, lower_inclusive: int | None = None, upper: int | None = None, upper_inclusive: int | None = None)

Check that length of a value is upper- or lower-bound by some constraints.

See Parser.len_bound() for more info.

class yuio.parse.OneOf(inner: Parser[T], values: Collection[T], /)

Check that the parsed value is one of the given set of values.

See Parser.one_of() for more info.

Functional parsers

class yuio.parse.Map(inner: Parser[U], fn: Callable[[U], T], /)

A wrapper that maps result of the given parser using the given function.

Example:

>>> # Run `Int` parser, then square the result.
>>> int_parser = Map(Int(), lambda x: x ** 2)
>>> int_parser.parse("8")
64

class yuio.parse.Apply(inner: Parser[T], fn: Callable[[T], None], /)

A wrapper that applies the given function to the result of a wrapped widget.

Example:

>>> # Run `Int` parser, then print its output before returning.
>>> print_output = Apply(Int(), print)
>>> result = print_output.parse("10")
10
>>> result
10

Deriving parsers from type hints

There is a way to automatically derive basic parsers from type hints (used by yuio.config):

yuio.parse.from_type_hint(ty: Any, /) → Parser[Any]

Create parser from a type hint.

Example:

>>> from_type_hint(list[int] | None)
Optional(List(Int))

To extend capabilities of the automatic converter, you can register your own types and parsers:

yuio.parse.register_type_hint_conversion(cb: Callable[[type, type | None, Tuple[object, ...]], Parser[object] | None] | None = None, /, *, uses_delim: bool = False) → Callable[[type, type | None, Tuple[object, ...]], Parser[object] | None] | Callable[[Callable[[type, type | None, Tuple[object, ...]], Parser[object] | None]], Callable[[type, type | None, Tuple[object, ...]], Parser[object] | None]]

Register a new converter from typehint to a parser.

This function takes a callback that accepts three positional arguments:

• a type hint,

• a type hint’s origin (as defined by typing.get_origin()),

• a type hint’s args (as defined by typing.get_args()).

The callback should return a parser if it can, or None otherwise.

All registered callbacks are tried in the same order as the were registered.

If uses_delim is True, callback can use suggest_delim_for_type_hint_conversion().

This function can be used as a decorator.

Example:

>>> @register_type_hint_conversion
... def my_type_conversion(ty, origin, args):
...     if ty is MyType:
...         return MyTypeParser()
...     else:
...         return None

>>> from_type_hint(MyType)
MyTypeParser

When implementing a callback, you might need to specify a delimiter for a collection parser. Use suggest_delim_for_type_hint_conversion():

yuio.parse.suggest_delim_for_type_hint_conversion() → str | None

Suggests a delimiter for use in type hint converters.

When creating a parser for a collection of items based on a type hint, it is important to use different delimiters for nested collections. This function can suggest such a delimiter based on the current type hint’s depth.

Example:

>>> @register_type_hint_conversion(uses_delim=True)
... def my_collection_conversion(ty, origin, args):
...     if origin is MyCollection:
...         return MyCollectionParser(
...             from_type_hint(args[0]),
...             delimiter=suggest_delim_for_type_hint_conversion(),
...         )
...     else:
...         return None

>>> parser = from_type_hint(MyCollection[MyCollection[str]])
>>> parser
MyCollectionParser(MyCollectionParser(Str))
>>> parser._delimiter is None
True
>>> parser._inner._delimiter == ","
True

Other parser methods

Parser defines some more methods and attributes. You don’t usually need because Yuio handles everything they do itself. However, you can still use them in case you need to.

class yuio.parse.Parser

Base class for parsers.

__wrapped_parser__: Parser[object] | None = None

An attribute for unwrapping parsers that validate or map results of other parsers.

abstract get_nargs() → Literal['-', '+', '*', '?'] | int | None

Generate nargs for argparse.

abstract describe() → str | None

Return a human-readable description of an expected input.

abstract describe_or_def() → str

Like describe(), but guaranteed to return something.

abstract describe_many() → str | Tuple[str, ...] | None

Return a human-readable description of a container element.

Used with parse_many().

abstract describe_many_or_def() → str | Tuple[str, ...]

Like describe_many(), but guaranteed to return something.

abstract describe_value(value: object, /) → str | None

Return a human-readable description of the given value.

Note that, since parser’s type parameter is covariant, this function is not guaranteed to receive a value of the same type that this parser produces. In this case, you can return None.

abstract describe_value_or_def(value: object, /) → str

Like describe_value(), but guaranteed to return something.

Note that, since parser’s type parameter is covariant, this function is not guaranteed to receive a value of the same type that this parser produces. In this case, you can return str(value) or "<empty>".

abstract completer() → Completer

Return a completer for values of this parser.

This function is used when assembling autocompletion functions for shells, and when reading values from user via yuio.io.ask().

abstract widget(default: object | ~typing.Literal[<missing>], default_description: str, /) → Literal[<missing>]]

Return a widget for reading values of this parser.

This function is used when reading values from user via yuio.io.ask().

The returned widget must produce values of type T. If default is given, and the user input is empty (or, in case of choice widgets, if the user chooses the default), the widget must produce the MISSING constant (not the default constant).

Validating parsers must wrap the widget they got from __wrapped_parser__() into Map or Apply in order to validate widget’s results.

Building your own parser

To implement your parser, you can subclass Parser and implement all abstract methods yourself.

We, however, recommend using the following classes to speed up the process and avoid common bugs:

class yuio.parse.ValueParser

Base implementation for a parser that returns a single value.

Implements all method, except for parse() and parse_config().

Example:

>>> class MyTypeParser(ValueParser[MyType]):
...     def parse(self, value: str, /) -> MyType:
...         return self.parse_config(value)
...
...     def parse_config(self, value: object, /) -> MyType:
...         if not isinstance(value, str):
...             raise ParsingError(f'expected a string, got {value!r}')
...         return MyType(value)

>>> MyTypeParser().parse('data')
MyType(value='data')

class yuio.parse.ValidatingParser(inner: Parser[T])

Base implementation for a parser that validates result of another parser.

This class wraps another parser and passes all method calls to it. All parsed values are additionally passed to _validate().

Example:

>>> class IsLower(ValidatingParser[str]):
...     def _validate(self, value: str, /):
...         if not value.islower():
...             raise ParsingError('value should be lowercase')

>>> IsLower(Str()).parse('Not lowercase!')
Traceback (most recent call last):
...
yuio.parse.ParsingError: value should be lowercase

__wrapped_parser__: Parser[T] = None

An attribute for unwrapping parsers that validate or map results of other parsers.

abstract _validate(value: T, /)

Implementation of value validation.

Should raise ParsingError if validation fails.

class yuio.parse.CollectionParser(inner: ~yuio.parse.Parser[~yuio.parse.T], ty: ~typing.Type[~yuio.parse.C], ctor: ~typing.Callable[[~typing.Iterable[~yuio.parse.T]], ~yuio.parse.C], /, *, iter: ~typing.Callable[[~yuio.parse.C], ~typing.Iterable[~yuio.parse.T]] = <built-in function iter>, config_type: ~typing.Type[~yuio.parse.C2] = <class 'list'>, config_type_iter: ~typing.Callable[[~yuio.parse.C2], ~typing.Iterable[~yuio.parse.T]] = <built-in function iter>, delimiter: str | None = None)

A base class for implementing collection parsing. It will split a string by the given delimiter, parse each item using a subparser, and then pass the result to the given constructor.

Parameters:

• inner – parser that will be used to parse collection items.

• ty – type of the collection that this parser returns.

• ctor – factory of instances of the collection that this parser returns. It should take an iterable of parsed items, and return a collection.

• iter – a function that is used to get an iterator from a collection. This defaults to iter(), but sometimes it may be different. For example, Dict is implemented as a collection of pairs, and its iter is dict.items().

• config_type – type of a collection that we expect to find when parsing a config. This will usually be a list.

• config_type_iter – a function that is used to get an iterator from a config value.

• delimiter – delimiter that will be passed to str.split().

The above parameters are exposed via protected attributes: self._inner, self._ty, etc.

For example, let’s implement a list parser that repeats each element twice:

>>> class DoubleList(CollectionParser[list[T], T], Generic[T]):
...     def __init__(self, inner: Parser[T], /, *, delimiter: str | None = None):
...         super().__init__(inner, list, self._ctor, delimiter=delimiter)
...
...     @staticmethod
...     def _ctor(values: Iterable[T]) -> list[T]:
...         return [x for value in values for x in [value, value]]

>>> DoubleList(Int()).parse('1 2 3 4')
[1, 1, 2, 2, 3, 3, 4, 4]

_allow_completing_duplicates: ClassVar[bool] = True

If set to false, autocompletion will not suggest item duplicates.