Coverage for yuio/config.py: 92%

1# Yuio project, MIT license.

3# https://github.com/taminomara/yuio/

5# You're free to copy this file to your project and edit it for your needs,

6# just keep this copyright line please :3

8"""

9This module provides a base class for configs that can be loaded from

10files, environment variables or command line arguments (via :mod:`yuio.app`).

12Derive your config from the :class:`Config` base class. Inside of its body,

13define config fields using type annotations, just like :mod:`dataclasses`:

15.. code-block:: python

17 class AppConfig(Config):

18 #: Trained model to execute.

19 model: pathlib.Path

21 #: Input data for the model.

22 data: pathlib.Path

24 #: Enable or disable gpu.

25 use_gpu: bool = True

27Then use config's constructors and the :meth:`~Config.update` method

28(or the ``|`` / ``|=`` operators) to load it from various sources::

30 # Load config from a file.

31 config = AppConfig.load_from_json_file("~/.my_app_cfg.json")

33 # Update config with values from env.

34 config |= AppConfig.load_from_env()

37Config base class

38-----------------

40.. autoclass:: Config

43Advanced field configuration

44----------------------------

46By default, :class:`Config` infers names for env variables and flags,

47appropriate parsers, and other things from field's name, type hint, and comments.

49If you need to override them, you can use :func:`yuio.app.field`

50and :func:`yuio.app.inline` functions (also available from :mod:`yuio.config`):

52.. code-block:: python

54 class AppConfig(Config):

55 model: pathlib.Path | None = field(

56 default=None,

57 help="trained model to execute",

58 )

61Nesting configs

62---------------

64You can nest configs to achieve modularity:

66.. code-block:: python

68 class ExecutorConfig(Config):

69 #: Number of threads to use.

70 threads: int

72 #: Enable or disable gpu.

73 use_gpu: bool = True

76 class AppConfig(Config):

77 #: Executor parameters.

78 executor: ExecutorConfig

80 #: Trained model to execute.

81 model: pathlib.Path

83To initialise a nested config, pass either an instance of if

84or a dict with its variables to the config's constructor:

86.. code-block:: python

88 # The following lines are equivalent:

89 config = AppConfig(executor=ExecutorConfig(threads=16))

90 config = AppConfig(executor={"threads": 16})

91 # ...although type checkers will complain about dict =(

94Parsing environment variables

95-----------------------------

97You can load config from environment through :meth:`~Config.load_from_env`.

99Names of environment variables are just capitalized field names.

100Use the :func:`yuio.app.field` function to override them:

101

102.. code-block:: python

103

104 class KillCmdConfig(Config):

105 # Will be loaded from `SIGNAL`.

106 signal: int

107

108 # Will be loaded from `PROCESS_ID`.

109 pid: int = field(env="PROCESS_ID")

110

111In nested configs, environment variable names are prefixed with name

112of a field that contains the nested config:

113

114.. code-block:: python

115

116 class BigConfig(Config):

117 # `kill_cmd.signal` will be loaded from `KILL_CMD_SIGNAL`.

118 kill_cmd: KillCmdConfig

119

120 # `kill_cmd_2.signal` will be loaded from `KILL_SIGNAL`.

121 kill_cmd_2: KillCmdConfig = field(env="KILL")

122

123 # `kill_cmd_3.signal` will be loaded from `SIGNAL`.

124 kill_cmd_3: KillCmdConfig = field(env="")

125

126You can also disable loading a field from an environment altogether:

127

128.. code-block:: python

129

130 class KillCmdConfig(Config):

131 # Will not be loaded from env.

132 pid: int = field(env=yuio.DISABLED)

133

134To prefix all variable names with some string, pass the `prefix` parameter

135to the :meth:`~Config.load_from_env` function:

136

137.. code-block:: python

138

139 # config.kill_cmd.field will be loaded

140 # from `MY_APP_KILL_CMD_SIGNAL`

141 config = BigConfig.load_from_env("MY_APP")

142

143

144Parsing config files

145--------------------

146

147You can load config from structured config files,

148such as `json`, `yaml` or `toml`:

149

150.. skip: next

151

152.. code-block:: python

153

154 class ExecutorConfig(Config):

155 threads: int

156 use_gpu: bool = True

157

158

159 class AppConfig(Config):

160 executor: ExecutorConfig

161 model: pathlib.Path

162

163

164 config = AppConfig.load_from_json_file("~/.my_app_cfg.json")

165

166In this example, contents of the above config would be:

167

168.. code-block:: json

169

170 {

171 "executor": {

172 "threads": 16,

173 "use_gpu": true

174 },

175 "model": "/path/to/model"

176 }

177

178Note that, unlike with environment variables,

179there is no way to inline nested configs.

180

181

182Merging configs

183---------------

184

185Configs are specially designed to be merge-able. The basic pattern is to create

186an empty config instance, then :meth:`~Config.update` it with every config source:

187

188.. skip: next

189

190.. code-block:: python

191

192 config = AppConfig()

193 config.update(AppConfig.load_from_json_file("~/.my_app_cfg.json"))

194 config.update(AppConfig.load_from_env())

195 # ...and so on.

196

197The :meth:`~Config.update` function ignores default values, and only overrides

198keys that were actually configured.

199

200If you need a more complex update behavior, you can add a merge function for a field:

201

202.. code-block:: python

203

204 class AppConfig(Config):

205 plugins: list[str] = field(

206 default=[],

207 merge=lambda left, right: [*left, *right],

208 )

209

210Here, whenever we :meth:`~Config.update` ``AppConfig``, ``plugins`` from both instances

211will be concatenated.

212

213.. warning::

214

215 Merge function shouldn't mutate its arguments.

216 It should produce a new value instead.

217

218.. warning::

219

220 Merge function will not be called for default value. It's advisable to keep the

221 default value empty, and add the actual default to the initial empty config:

222

223 .. skip: next

224

225 .. code-block:: python

226

227 config = AppConfig(plugins=["markdown", "rst"])

228 config.update(...)

229

230.. seealso::

231

232 See :func:`yuio.util.merge_dicts` helper that can medge nested dicts.

233

234

235Collections of configs

236----------------------

237

238When you use configs inside of other collections, the config will be parsed from JSON.

239This is mostly useful when loading configs from files.

240

241.. autoclass:: ConfigParser

242

243

244Re-imports

245----------

246

247.. function:: field

248 :no-index:

249

250 Alias of :obj:`yuio.app.field`

251

252.. function:: inline

253 :no-index:

254

255 Alias of :obj:`yuio.app.inline`

256

257.. function:: bool_option

258 :no-index:

259

260 Alias of :obj:`yuio.app.bool_option`

261

262.. function:: count_option

263 :no-index:

264

265 Alias of :obj:`yuio.app.count_option`

266

267.. function:: parse_many_option

268 :no-index:

269

270 Alias of :obj:`yuio.app.parse_many_option`

271

272.. function:: parse_one_option

273 :no-index:

274

275 Alias of :obj:`yuio.app.parse_one_option`

276

277.. function:: store_const_option

278 :no-index:

279

280 Alias of :obj:`yuio.app.store_const_option`

281

282.. function:: store_false_option

283 :no-index:

284

285 Alias of :obj:`yuio.app.store_false_option`

286

287.. function:: store_true_option

288 :no-index:

289

290 Alias of :obj:`yuio.app.store_true_option`

291

292

293.. function:: merge_dicts

294 :no-index:

295

296 Alias of :func:`yuio.util.merge_dicts`

297

298

299.. function:: merge_dicts_opt

300 :no-index:

301

302 Alias of :func:`yuio.util.merge_dicts_opt`

303

304.. type:: HelpGroup

305 :no-index:

306

307 Alias of :obj:`yuio.cli.HelpGroup`.

308

309.. type:: MutuallyExclusiveGroup

310 :no-index:

311

312 Alias of :obj:`yuio.cli.MutuallyExclusiveGroup`.

313

314.. type:: OptionCtor

315 :no-index:

316

317 Alias of :obj:`yuio.app.OptionCtor`.

318

319.. type:: OptionSettings

320 :no-index:

321

322 Alias of :obj:`yuio.app.OptionSettings`.

323

324.. data:: MISC_GROUP

325 :no-index:

326

327 Alias of :obj:`yuio.cli.MISC_GROUP`.

328

329.. data:: OPTS_GROUP

330 :no-index:

331

332 Alias of :obj:`yuio.cli.OPTS_GROUP`.

333

334.. data:: SUBCOMMANDS_GROUP

335 :no-index:

336

337 Alias of :obj:`yuio.cli.SUBCOMMANDS_GROUP`.

338

339"""

340

341from __future__ import annotations

342

343import copy

344import json

345import os

346import pathlib

347import textwrap

348import types

349import warnings

350from dataclasses import dataclass

351

352import yuio

353import yuio.cli

354import yuio.complete

355import yuio.json_schema

356import yuio.parse

357import yuio.string

358from yuio.cli import (

359 MISC_GROUP,

360 OPTS_GROUP,

361 SUBCOMMANDS_GROUP,

362 HelpGroup,

363 MutuallyExclusiveGroup,

364)

365from yuio.util import dedent as _dedent

366from yuio.util import find_docs as _find_docs

367from yuio.util import merge_dicts, merge_dicts_opt

368

369import yuio._typing_ext as _tx

370from typing import TYPE_CHECKING

371

372if TYPE_CHECKING:

373 import typing_extensions as _t

374else:

375 from yuio import _typing as _t

376

377__all__ = [

378 "MISC_GROUP",

379 "OPTS_GROUP",

380 "SUBCOMMANDS_GROUP",

381 "Config",

382 "ConfigParser",

383 "HelpGroup",

384 "MutuallyExclusiveGroup",

385 "OptionCtor",

386 "OptionSettings",

387 "bool_option",

388 "collect_option",

389 "count_option",

390 "field",

391 "inline",

392 "merge_dicts",

393 "merge_dicts_opt",

394 "parse_many_option",

395 "parse_one_option",

396 "positional",

397 "store_const_option",

398 "store_false_option",

399 "store_true_option",

400]

401

402T = _t.TypeVar("T")

403Cfg = _t.TypeVar("Cfg", bound="Config")

404

405

406@dataclass(frozen=True, slots=True)

407class _FieldSettings:

408 default: _t.Any

409 parser: yuio.parse.Parser[_t.Any] | None = None

410 env: str | yuio.Disabled | None = None

411 flags: str | list[str] | yuio.Positional | yuio.Disabled | None = None

412 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING

413 required: bool | None = None

414 merge: _t.Callable[[_t.Any, _t.Any], _t.Any] | None = None

415 mutex_group: MutuallyExclusiveGroup | None = None

416 option_ctor: _t.Callable[[OptionSettings], yuio.cli.Option[_t.Any]] | None = None

417 help: str | yuio.Disabled | None = None

418 help_group: HelpGroup | yuio.Collapse | None | yuio.Missing = yuio.MISSING

419 metavar: str | None = None

420 usage: yuio.Collapse | bool | None = None

421 default_desc: str | None = None

422 show_if_inherited: bool | None = None

423

424 def _update_defaults(

425 self,

426 qualname: str,

427 name: str,

428 ty_with_extras: _t.Any,

429 parsed_help: str | None,

430 allow_positionals: bool,

431 cut_help: bool,

432 ) -> _Field:

433 ty = ty_with_extras

434 while _t.get_origin(ty) is _t.Annotated:

435 ty = _t.get_args(ty)[0]

436 is_subconfig = isinstance(ty, type) and issubclass(ty, Config)

437

438 env: str | yuio.Disabled

439 if self.env is not None:

440 env = self.env

441 else:

442 env = name.upper()

443 if env == "" and not is_subconfig:

444 raise TypeError(f"{qualname} got an empty env variable name")

445

446 flags: list[str] | yuio.Positional | yuio.Disabled

447 if self.flags is yuio.DISABLED:

448 flags = self.flags

449 elif self.flags is yuio.POSITIONAL:

450 if not allow_positionals:

451 raise TypeError(

452 f"{qualname}: positional arguments are not allowed in configs"

453 )

454 if is_subconfig:

455 raise TypeError(

456 f"error in {qualname}: nested configs can't be positional"

457 )

458 flags = self.flags

459 elif self.flags is None:

460 flags = ["--" + name.replace("_", "-")]

461 else:

462 if isinstance(self.flags, str):

463 if "," in self.flags:

464 flags = [

465 norm_flag

466 for flag in self.flags.split(",")

467 if (norm_flag := flag.strip())

468 ]

469 else:

470 flags = self.flags.split()

471 if not flags:

472 flags = [""]

473 else:

474 flags = self.flags

475

476 if is_subconfig:

477 if not flags:

478 raise TypeError(

479 f"error in {qualname}: nested configs should have exactly one flag; "

480 "to disable prefixing, pass an empty string as a flag"

481 )

482 if len(flags) > 1:

483 raise TypeError(

484 f"error in {qualname}: nested configs can't have multiple flags"

485 )

486 if flags[0]:

487 if not flags[0].startswith("--"):

488 raise TypeError(

489 f"error in {qualname}: nested configs can't have a short flag"

490 )

491 try:

492 yuio.cli._check_flag(flags[0])

493 except TypeError as e:

494 raise TypeError(f"error in {qualname}: {e}") from None

495 else:

496 if not flags:

497 raise TypeError(f"{qualname} should have at least one flag")

498 for flag in flags:

499 try:

500 yuio.cli._check_flag(flag)

501 except TypeError as e:

502 raise TypeError(f"error in {qualname}: {e}") from None

503

504 default = self.default

505 if is_subconfig and default is not yuio.MISSING:

506 raise TypeError(f"error in {qualname}: nested configs can't have defaults")

507

508 parser = self.parser

509 if is_subconfig and parser is not None:

510 raise TypeError(f"error in {qualname}: nested configs can't have parsers")

511 elif not is_subconfig and parser is None:

512 try:

513 parser = yuio.parse.from_type_hint(ty_with_extras)

514 except TypeError as e:

515 raise TypeError(

516 f"can't derive parser for {qualname}:\n"

517 + textwrap.indent(str(e), " ")

518 ) from None

519 if parser is not None:

520 origin = _t.get_origin(ty)

521 args = _t.get_args(ty)

522 is_optional = (

523 default is None or _tx.is_union(origin) and types.NoneType in args

524 )

525 if is_optional and not yuio.parse._is_optional_parser(parser):

526 parser = yuio.parse.Optional(parser)

527 completer = self.completer

528 metavar = self.metavar

529 if not metavar and flags is yuio.POSITIONAL:

530 metavar = f"<{name.replace('_', '-')}>"

531 if completer is not None or metavar is not None:

532 parser = yuio.parse.WithMeta(parser, desc=metavar, completer=completer)

533

534 required = self.required

535 if is_subconfig and required:

536 raise TypeError(f"error in {qualname}: nested configs can't be required")

537 if required is None:

538 if is_subconfig:

539 required = False

540 elif allow_positionals:

541 required = default is yuio.MISSING

542 else:

543 required = False

544

545 merge = self.merge

546 if is_subconfig and merge is not None:

547 raise TypeError(

548 f"error in {qualname}: nested configs can't have merge function"

549 )

550

551 mutex_group = self.mutex_group

552 if is_subconfig and mutex_group is not None:

553 raise TypeError(

554 f"error in {qualname}: nested configs can't be a part "

555 "of a mutually exclusive group"

556 )

557 if flags is yuio.POSITIONAL and mutex_group is not None:

558 raise TypeError(

559 f"error in {qualname}: positional arguments can't appear in mutually exclusive groups"

560 )

561

562 option_ctor = self.option_ctor

563 if option_ctor is not None and is_subconfig:

564 raise TypeError(

565 f"error in {qualname}: nested configs can't have option constructors"

566 )

567

568 help: str | yuio.Disabled

569 if self.help is not None:

570 help = _dedent(self.help) if self.help is not yuio.DISABLED else self.help

571 full_help = help or ""

572 elif parsed_help is not None:

573 help = full_help = parsed_help # Already dedented by comment parser

574 if cut_help and (index := help.find("\n\n")) != -1:

575 help = help[:index]

576 else:

577 help = full_help = ""

578

579 help_group = self.help_group

580 if help_group is yuio.COLLAPSE and not is_subconfig:

581 raise TypeError(

582 f"error in {qualname}: help_group=yuio.COLLAPSE only allowed for nested configs"

583 )

584

585 usage = self.usage

586

587 default_desc = self.default_desc

588

589 show_if_inherited = self.show_if_inherited

590

591 return _Field(

592 name=name,

593 qualname=qualname,

594 default=default,

595 parser=parser,

596 env=env,

597 flags=flags,

598 is_subconfig=is_subconfig,

599 ty=ty,

600 required=required,

601 merge=merge,

602 mutex_group=mutex_group,

603 option_ctor=option_ctor,

604 help=help,

605 full_help=full_help,

606 help_group=help_group,

607 usage=usage,

608 default_desc=default_desc,

609 show_if_inherited=show_if_inherited,

610 )

611

612

613@dataclass(frozen=True, slots=True)

614class _Field:

615 name: str

616 qualname: str

617 default: _t.Any

618 parser: yuio.parse.Parser[_t.Any] | None

619 env: str | yuio.Disabled

620 flags: list[str] | yuio.Positional | yuio.Disabled

621 is_subconfig: bool

622 ty: type

623 required: bool

624 merge: _t.Callable[[_t.Any, _t.Any], _t.Any] | None

625 mutex_group: MutuallyExclusiveGroup | None

626 option_ctor: _t.Callable[[OptionSettings], yuio.cli.Option[_t.Any]] | None

627 help: str | yuio.Disabled

628 full_help: str

629 help_group: HelpGroup | yuio.Collapse | None | yuio.Missing

630 usage: yuio.Collapse | bool | None

631 default_desc: str | None

632 show_if_inherited: bool | None

633

634

635@_t.overload

636def field(

637 *,

638 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING,

639 env: str | yuio.Disabled | None = None,

640 flags: str | list[str] | yuio.Disabled | None = None,

641 required: bool | None = None,

642 mutex_group: MutuallyExclusiveGroup | None = None,

643 help: str | yuio.Disabled | None = None,

644 help_group: HelpGroup | yuio.Collapse | None | yuio.Missing = yuio.MISSING,

645 metavar: str | None = None,

646 usage: yuio.Collapse | bool | None = None,

647 default_desc: str | None = None,

648 show_if_inherited: bool | None = None,

649) -> _t.Any: ...

650@_t.overload

651def field(

652 *,

653 default: None,

654 parser: yuio.parse.Parser[T] | None = None,

655 env: str | yuio.Disabled | None = None,

656 flags: str | list[str] | yuio.Disabled | None = None,

657 required: bool | None = None,

658 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING,

659 merge: _t.Callable[[T, T], T] | None = None,

660 mutex_group: MutuallyExclusiveGroup | None = None,

661 option_ctor: OptionCtor[T] | None = None,

662 help: str | yuio.Disabled | None = None,

663 help_group: HelpGroup | yuio.Collapse | None | yuio.Missing = yuio.MISSING,

664 metavar: str | None = None,

665 usage: yuio.Collapse | bool | None = None,

666 default_desc: str | None = None,

667 show_if_inherited: bool | None = None,

668) -> T | None: ...

669@_t.overload

670def field(

671 *,

672 default: T | yuio.Missing = yuio.MISSING,

673 parser: yuio.parse.Parser[T] | None = None,

674 env: str | yuio.Disabled | None = None,

675 flags: str | list[str] | yuio.Disabled | None = None,

676 required: bool | None = None,

677 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING,

678 merge: _t.Callable[[T, T], T] | None = None,

679 mutex_group: MutuallyExclusiveGroup | None = None,

680 option_ctor: OptionCtor[T] | None = None,

681 help: str | yuio.Disabled | None = None,

682 help_group: HelpGroup | yuio.Collapse | None | yuio.Missing = yuio.MISSING,

683 metavar: str | None = None,

684 usage: yuio.Collapse | bool | None = None,

685 default_desc: str | None = None,

686 show_if_inherited: bool | None = None,

687) -> T: ...

688@_t.overload

689@_t.deprecated(

690 "prefer using positional-only function arguments instead",

691 category=yuio.YuioPendingDeprecationWarning,

692)

693def field(

694 *,

695 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING,

696 env: str | yuio.Disabled | None = None,

697 flags: yuio.Positional,

698 required: bool | None = None,

699 mutex_group: MutuallyExclusiveGroup | None = None,

700 help: str | yuio.Disabled | None = None,

701 help_group: HelpGroup | yuio.Collapse | None | yuio.Missing = yuio.MISSING,

702 metavar: str | None = None,

703 usage: yuio.Collapse | bool | None = None,

704 default_desc: str | None = None,

705 show_if_inherited: bool | None = None,

706) -> _t.Any: ...

707@_t.overload

708@_t.deprecated(

709 "passing flags=yuio.POSITIONAL is discouraged, "

710 "prefer using positional-only function arguments instead",

711 category=yuio.YuioPendingDeprecationWarning,

712)

713def field(

714 *,

715 default: None,

716 parser: yuio.parse.Parser[T] | None = None,

717 env: str | yuio.Disabled | None = None,

718 flags: yuio.Positional,

719 required: bool | None = None,

720 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING,

721 merge: _t.Callable[[T, T], T] | None = None,

722 mutex_group: MutuallyExclusiveGroup | None = None,

723 option_ctor: OptionCtor[T] | None = None,

724 help: str | yuio.Disabled | None = None,

725 help_group: HelpGroup | yuio.Collapse | None | yuio.Missing = yuio.MISSING,

726 metavar: str | None = None,

727 usage: yuio.Collapse | bool | None = None,

728 default_desc: str | None = None,

729 show_if_inherited: bool | None = None,

730) -> T | None: ...

731@_t.overload

732@_t.deprecated(

733 "passing flags=yuio.POSITIONAL is discouraged, "

734 "prefer using positional-only function arguments instead",

735 category=yuio.YuioPendingDeprecationWarning,

736)

737def field(

738 *,

739 default: T | yuio.Missing = yuio.MISSING,

740 parser: yuio.parse.Parser[T] | None = None,

741 env: str | yuio.Disabled | None = None,

742 flags: yuio.Positional,

743 required: bool | None = None,

744 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING,

745 merge: _t.Callable[[T, T], T] | None = None,

746 mutex_group: MutuallyExclusiveGroup | None = None,

747 option_ctor: OptionCtor[T] | None = None,

748 help: str | yuio.Disabled | None = None,

749 help_group: HelpGroup | yuio.Collapse | None | yuio.Missing = yuio.MISSING,

750 metavar: str | None = None,

751 usage: yuio.Collapse | bool | None = None,

752 default_desc: str | None = None,

753 show_if_inherited: bool | None = None,

754) -> T: ...

755def field(

756 *,

757 default: _t.Any = yuio.MISSING,

758 parser: yuio.parse.Parser[_t.Any] | None = None,

759 env: str | yuio.Disabled | None = None,

760 flags: str | list[str] | yuio.Positional | yuio.Disabled | None = None,

761 required: bool | None = None,

762 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING,

763 merge: _t.Callable[[_t.Any, _t.Any], _t.Any] | None = None,

764 mutex_group: MutuallyExclusiveGroup | None = None,

765 option_ctor: _t.Callable[..., _t.Any] | None = None,

766 help: str | yuio.Disabled | None = None,

767 help_group: HelpGroup | yuio.Collapse | None | yuio.Missing = yuio.MISSING,

768 metavar: str | None = None,

769 usage: yuio.Collapse | bool | None = None,

770 default_desc: str | None = None,

771 show_if_inherited: bool | None = None,

772) -> _t.Any:

773 """

774 Field descriptor, used for additional configuration of CLI options

775 and config fields.

776

777 :param default:

778 default value for the field or CLI option.

779 :param parser:

780 parser that will be used to parse config values and CLI options.

781 :param env:

782 specifies name of environment variable that will be used if loading config

783 from environment.

784

785 Pass :data:`~yuio.DISABLED` to disable loading this field form environment.

786

787 In sub-config fields, controls prefix for all environment variables within

788 this sub-config; pass an empty string to disable prefixing.

789 :param flags:

790 list of names (or a single name) of CLI flags that will be used for this field.

791

792 In configs, pass :data:`~yuio.DISABLED` to disable loading this field

793 form CLI arguments.

794

795 In sub-config fields, controls prefix for all flags withing this sub-config;

796 pass an empty string to disable prefixing.

797 :param completer:

798 completer that will be used for autocompletion in CLI. Using this option

799 is equivalent to overriding `completer` with :class:`yuio.parse.WithMeta`.

800 :param merge:

801 defines how values of this field are merged when configs are updated.

802 :param mutex_group:

803 defines mutually exclusive group for this field.

804 :param option_ctor:

805 this parameter is similar to :mod:`argparse`\\ 's ``action``: it allows

806 overriding logic for handling CLI arguments by providing a custom

807 :class:`~yuio.cli.Option` implementation.

808

809 `option_ctor` should be a callable which takes a single positional argument

810 of type :class:`~yuio.app.OptionSettings`, and returns an instance

811 of :class:`yuio.cli.Option`.

812 :param help:

813 help message that will be used in CLI option description,

814 formatted using RST or Markdown

815 (see :attr:`App.doc_format <yuio.app.App.doc_format>`).

816

817 Pass :data:`yuio.DISABLED` to remove this field from CLI help.

818 :param help_group:

819 overrides group in which this field will be placed when generating CLI help

820 message.

821

822 Pass :class:`yuio.COLLAPSE` to create a collapsed group.

823 :param metavar:

824 value description that will be used for CLI help messages. Using this option

825 is equivalent to overriding `desc` with :class:`yuio.parse.WithMeta`.

826 :param usage:

827 controls how this field renders in CLI usage section.

828

829 Pass :data:`False` to remove this field from usage.

830

831 Pass :class:`yuio.COLLAPSE` to omit this field and add a single string

832 ``<options>`` instead.

833

834 Setting `usage` on sub-config fields overrides default `usage` for all

835 fields within this sub-config.

836 :param default_desc:

837 overrides description for default value in CLI help message.

838

839 Pass an empty string to hide default value.

840 :param show_if_inherited:

841 for fields with flags, enables showing this field in CLI help message

842 for subcommands.

843 :returns:

844 a magic object that will be replaced with field's default value once a new

845 config class is created.

846 :example:

847 In apps:

848

849 .. invisible-code-block: python

850

851 import yuio.app

852

853 .. code-block:: python

854

855 @yuio.app.app

856 def main(

857 # Will be loaded from `--input`.

858 input: pathlib.Path | None = None,

859 # Will be loaded from `-o` or `--output`.

860 output: pathlib.Path | None = field(

861 default=None, flags=["-o", "--output"]

862 ),

863 ): ...

864

865 In configs:

866

867 .. code-block:: python

868

869 class AppConfig(Config):

870 model: pathlib.Path | None = field(

871 default=None,

872 help="trained model to execute",

873 )

874

875 """

876

877 if flags is yuio.POSITIONAL:

878 warnings.warn(

879 "passing flags=yuio.POSITIONAL is discouraged, "

880 "prefer using positional-only function arguments instead",

881 category=yuio.YuioPendingDeprecationWarning,

882 stacklevel=2,

883 )

884

885 return _FieldSettings(

886 default=default,

887 parser=parser,

888 env=env,

889 flags=flags,

890 completer=completer,

891 required=required,

892 merge=merge,

893 mutex_group=mutex_group,

894 option_ctor=option_ctor,

895 help=help,

896 help_group=help_group,

897 metavar=metavar,

898 usage=usage,

899 default_desc=default_desc,

900 show_if_inherited=show_if_inherited,

901 )

902

903

904def inline(

905 help: str | yuio.Disabled | None = None,

906 help_group: HelpGroup | yuio.Collapse | None | yuio.Missing = yuio.MISSING,

907 usage: yuio.Collapse | bool | None = None,

908 show_if_inherited: bool | None = None,

909) -> _t.Any:

910 """

911 A shortcut for inlining nested configs.

912

913 Equivalent to calling :func:`~yuio.app.field` with `env` and `flags`

914 set to an empty string.

915

916 """

917

918 return field(

919 env="",

920 flags="",

921 help=help,

922 help_group=help_group,

923 usage=usage,

924 show_if_inherited=show_if_inherited,

925 )

926

927

928@_t.overload

929def positional(

930 *,

931 env: str | yuio.Disabled | None = None,

932 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING,

933 help: str | yuio.Disabled | None = None,

934 metavar: str | None = None,

935 usage: yuio.Collapse | bool | None = None,

936 default_desc: str | None = None,

937) -> _t.Any: ...

938@_t.overload

939def positional(

940 *,

941 default: None,

942 parser: yuio.parse.Parser[T] | None = None,

943 env: str | yuio.Disabled | None = None,

944 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING,

945 help: str | yuio.Disabled | None = None,

946 metavar: str | None = None,

947 usage: yuio.Collapse | bool | None = None,

948 default_desc: str | None = None,

949) -> T | None: ...

950@_t.overload

951def positional(

952 *,

953 default: T | yuio.Missing = yuio.MISSING,

954 parser: yuio.parse.Parser[T] | None = None,

955 env: str | yuio.Disabled | None = None,

956 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING,

957 help: str | yuio.Disabled | None = None,

958 metavar: str | None = None,

959 usage: yuio.Collapse | bool | None = None,

960 default_desc: str | None = None,

961) -> T: ...

962

963

964@_t.deprecated(

965 "prefer using positional-only function arguments instead",

966 category=yuio.YuioPendingDeprecationWarning,

967)

968def positional(

969 *,

970 default: _t.Any = yuio.MISSING,

971 parser: yuio.parse.Parser[_t.Any] | None = None,

972 env: str | yuio.Disabled | None = None,

973 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING,

974 help: str | yuio.Disabled | None = None,

975 metavar: str | None = None,

976 usage: yuio.Collapse | bool | None = None,

977 default_desc: str | None = None,

978) -> _t.Any:

979 """

980 A shortcut for adding a positional argument.

981

982 Equivalent to calling :func:`field` with `flags` set to :data:`~yuio.POSITIONAL`.

983

984 """

985

986 return _FieldSettings(

987 default=default,

988 parser=parser,

989 env=env,

990 flags=yuio.POSITIONAL,

991 completer=completer,

992 help=help,

993 metavar=metavar,

994 usage=usage,

995 default_desc=default_desc,

996 )

997

998

999@_t.dataclass_transform(

1000 eq_default=False,

1001 order_default=False,

1002 kw_only_default=True,

1003 frozen_default=False,

1004 field_specifiers=(field, inline, positional),

1005)

1006class Config:

1007 """

1008 Base class for configs.

1009

1010 Pass keyword args to set fields, or pass another config to copy it::

1011

1012 Config(config1, config2, ..., field1=value1, ...)

1013

1014 Upon creation, all fields that aren't explicitly initialized

1015 and don't have defaults are considered missing.

1016 Accessing them will raise :class:`AttributeError`.

1017

1018 .. note::

1019

1020 Unlike dataclasses, Yuio does not provide an option to create new instances

1021 of default values upon config instantiation. This is done so that default

1022 values don't override non-default ones when you update one config from another.

1023

1024 .. automethod:: update

1025

1026 .. automethod:: __or__

1027

1028 .. automethod:: __ior__

1029

1030 .. automethod:: load_from_env

1031

1032 .. automethod:: load_from_json_file

1033

1034 .. automethod:: load_from_yaml_file

1035

1036 .. automethod:: load_from_toml_file

1037

1038 .. automethod:: load_from_parsed_file

1039

1040 .. automethod:: to_json_schema

1041

1042 .. automethod:: to_json_value

1043

1044 """

1045

1046 @classmethod

1047 def __get_fields(cls) -> dict[str, _Field]:

1048 if cls.__fields is not None:

1049 return cls.__fields

1050

1051 docs = getattr(cls, "__yuio_pre_parsed_docs__", None)

1052 if docs is None:

1053 try:

1054 docs = _find_docs(cls)

1055 except Exception:

1056 yuio._logger.warning(

1057 "unable to get documentation for class %s.%s",

1058 cls.__module__,

1059 cls.__qualname__,

1060 )

1061 docs = {}

1062

1063 fields = {}

1064

1065 for base in reversed(cls.__mro__):

1066 if base is not cls and hasattr(base, "_Config__get_fields"):

1067 fields.update(getattr(base, "_Config__get_fields")())

1068

1069 try:

1070 types = _t.get_type_hints(cls, include_extras=True)

1071 except NameError as e:

1072 if "<locals>" in cls.__qualname__:

1073 raise NameError(

1074 f"{e}. "

1075 f"Note: forward references do not work inside functions "

1076 f"(see https://github.com/python/typing/issues/797)"

1077 ) from None

1078 raise # pragma: no cover

1079

1080 cut_help = getattr(cls, "__yuio_short_help__", False)

1081

1082 for name, field in cls.__gathered_fields.items():

1083 if not isinstance(field, _FieldSettings):

1084 field = _FieldSettings(default=field)

1085

1086 fields[name] = field._update_defaults(

1087 f"{cls.__qualname__}.{name}",

1088 name,

1089 types[name],

1090 docs.get(name),

1091 cls.__allow_positionals,

1092 cut_help,

1093 )

1094 cls.__fields = fields

1095

1096 return fields

1097

1098 def __init_subclass__(cls, _allow_positionals=None, **kwargs):

1099 super().__init_subclass__(**kwargs)

1100

1101 if _allow_positionals is not None:

1102 cls.__allow_positionals: bool = _allow_positionals

1103 cls.__fields: dict[str, _Field] | None = None

1104

1105 cls.__gathered_fields: dict[str, _FieldSettings | _t.Any] = {}

1106 for name in cls.__annotations__:

1107 if not name.startswith("_"):

1108 cls.__gathered_fields[name] = cls.__dict__.get(name, yuio.MISSING)

1109 for name, value in cls.__dict__.items():

1110 if isinstance(value, _FieldSettings) and name not in cls.__gathered_fields:

1111 qualname = f"{cls.__qualname__}.{name}"

1112 if name.startswith("_"):

1113 raise TypeError(f"error in {qualname}: fields can't be private")

1114 else:

1115 raise TypeError(

1116 f"error in {qualname}: field without annotations is not allowed"

1117 )

1118 for name, value in cls.__gathered_fields.items():

1119 if isinstance(value, _FieldSettings):

1120 value = value.default

1121 setattr(cls, name, value)

1122

1123 def __init__(self, *args: _t.Self | dict[str, _t.Any], **kwargs):

1124 for name, field in self.__get_fields().items():

1125 if field.is_subconfig:

1126 setattr(self, name, field.ty())

1127

1128 for arg in args:

1129 self.update(arg)

1130

1131 self.update(kwargs)

1132

1133 def update(self, other: _t.Self | dict[str, _t.Any], /):

1134 """

1135 Update fields in this config with fields from another config.

1136

1137 This function is similar to :meth:`dict.update`.

1138

1139 Nested configs are updated recursively.

1140

1141 :param other:

1142 data for update.

1143

1144 """

1145

1146 if not other:

1147 return

1148

1149 if isinstance(other, Config):

1150 if (

1151 self.__class__ not in other.__class__.__mro__

1152 and other.__class__ not in self.__class__.__mro__

1153 ):

1154 raise TypeError("updating from an incompatible config")

1155 ns = other.__dict__

1156 elif isinstance(other, dict):

1157 ns = other

1158 for name in ns:

1159 if name not in self.__get_fields():

1160 raise TypeError(f"unknown field: {name}")

1161 else:

1162 raise TypeError("expected a dict or a config class")

1163

1164 for name, field in self.__get_fields().items():

1165 if name in ns:

1166 if field.is_subconfig:

1167 getattr(self, name).update(ns[name])

1168 elif ns[name] is not yuio.MISSING:

1169 if field.merge is not None and name in self.__dict__:

1170 setattr(self, name, field.merge(getattr(self, name), ns[name]))

1171 else:

1172 setattr(self, name, ns[name])

1173

1174 @classmethod

1175 def load_from_env(cls, prefix: str = "") -> _t.Self:

1176 """

1177 Load config from environment variables.

1178

1179 :param prefix:

1180 if given, names of all environment variables will be prefixed with

1181 this string and an underscore.

1182 :returns:

1183 a parsed config.

1184 :raises:

1185 :class:`~yuio.parse.ParsingError`.

1186

1187 """

1188

1189 return cls.__load_from_env(prefix)

1190

1191 @classmethod

1192 def __load_from_env(cls, prefix: str = "") -> _t.Self:

1193 fields = {}

1194

1195 for name, field in cls.__get_fields().items():

1196 if field.env is yuio.DISABLED:

1197 continue

1198

1199 if prefix and field.env:

1200 env = f"{prefix}_{field.env}"

1201 else:

1202 env = f"{prefix}{field.env}"

1203

1204 if field.is_subconfig:

1205 fields[name] = field.ty.load_from_env(prefix=env)

1206 elif env in os.environ:

1207 assert field.parser is not None

1208 try:

1209 fields[name] = field.parser.parse(os.environ[env])

1210 except yuio.parse.ParsingError as e:

1211 raise yuio.parse.ParsingError(

1212 "Can't parse environment variable `%s`:\n%s",

1213 env,

1214 yuio.string.Indent(e),

1215 ) from None

1216

1217 return cls(**fields)

1218

1219 @classmethod

1220 def _build_options(cls):

1221 return cls.__build_options("", "", None, True, False)

1222

1223 @classmethod

1224 def __build_options(

1225 cls,

1226 prefix: str,

1227 dest_prefix: str,

1228 help_group: yuio.cli.HelpGroup | None,

1229 usage: yuio.Collapse | bool,

1230 show_if_inherited: bool,

1231 ) -> list[yuio.cli.Option[_t.Any]]:

1232 options: list[yuio.cli.Option[_t.Any]] = []

1233

1234 if prefix:

1235 prefix += "-"

1236

1237 for name, field in cls.__get_fields().items():

1238 if field.flags is yuio.DISABLED:

1239 continue

1240

1241 dest = dest_prefix + name

1242

1243 flags: list[str] | yuio.Positional

1244 if prefix and field.flags is not yuio.POSITIONAL:

1245 flags = [prefix + flag.lstrip("-") for flag in field.flags]

1246 else:

1247 flags = field.flags

1248

1249 field_usage = field.usage

1250 if field_usage is None:

1251 field_usage = usage

1252

1253 field_show_if_inherited = field.show_if_inherited

1254 if field_show_if_inherited is None:

1255 field_show_if_inherited = show_if_inherited

1256

1257 if field.is_subconfig:

1258 assert flags is not yuio.POSITIONAL

1259 assert issubclass(field.ty, Config)

1260 if field.help is yuio.DISABLED:

1261 subgroup = yuio.cli.HelpGroup("", help=yuio.DISABLED)

1262 elif field.help_group is yuio.MISSING:

1263 if field.full_help:

1264 lines = field.full_help.split("\n\n", 1)

1265 title = lines[0].replace("\n", " ").rstrip(".").strip() or name

1266 help = lines[1] if len(lines) > 1 else ""

1267 subgroup = yuio.cli.HelpGroup(title=title, help=help)

1268 else:

1269 subgroup = help_group

1270 elif field.help_group is yuio.COLLAPSE:

1271 if field.full_help:

1272 lines = field.full_help.split("\n\n", 1)

1273 title = lines[0].replace("\n", " ").rstrip(".").strip() or name

1274 help = lines[1] if len(lines) > 1 else ""

1275 subgroup = yuio.cli.HelpGroup(title=title, help=help)

1276 else:

1277 subgroup = yuio.cli.HelpGroup(title=field.name)

1278 subgroup.collapse = True

1279 subgroup._slug = field.name

1280 else:

1281 subgroup = field.help_group

1282 options.extend(

1283 field.ty.__build_options(

1284 flags[0],

1285 dest + ".",

1286 subgroup,

1287 field_usage,

1288 field_show_if_inherited,

1289 )

1290 )

1291 continue

1292

1293 assert field.parser is not None

1294

1295 option_ctor = field.option_ctor or _default_option

1296 option = option_ctor(

1297 OptionSettings(

1298 name=name,

1299 qualname=field.qualname,

1300 parser=field.parser,

1301 flags=flags,

1302 required=field.required,

1303 mutex_group=field.mutex_group,

1304 usage=field_usage,

1305 help=field.help,

1306 help_group=field.help_group if field.help_group else help_group,

1307 show_if_inherited=field_show_if_inherited,

1308 merge=field.merge,

1309 dest=dest,

1310 default=field.default,

1311 default_desc=field.default_desc,

1312 long_flag_prefix=prefix or "--",

1313 )

1314 )

1315 options.append(option)

1316

1317 return options

1318

1319 def __getattribute(self, item):

1320 value = super().__getattribute__(item)

1321 if value is yuio.MISSING:

1322 raise AttributeError(f"{item} is not configured")

1323 else:

1324 return value

1325

1326 # A dirty hack to hide `__getattribute__` from type checkers.

1327 locals()["__getattribute__"] = __getattribute

1328

1329 def __repr__(self):

1330 field_reprs = ", ".join(

1331 f"{name}={getattr(self, name, yuio.MISSING)!r}"

1332 for name in self.__get_fields()

1333 )

1334 return f"{self.__class__.__name__}({field_reprs})"

1335

1336 def __rich_repr__(self):

1337 for name in self.__get_fields():

1338 yield name, getattr(self, name, yuio.MISSING)

1339

1340 def __copy__(self):

1341 return type(self)(self)

1342

1343 def __deepcopy__(self, memo: dict[int, _t.Any] | None = None):

1344 return type(self)(copy.deepcopy(self.__dict__, memo))

1345

1346 def __ior__(self, value: _t.Self, /) -> _t.Self:

1347 """

1348 Update this config in-place using the ``|=`` operator.

1349

1350 Equivalent to calling :meth:`update`::

1351

1352 config |= other

1353 # same as

1354 config.update(other)

1355

1356 :param value:

1357 config to merge from.

1358

1359 """

1360

1361 self.update(value)

1362 return self

1363

1364 def __or__(self, value: _t.Self, /) -> _t.Self:

1365 """

1366 Merge two configs using the ``|`` operator, returning a new config.

1367

1368 Creates a copy of this config, updates it with *value*,

1369 and returns the result. Neither operand is modified::

1370

1371 merged = config1 | config2

1372

1373 :param value:

1374 config to merge from.

1375 :returns:

1376 a new config with fields from both operands.

1377

1378 """

1379

1380 lhs = self.__class__(self)

1381 lhs.update(value)

1382 return lhs

1383

1384 @classmethod

1385 def load_from_json_file(

1386 cls,

1387 path: str | pathlib.Path,

1388 /,

1389 *,

1390 ignore_unknown_fields: bool = False,

1391 ignore_missing_file: bool = False,

1392 ) -> _t.Self:

1393 """

1394 Load config from a ``.json`` file.

1395

1396 :param path:

1397 path of the config file.

1398 :param ignore_unknown_fields:

1399 if :data:`True`, this method will ignore fields that aren't listed

1400 in config class.

1401 :param ignore_missing_file:

1402 if :data:`True`, silently ignore a missing file error. This is useful

1403 when loading a config from a home directory.

1404 :returns:

1405 a parsed config.

1406 :raises:

1407 :class:`~yuio.parse.ParsingError` if config parsing has failed

1408 or if config file doesn't exist.

1409

1410 """

1411

1412 return cls.__load_from_file(

1413 path, json.loads, ignore_unknown_fields, ignore_missing_file

1414 )

1415

1416 @classmethod

1417 def load_from_yaml_file(

1418 cls,

1419 path: str | pathlib.Path,

1420 /,

1421 *,

1422 ignore_unknown_fields: bool = False,

1423 ignore_missing_file: bool = False,

1424 ) -> _t.Self:

1425 """

1426 Load config from a ``.yaml`` file.

1427

1428 This requires `PyYaml <https://pypi.org/project/PyYAML/>`__ package

1429 to be installed.

1430

1431 :param path:

1432 path of the config file.

1433 :param ignore_unknown_fields:

1434 if :data:`True`, this method will ignore fields that aren't listed

1435 in config class.

1436 :param ignore_missing_file:

1437 if :data:`True`, silently ignore a missing file error. This is useful

1438 when loading a config from a home directory.

1439 :returns:

1440 a parsed config.

1441 :raises:

1442 :class:`~yuio.parse.ParsingError` if config parsing has failed

1443 or if config file doesn't exist. Can raise :class:`ImportError`

1444 if ``PyYaml`` is not available.

1445

1446 """

1447

1448 try:

1449 import yaml

1450 except ImportError:

1451 raise ImportError("PyYaml is not available")

1452

1453 return cls.__load_from_file(

1454 path, yaml.safe_load, ignore_unknown_fields, ignore_missing_file

1455 )

1456

1457 @classmethod

1458 def load_from_toml_file(

1459 cls,

1460 path: str | pathlib.Path,

1461 /,

1462 *,

1463 ignore_unknown_fields: bool = False,

1464 ignore_missing_file: bool = False,

1465 ) -> _t.Self:

1466 """

1467 Load config from a ``.toml`` file.

1468

1469 This requires

1470 `tomllib <https://docs.python.org/3/library/tomllib.html>`_ or

1471 `toml <https://pypi.org/project/toml/>`_ package

1472 to be installed.

1473

1474 :param path:

1475 path of the config file.

1476 :param ignore_unknown_fields:

1477 if :data:`True`, this method will ignore fields that aren't listed

1478 in config class.

1479 :param ignore_missing_file:

1480 if :data:`True`, silently ignore a missing file error. This is useful

1481 when loading a config from a home directory.

1482 :returns:

1483 a parsed config.

1484 :raises:

1485 :class:`~yuio.parse.ParsingError` if config parsing has failed

1486 or if config file doesn't exist. Can raise :class:`ImportError`

1487 if ``toml`` is not available.

1488

1489 """

1490

1491 try:

1492 import toml

1493 except ImportError:

1494 try:

1495 import tomllib as toml

1496 except ImportError:

1497 raise ImportError("toml is not available")

1498

1499 return cls.__load_from_file(

1500 path, toml.loads, ignore_unknown_fields, ignore_missing_file

1501 )

1502

1503 @classmethod

1504 def __load_from_file(

1505 cls,

1506 path: str | pathlib.Path,

1507 file_parser: _t.Callable[[str], _t.Any],

1508 ignore_unknown_fields: bool = False,

1509 ignore_missing_file: bool = False,

1510 ) -> _t.Self:

1511 path = pathlib.Path(path)

1512

1513 if ignore_missing_file and (not path.exists() or not path.is_file()):

1514 return cls()

1515

1516 try:

1517 loaded = file_parser(path.read_text())

1518 except Exception as e:

1519 raise yuio.parse.ParsingError(

1520 "Invalid config <c path>%s</c>:\n%s",

1521 path,

1522 yuio.string.Indent(e),

1523 ) from None

1524

1525 return cls.load_from_parsed_file(

1526 loaded, ignore_unknown_fields=ignore_unknown_fields, path=path

1527 )

1528

1529 @classmethod

1530 def load_from_parsed_file(

1531 cls,

1532 parsed: dict[str, object],

1533 /,

1534 *,

1535 ignore_unknown_fields: bool = False,

1536 path: str | pathlib.Path | None = None,

1537 ) -> _t.Self:

1538 """

1539 Load config from parsed config file.

1540

1541 This method takes a dict with arbitrary values that you'd get from

1542 parsing type-rich configs such as ``yaml`` or ``json``.

1543

1544 For example::

1545

1546 with open("conf.yaml") as file:

1547 config = Config.load_from_parsed_file(yaml.load(file))

1548

1549 :param parsed:

1550 data from parsed file.

1551 :param ignore_unknown_fields:

1552 if :data:`True`, this method will ignore fields that aren't listed

1553 in config class.

1554 :param path:

1555 path of the original file, used for error reporting.

1556 :returns:

1557 a parsed config.

1558 :raises:

1559 :class:`~yuio.parse.ParsingError`.

1560

1561 """

1562

1563 try:

1564 return cls.__load_from_parsed_file(

1565 yuio.parse.ConfigParsingContext(parsed), ignore_unknown_fields, ""

1566 )

1567 except yuio.parse.ParsingError as e:

1568 if path is None:

1569 raise

1570 else:

1571 raise yuio.parse.ParsingError(

1572 "Invalid config <c path>%s</c>:\n%s",

1573 path,

1574 yuio.string.Indent(e),

1575 ) from None

1576

1577 @classmethod

1578 def __load_from_parsed_file(

1579 cls,

1580 ctx: yuio.parse.ConfigParsingContext,

1581 ignore_unknown_fields: bool = False,

1582 field_prefix: str = "",

1583 ) -> _t.Self:

1584 value = ctx.value

1585

1586 if not isinstance(value, dict):

1587 raise yuio.parse.ParsingError.type_mismatch(value, dict, ctx=ctx)

1588

1589 fields = {}

1590

1591 if not ignore_unknown_fields:

1592 for name in value:

1593 if name not in cls.__get_fields() and name != "$schema":

1594 raise yuio.parse.ParsingError(

1595 "Unknown field `%s`", f"{field_prefix}{name}", ctx=ctx

1596 )

1597

1598 for name, field in cls.__get_fields().items():

1599 if name in value:

1600 if field.is_subconfig:

1601 fields[name] = field.ty.__load_from_parsed_file(

1602 ctx.descend(value[name], name),

1603 ignore_unknown_fields,

1604 field_prefix=name + ".",

1605 )

1606 else:

1607 assert field.parser is not None

1608 fields[name] = field.parser.parse_config_with_ctx(

1609 ctx.descend(value[name], name)

1610 )

1611

1612 return cls(**fields)

1613

1614 @classmethod

1615 def to_json_schema(

1616 cls, ctx: yuio.json_schema.JsonSchemaContext

1617 ) -> yuio.json_schema.JsonSchemaType:

1618 """

1619 Create a JSON schema object based on this config.

1620

1621 The purpose of this method is to make schemas for use in IDEs, i.e. to provide

1622 autocompletion or simple error checking. The returned schema is not guaranteed

1623 to reflect all constraints added to the parser.

1624

1625 :param ctx:

1626 context for building a schema.

1627 :returns:

1628 a JSON schema that describes structure of this config.

1629

1630 """

1631

1632 return ctx.add_type(cls, _tx.type_repr(cls), lambda: cls.__to_json_schema(ctx))

1633

1634 def to_json_value(

1635 self, *, include_defaults: bool = True

1636 ) -> yuio.json_schema.JsonValue:

1637 """

1638 Convert this config to a representation suitable for JSON serialization.

1639

1640 :param include_defaults:

1641 if :data:`False`, default values will be skipped.

1642 :returns:

1643 a config converted to JSON-serializable representation.

1644 :raises:

1645 :class:`TypeError` if any of the config fields contain values that can't

1646 be converted to JSON by their respective parsers.

1647

1648 """

1649

1650 data = {}

1651 for name, field in self.__get_fields().items():

1652 if not include_defaults and name not in self.__dict__:

1653 continue

1654 if field.is_subconfig:

1655 value = getattr(self, name).to_json_value(

1656 include_defaults=include_defaults

1657 )

1658 if value:

1659 data[name] = value

1660 else:

1661 assert field.parser

1662 try:

1663 value = getattr(self, name)

1664 except AttributeError:

1665 pass

1666 else:

1667 data[name] = field.parser.to_json_value(value)

1668 return data

1669

1670 @classmethod

1671 def __to_json_schema(

1672 cls, ctx: yuio.json_schema.JsonSchemaContext

1673 ) -> yuio.json_schema.JsonSchemaType:

1674 properties: dict[str, yuio.json_schema.JsonSchemaType] = {}

1675 defaults = {}

1676

1677 properties["$schema"] = yuio.json_schema.String()

1678

1679 for name, field in cls.__get_fields().items():

1680 if field.is_subconfig:

1681 properties[name] = field.ty.to_json_schema(ctx)

1682 else:

1683 assert field.parser

1684 field_schema = field.parser.to_json_schema(ctx)

1685 if field.help and field.help is not yuio.DISABLED:

1686 field_schema = yuio.json_schema.Meta(

1687 field_schema, description=field.help

1688 )

1689 properties[name] = field_schema

1690 if field.default is not yuio.MISSING:

1691 try:

1692 defaults[name] = field.parser.to_json_value(field.default)

1693 except TypeError:

1694 pass

1695

1696 return yuio.json_schema.Meta(

1697 yuio.json_schema.Object(properties),

1698 title=cls.__name__,

1699 description=_dedent(cls.__doc__) if cls.__doc__ else None,

1700 default=defaults,

1701 )

1702

1703

1704Config.__init_subclass__(_allow_positionals=False)

1705

1706

1707@dataclass(eq=False, kw_only=True)

1708class OptionSettings:

1709 """

1710 Settings for creating an :class:`~yuio.cli.Option` derived from field's type

1711 and configuration.

1712

1713 """

1714

1715 name: str | None

1716 """

1717 Name of config field or app parameter that caused creation of this option.

1718

1719 """

1720

1721 qualname: str | None

1722 """

1723 Fully qualified name of config field or app parameter that caused creation

1724 of this option. Useful for reporting errors.

1725

1726 """

1727

1728 default: _t.Any | yuio.Missing

1729 """

1730 See :attr:`yuio.cli.ValueOption.default`.

1731

1732 """

1733

1734 parser: yuio.parse.Parser[_t.Any]

1735 """

1736 Parser associated with this option.

1737

1738 """

1739

1740 flags: list[str] | yuio.Positional

1741 """

1742 See :attr:`yuio.cli.Option.flags`.

1743

1744 """

1745

1746 required: bool

1747 """

1748 See :attr:`yuio.cli.Option.required`.

1749

1750 """

1751

1752 merge: _t.Callable[[_t.Any, _t.Any], _t.Any] | None

1753 """

1754 See :attr:`yuio.cli.ValueOption.merge`.

1755

1756 """

1757

1758 mutex_group: None | MutuallyExclusiveGroup

1759 """

1760 See :attr:`yuio.cli.Option.mutex_group`.

1761

1762 """

1763

1764 dest: str

1765 """

1766 See :attr:`yuio.cli.Option.dest`. We don't provide any guarantees about `dest`\\ 's

1767 contents and recommend treating it as an opaque value.

1768

1769 """

1770

1771 help: str | yuio.Disabled

1772 """

1773 See :attr:`yuio.cli.Option.help`.

1774

1775 """

1776

1777 help_group: HelpGroup | None

1778 """

1779 See :attr:`yuio.cli.Option.help_group`.

1780

1781 """

1782

1783 usage: yuio.Collapse | bool

1784 """

1785 See :attr:`yuio.cli.Option.usage`.

1786

1787 """

1788

1789 default_desc: str | None

1790 """

1791 See :attr:`yuio.cli.Option.default_desc`.

1792

1793 """

1794

1795 show_if_inherited: bool

1796 """

1797 See :attr:`yuio.cli.Option.show_if_inherited`.

1798

1799 """

1800

1801 long_flag_prefix: str

1802 """

1803 This argument will contain prefix that was added to all :attr:`~OptionSettings.flags`.

1804 For apps and top level configs it will be ``"--"``, for nested configs it will

1805 include additional prefixes, for example ``"--nested-"``.

1806

1807 """

1808

1809

1810OptionCtor: _t.TypeAlias = _t.Callable[[OptionSettings], yuio.cli.Option[T]]

1811

1812

1813def _default_option(s: OptionSettings):

1814 if s.flags is not yuio.POSITIONAL and yuio.parse._is_bool_parser(s.parser):

1815 return bool_option()(s)

1816 elif s.parser.supports_parse_many():

1817 return parse_many_option()(s)

1818 else:

1819 return parse_one_option()(s)

1820

1821

1822def bool_option(*, neg_flags: list[str] | None = None) -> OptionCtor[bool]:

1823 """

1824 Factory for :class:`yuio.cli.BoolOption`.

1825

1826 :param neg_flags:

1827 additional set of flags that will set option's value to :data:`False`. If not

1828 given, a negative flag will be created by adding prefix ``no-`` to the first

1829 long flag of the option.

1830 :example:

1831 Boolean flag :flag:`--json` implicitly creates flag :flag:`--no-json`:

1832

1833 .. code-block:: python

1834 :emphasize-lines: 5

1835

1836 @yuio.app.app

1837 def main(

1838 json: bool = yuio.app.field(

1839 default=False,

1840 option_ctor=yuio.app.bool_option(),

1841 ),

1842 ): ...

1843

1844 Boolean flag :flag:`--json` with explicitly provided flag

1845 :flag:`--disable-json`:

1846

1847 .. code-block:: python

1848 :emphasize-lines: 5-7

1849

1850 @yuio.app.app

1851 def main(

1852 json: bool = yuio.app.field(

1853 default=False,

1854 option_ctor=yuio.app.bool_option(

1855 neg_flags=["--disable-json"],

1856 ),

1857 ),

1858 ): ...

1859

1860 """

1861

1862 def ctor(s: OptionSettings, /):

1863 if s.flags is yuio.POSITIONAL:

1864 raise TypeError(f"error in {s.qualname}: BoolOption can't be positional")

1865 if neg_flags is None:

1866 _neg_flags = []

1867 for flag in s.flags:

1868 if not yuio.cli._is_short(flag) and flag.startswith(s.long_flag_prefix):

1869 prefix = s.long_flag_prefix.strip("-")

1870 if prefix:

1871 prefix += "-"

1872 suffix = flag[len(s.long_flag_prefix) :].removeprefix("-")

1873 _neg_flags.append(f"--{prefix}no-{suffix}")

1874 break

1875 elif s.long_flag_prefix == "--":

1876 _neg_flags = neg_flags

1877 else:

1878 _neg_flags = []

1879 for flag in neg_flags:

1880 _neg_flags.append(s.long_flag_prefix + flag.lstrip("-"))

1881 return yuio.cli.BoolOption(

1882 pos_flags=s.flags,

1883 neg_flags=_neg_flags,

1884 required=s.required,

1885 mutex_group=s.mutex_group,

1886 usage=s.usage,

1887 help=s.help,

1888 help_group=s.help_group,

1889 show_if_inherited=s.show_if_inherited,

1890 dest=s.dest,

1891 parser=s.parser,

1892 merge=s.merge,

1893 default=s.default,

1894 default_desc=s.default_desc,

1895 )

1896

1897 return ctor

1898

1899

1900def parse_one_option() -> OptionCtor[_t.Any]:

1901 """

1902 Factory for :class:`yuio.cli.ParseOneOption`.

1903

1904 This option takes one argument and passes it

1905 to :meth:`Parser.parse() <yuio.parse.Parser.parse>`.

1906

1907 :example:

1908 Forcing a field which can use :func:`parse_many_option`

1909 to use :func:`parse_one_option` instead.

1910

1911 .. code-block:: python

1912 :emphasize-lines: 6

1913

1914 @yuio.app.app

1915 def main(

1916 files: list[str] = yuio.app.field(

1917 default=[],

1918 parser=yuio.parse.List(yuio.parse.Int(), delimiter=","),

1919 option_ctor=yuio.app.parse_one_option(),

1920 ),

1921 ): ...

1922

1923 This will disable multi-argument syntax:

1924

1925 .. code-block:: console

1926

1927 $ prog --files a.txt,b.txt # Ok

1928 $ prog --files a.txt b.txt # Error: `--files` takes one argument.

1929

1930 """

1931

1932 def ctor(s: OptionSettings, /):

1933 return yuio.cli.ParseOneOption(

1934 flags=s.flags,

1935 required=s.required,

1936 mutex_group=s.mutex_group,

1937 usage=s.usage,

1938 help=s.help,

1939 help_group=s.help_group,

1940 show_if_inherited=s.show_if_inherited,

1941 dest=s.dest,

1942 parser=s.parser,

1943 merge=s.merge,

1944 default=s.default,

1945 default_desc=s.default_desc,

1946 )

1947

1948 return ctor

1949

1950

1951def parse_many_option() -> OptionCtor[_t.Any]:

1952 """

1953 Factory for :class:`yuio.cli.ParseManyOption`.

1954

1955 This option takes multiple arguments and passes them

1956 to :meth:`Parser.parse_many() <yuio.parse.Parser.parse_many>`.

1957

1958 """

1959

1960 def ctor(s: OptionSettings, /):

1961 return yuio.cli.ParseManyOption(

1962 flags=s.flags,

1963 required=s.required,

1964 mutex_group=s.mutex_group,

1965 usage=s.usage,

1966 help=s.help,

1967 help_group=s.help_group,

1968 show_if_inherited=s.show_if_inherited,

1969 dest=s.dest,

1970 parser=s.parser,

1971 merge=s.merge,

1972 default=s.default,

1973 default_desc=s.default_desc,

1974 )

1975

1976 return ctor

1977

1978

1979def collect_option() -> OptionCtor[_t.Any]:

1980 """

1981 Factory for :class:`yuio.cli.ParseManyOption`.

1982

1983 This option takes single argument; it collects all arguments across all uses

1984 of this option, and passes them

1985 to :meth:`Parser.parse_many() <yuio.parse.Parser.parse_many>`.

1986

1987 :example:

1988 Forcing a field which can use :func:`parse_many_option`

1989 to collect arguments one-by-one.

1990

1991 .. code-block:: python

1992 :emphasize-lines: 5

1993

1994 @yuio.app.app

1995 def main(

1996 files: list[str] = yuio.app.field(

1997 default=[],

1998 option_ctor=yuio.app.collect_option(),

1999 flags="--file",

2000 ),

2001 ): ...

2002

2003 This will disable multi-argument syntax, but allow giving option multiple

2004 times without overriding previous value:

2005

2006 .. code-block:: console

2007

2008 $ prog --file a.txt --file b.txt # Ok

2009 $ prog --files a.txt b.txt # Error: `--file` takes one argument.

2010

2011 """

2012

2013 def ctor(s: OptionSettings, /):

2014 return yuio.cli.CollectOption(

2015 flags=s.flags,

2016 required=s.required,

2017 mutex_group=s.mutex_group,

2018 usage=s.usage,

2019 help=s.help,

2020 help_group=s.help_group,

2021 show_if_inherited=s.show_if_inherited,

2022 dest=s.dest,

2023 parser=s.parser,

2024 merge=s.merge,

2025 default=s.default,

2026 default_desc=s.default_desc,

2027 )

2028

2029 return ctor

2030

2031

2032def store_const_option(const: T) -> OptionCtor[T]:

2033 """

2034 Factory for :class:`yuio.cli.StoreConstOption`.

2035

2036 This options takes no arguments. When it's encountered amongst CLI arguments,

2037 it writes `const` to the resulting config.

2038

2039 """

2040

2041 def ctor(s: OptionSettings, /):

2042 if s.flags is yuio.POSITIONAL:

2043 raise TypeError(

2044 f"error in {s.qualname}: StoreConstOption can't be positional"

2045 )

2046

2047 return yuio.cli.StoreConstOption(

2048 flags=s.flags,

2049 required=s.required,

2050 mutex_group=s.mutex_group,

2051 usage=s.usage,

2052 help=s.help,

2053 help_group=s.help_group,

2054 show_if_inherited=s.show_if_inherited,

2055 dest=s.dest,

2056 merge=s.merge,

2057 default=s.default,

2058 default_desc=s.default_desc,

2059 const=const,

2060 )

2061

2062 return ctor

2063

2064

2065def count_option() -> OptionCtor[int]:

2066 """

2067 Factory for :class:`yuio.cli.CountOption`.

2068

2069 This option counts number of times it's encountered amongst CLI arguments.

2070

2071 Equivalent to using :func:`store_const_option` with ``const=1``

2072 and ``merge=lambda a, b: a + b``.

2073

2074 :example:

2075

2076 .. code-block:: python

2077

2078 @yuio.app.app

2079 def main(

2080 quiet: int = yuio.app.field(

2081 default=0,

2082 flags=["-q", "--quiet"],

2083 option_ctor=yuio.app.count_option(),

2084 ),

2085 ): ...

2086

2087 .. code-block:: console

2088

2089 prog -qq # quiet=2

2090

2091 """

2092

2093 def ctor(s: OptionSettings, /):

2094 if s.flags is yuio.POSITIONAL:

2095 raise TypeError(f"error in {s.qualname}: CountOption can't be positional")

2096

2097 return yuio.cli.CountOption(

2098 flags=s.flags,

2099 required=s.required,

2100 mutex_group=s.mutex_group,

2101 usage=s.usage,

2102 help=s.help,

2103 help_group=s.help_group,

2104 show_if_inherited=s.show_if_inherited,

2105 dest=s.dest,

2106 default=s.default,

2107 default_desc=s.default_desc,

2108 )

2109

2110 return ctor

2111

2112

2113def store_true_option() -> OptionCtor[bool]:

2114 """

2115 Factory for :class:`yuio.cli.StoreTrueOption`.

2116

2117 Equivalent to using :func:`store_const_option` with ``const=True``.

2118

2119 """

2120

2121 def ctor(s: OptionSettings, /):

2122 if s.flags is yuio.POSITIONAL:

2123 raise TypeError(

2124 f"error in {s.qualname}: StoreTrueOption can't be positional"

2125 )

2126

2127 return yuio.cli.StoreTrueOption(

2128 flags=s.flags,

2129 required=s.required,

2130 mutex_group=s.mutex_group,

2131 usage=s.usage,

2132 help=s.help,

2133 help_group=s.help_group,

2134 show_if_inherited=s.show_if_inherited,

2135 dest=s.dest,

2136 default=s.default,

2137 default_desc=s.default_desc,

2138 )

2139

2140 return ctor

2141

2142

2143def store_false_option() -> OptionCtor[bool]:

2144 """

2145 Factory for :class:`yuio.cli.StoreFalseOption`.

2146

2147 Equivalent to using :func:`store_const_option` with ``const=False``.

2148

2149 """

2150

2151 def ctor(s: OptionSettings, /):

2152 if s.flags is yuio.POSITIONAL:

2153 raise TypeError(

2154 f"error in {s.qualname}: StoreFalseOption can't be positional"

2155 )

2156

2157 return yuio.cli.StoreFalseOption(

2158 flags=s.flags,

2159 required=s.required,

2160 mutex_group=s.mutex_group,

2161 usage=s.usage,

2162 help=s.help,

2163 help_group=s.help_group,

2164 show_if_inherited=s.show_if_inherited,

2165 dest=s.dest,

2166 default=s.default,

2167 default_desc=s.default_desc,

2168 )

2169

2170 return ctor

2171

2172

2173class ConfigParser(

2174 yuio.parse.WrappingParser[Cfg, type[Cfg]],

2175 yuio.parse.ValueParser[Cfg],

2176 _t.Generic[Cfg],

2177):

2178 """

2179 Parser for configs that reads them as JSON strings.

2180

2181 This parser kicks in when you use configs as collection members, i.e. ``list[Config]``

2182 or ``dict[str, Config]``. On top level, the usual logic for nested configs applies.

2183

2184 """

2185

2186 if TYPE_CHECKING:

2187

2188 @_t.overload

2189 def __new__(cls, inner: type[Cfg], /) -> ConfigParser[Cfg]: ...

2190

2191 @_t.overload

2192 def __new__(cls, /) -> yuio.parse.PartialParser: ...

2193

2194 def __new__(cls, inner: type[Cfg] | None = None, /) -> _t.Any: ...

2195

2196 def __init__(

2197 self,

2198 inner: type[Cfg] | None = None,

2199 /,

2200 ):

2201 super().__init__(inner, inner)

2202

2203 def parse_with_ctx(self, ctx: yuio.parse.StrParsingContext, /) -> Cfg:

2204 ctx = ctx.strip_if_non_space()

2205 try:

2206 config_value: yuio.parse.JsonValue = json.loads(ctx.value)

2207 except json.JSONDecodeError as e:

2208 raise yuio.parse.ParsingError(

2209 "Can't parse `%r` as `JsonValue`:\n%s",

2210 ctx.value,

2211 yuio.string.Indent(e),

2212 ctx=ctx,

2213 fallback_msg="Can't parse value as `JsonValue`",

2214 ) from None

2215 try:

2216 return self.parse_config_with_ctx(

2217 yuio.parse.ConfigParsingContext(config_value)

2218 )

2219 except yuio.parse.ParsingError as e:

2220 raise yuio.parse.ParsingError(

2221 "Error in parsed json value:\n%s",

2222 yuio.string.Indent(e),

2223 ctx=ctx,

2224 fallback_msg="Error in parsed json value",

2225 ) from None

2226

2227 def parse_config_with_ctx(self, ctx: yuio.parse.ConfigParsingContext, /) -> Cfg:

2228 if not isinstance(ctx.value, dict):

2229 raise yuio.parse.ParsingError.type_mismatch(ctx.value, dict, ctx=ctx)

2230 for key, value in ctx.value.items():

2231 if not isinstance(key, str):

2232 raise yuio.parse.ParsingError.type_mismatch(

2233 key, str, ctx=ctx.descend(value, key)

2234 )

2235 return self._inner._Config__load_from_parsed_file(ctx) # pyright: ignore[reportAttributeAccessIssue]

2236

2237 def to_json_schema(

2238 self, ctx: yuio.json_schema.JsonSchemaContext, /

2239 ) -> yuio.json_schema.JsonSchemaType:

2240 return self._inner.to_json_schema(ctx)

2241

2242 def to_json_value(self, value: object, /) -> yuio.json_schema.JsonValue:

2243 assert self.assert_type(value)

2244 return value.to_json_value()

2245

2246 def __repr__(self):

2247 if self._inner_raw is not None:

2248 return f"{self.__class__.__name__}({self._inner_raw.__name__!r})"

2249 else:

2250 return super().__repr__()

2251

2252

2253yuio.parse.register_type_hint_conversion(

2254 lambda ty, origin, args: (

2255 ConfigParser(ty) if isinstance(ty, type) and issubclass(ty, Config) else None

2256 )

2257)

Coverage for yuio / config.py: 92%

548 statements