Coverage for yuio/config.py: 93%

1# Yuio project, MIT license.

3# https://github.com/taminomara/yuio/

5# You're free to copy this file to your project and edit it for your needs,

6# just keep this copyright line please :3

8"""

9This module provides a base class for configs that can be loaded from

10files, environment variables or command line arguments (via :mod:`yuio.app`).

12Derive your config from the :class:`Config` base class. Inside of its body,

13define config fields using type annotations, just like :mod:`dataclasses`:

15.. code-block:: python

17 class AppConfig(Config):

18 #: Trained model to execute.

19 model: pathlib.Path

21 #: Input data for the model.

22 data: pathlib.Path

24 #: Enable or disable gpu.

25 use_gpu: bool = True

27Then use config's constructors and the :meth:`~Config.update` method

28to load it from various sources::

30 # Load config from a file.

31 config = AppConfig.load_from_json_file("~/.my_app_cfg.json")

33 # Update config with values from env.

34 config.update(AppConfig.load_from_env())

37Config base class

38-----------------

40.. autoclass:: Config

43Advanced field configuration

44----------------------------

46By default, :class:`Config` infers names for env variables and flags,

47appropriate parsers, and other things from field's name, type hint, and comments.

49If you need to override them, you can use :func:`yuio.app.field`

50and :func:`yuio.app.inline` functions (also available from :mod:`yuio.config`):

52.. code-block:: python

54 class AppConfig(Config):

55 model: pathlib.Path | None = field(

56 default=None,

57 help="trained model to execute",

58 )

61Nesting configs

62---------------

64You can nest configs to achieve modularity:

66.. code-block:: python

68 class ExecutorConfig(Config):

69 #: Number of threads to use.

70 threads: int

72 #: Enable or disable gpu.

73 use_gpu: bool = True

76 class AppConfig(Config):

77 #: Executor parameters.

78 executor: ExecutorConfig

80 #: Trained model to execute.

81 model: pathlib.Path

83To initialise a nested config, pass either an instance of if

84or a dict with its variables to the config's constructor:

86.. code-block:: python

88 # The following lines are equivalent:

89 config = AppConfig(executor=ExecutorConfig(threads=16))

90 config = AppConfig(executor={"threads": 16})

91 # ...although type checkers will complain about dict =(

94Parsing environment variables

95-----------------------------

97You can load config from environment through :meth:`~Config.load_from_env`.

99Names of environment variables are just capitalized field names.

100Use the :func:`yuio.app.field` function to override them:

101

102.. code-block:: python

103

104 class KillCmdConfig(Config):

105 # Will be loaded from `SIGNAL`.

106 signal: int

107

108 # Will be loaded from `PROCESS_ID`.

109 pid: int = field(env="PROCESS_ID")

110

111In nested configs, environment variable names are prefixed with name

112of a field that contains the nested config:

113

114.. code-block:: python

115

116 class BigConfig(Config):

117 # `kill_cmd.signal` will be loaded from `KILL_CMD_SIGNAL`.

118 kill_cmd: KillCmdConfig

119

120 # `kill_cmd_2.signal` will be loaded from `KILL_SIGNAL`.

121 kill_cmd_2: KillCmdConfig = field(env="KILL")

122

123 # `kill_cmd_3.signal` will be loaded from `SIGNAL`.

124 kill_cmd_3: KillCmdConfig = field(env="")

125

126You can also disable loading a field from an environment altogether:

127

128.. code-block:: python

129

130 class KillCmdConfig(Config):

131 # Will not be loaded from env.

132 pid: int = field(env=yuio.DISABLED)

133

134To prefix all variable names with some string, pass the `prefix` parameter

135to the :meth:`~Config.load_from_env` function:

136

137.. code-block:: python

138

139 # config.kill_cmd.field will be loaded

140 # from `MY_APP_KILL_CMD_SIGNAL`

141 config = BigConfig.load_from_env("MY_APP")

142

143

144Parsing config files

145--------------------

146

147You can load config from structured config files,

148such as `json`, `yaml` or `toml`:

149

150.. skip: next

151

152.. code-block:: python

153

154 class ExecutorConfig(Config):

155 threads: int

156 use_gpu: bool = True

157

158

159 class AppConfig(Config):

160 executor: ExecutorConfig

161 model: pathlib.Path

162

163

164 config = AppConfig.load_from_json_file("~/.my_app_cfg.json")

165

166In this example, contents of the above config would be:

167

168.. code-block:: json

169

170 {

171 "executor": {

172 "threads": 16,

173 "use_gpu": true

174 },

175 "model": "/path/to/model"

176 }

177

178Note that, unlike with environment variables,

179there is no way to inline nested configs.

180

181

182Merging configs

183---------------

184

185Configs are specially designed to be merge-able. The basic pattern is to create

186an empty config instance, then :meth:`~Config.update` it with every config source:

187

188.. skip: next

189

190.. code-block:: python

191

192 config = AppConfig()

193 config.update(AppConfig.load_from_json_file("~/.my_app_cfg.json"))

194 config.update(AppConfig.load_from_env())

195 # ...and so on.

196

197The :meth:`~Config.update` function ignores default values, and only overrides

198keys that were actually configured.

199

200If you need a more complex update behavior, you can add a merge function for a field:

201

202.. code-block:: python

203

204 class AppConfig(Config):

205 plugins: list[str] = field(

206 default=[],

207 merge=lambda left, right: [*left, *right],

208 )

209

210Here, whenever we :meth:`~Config.update` ``AppConfig``, ``plugins`` from both instances

211will be concatenated.

212

213.. warning::

214

215 Merge function shouldn't mutate its arguments.

216 It should produce a new value instead.

217

218.. warning::

219

220 Merge function will not be called for default value. It's advisable to keep the

221 default value empty, and add the actual default to the initial empty config:

222

223 .. skip: next

224

225 .. code-block:: python

226

227 config = AppConfig(plugins=["markdown", "rst"])

228 config.update(...)

229

230

231Re-imports

232----------

233

234.. function:: field

235 :no-index:

236

237 Alias of :obj:`yuio.app.field`

238

239.. function:: inline

240 :no-index:

241

242 Alias of :obj:`yuio.app.inline`

243

244.. function:: bool_option

245 :no-index:

246

247 Alias of :obj:`yuio.app.bool_option`

248

249.. function:: count_option

250 :no-index:

251

252 Alias of :obj:`yuio.app.count_option`

253

254.. function:: parse_many_option

255 :no-index:

256

257 Alias of :obj:`yuio.app.parse_many_option`

258

259.. function:: parse_one_option

260 :no-index:

261

262 Alias of :obj:`yuio.app.parse_one_option`

263

264.. function:: store_const_option

265 :no-index:

266

267 Alias of :obj:`yuio.app.store_const_option`

268

269.. function:: store_false_option

270 :no-index:

271

272 Alias of :obj:`yuio.app.store_false_option`

273

274.. function:: store_true_option

275 :no-index:

276

277 Alias of :obj:`yuio.app.store_true_option`

278

279.. type:: HelpGroup

280 :no-index:

281

282 Alias of :obj:`yuio.cli.HelpGroup`.

283

284.. type:: MutuallyExclusiveGroup

285 :no-index:

286

287 Alias of :obj:`yuio.cli.MutuallyExclusiveGroup`.

288

289.. type:: OptionCtor

290 :no-index:

291

292 Alias of :obj:`yuio.app.OptionCtor`.

293

294.. type:: OptionSettings

295 :no-index:

296

297 Alias of :obj:`yuio.app.OptionSettings`.

298

299.. data:: MISC_GROUP

300 :no-index:

301

302 Alias of :obj:`yuio.cli.MISC_GROUP`.

303

304.. data:: OPTS_GROUP

305 :no-index:

306

307 Alias of :obj:`yuio.cli.OPTS_GROUP`.

308

309.. data:: SUBCOMMANDS_GROUP

310 :no-index:

311

312 Alias of :obj:`yuio.cli.SUBCOMMANDS_GROUP`.

313

314"""

315

316from __future__ import annotations

317

318import copy

319import json

320import os

321import pathlib

322import textwrap

323import types

324import warnings

325from dataclasses import dataclass

326

327import yuio

328import yuio.cli

329import yuio.complete

330import yuio.json_schema

331import yuio.parse

332import yuio.string

333from yuio.cli import (

334 MISC_GROUP,

335 OPTS_GROUP,

336 SUBCOMMANDS_GROUP,

337 HelpGroup,

338 MutuallyExclusiveGroup,

339)

340from yuio.util import find_docs as _find_docs

341

342import yuio._typing_ext as _tx

343from typing import TYPE_CHECKING

344

345if TYPE_CHECKING:

346 import typing_extensions as _t

347else:

348 from yuio import _typing as _t

349

350__all__ = [

351 "MISC_GROUP",

352 "OPTS_GROUP",

353 "SUBCOMMANDS_GROUP",

354 "Config",

355 "HelpGroup",

356 "MutuallyExclusiveGroup",

357 "OptionCtor",

358 "OptionSettings",

359 "bool_option",

360 "collect_option",

361 "count_option",

362 "field",

363 "inline",

364 "parse_many_option",

365 "parse_one_option",

366 "positional",

367 "store_const_option",

368 "store_false_option",

369 "store_true_option",

370]

371

372T = _t.TypeVar("T")

373Cfg = _t.TypeVar("Cfg", bound="Config")

374

375

376@dataclass(frozen=True, slots=True)

377class _FieldSettings:

378 default: _t.Any

379 parser: yuio.parse.Parser[_t.Any] | None = None

380 env: str | yuio.Disabled | None = None

381 flags: str | list[str] | yuio.Positional | yuio.Disabled | None = None

382 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING

383 required: bool | None = None

384 merge: _t.Callable[[_t.Any, _t.Any], _t.Any] | None = None

385 mutex_group: MutuallyExclusiveGroup | None = None

386 option_ctor: _t.Callable[[OptionSettings], yuio.cli.Option[_t.Any]] | None = None

387 help: str | yuio.Disabled | None = None

388 help_group: HelpGroup | yuio.Collapse | None | yuio.Missing = yuio.MISSING

389 metavar: str | None = None

390 usage: yuio.Collapse | bool | None = None

391 default_desc: str | None = None

392 show_if_inherited: bool | None = None

393

394 def _update_defaults(

395 self,

396 qualname: str,

397 name: str,

398 ty_with_extras: _t.Any,

399 parsed_help: str | None,

400 allow_positionals: bool,

401 cut_help: bool,

402 ) -> _Field:

403 ty = ty_with_extras

404 while _t.get_origin(ty) is _t.Annotated:

405 ty = _t.get_args(ty)[0]

406 is_subconfig = isinstance(ty, type) and issubclass(ty, Config)

407

408 env: str | yuio.Disabled

409 if self.env is not None:

410 env = self.env

411 else:

412 env = name.upper()

413 if env == "" and not is_subconfig:

414 raise TypeError(f"{qualname} got an empty env variable name")

415

416 flags: list[str] | yuio.Positional | yuio.Disabled

417 if self.flags is yuio.DISABLED:

418 flags = self.flags

419 elif self.flags is yuio.POSITIONAL:

420 if not allow_positionals:

421 raise TypeError(

422 f"{qualname}: positional arguments are not allowed in configs"

423 )

424 if is_subconfig:

425 raise TypeError(

426 f"error in {qualname}: nested configs can't be positional"

427 )

428 flags = self.flags

429 elif self.flags is None:

430 flags = ["--" + name.replace("_", "-")]

431 else:

432 if isinstance(self.flags, str):

433 if "," in self.flags:

434 flags = [

435 norm_flag

436 for flag in self.flags.split(",")

437 if (norm_flag := flag.strip())

438 ]

439 else:

440 flags = self.flags.split()

441 if not flags:

442 flags = [""]

443 else:

444 flags = self.flags

445

446 if is_subconfig:

447 if not flags:

448 raise TypeError(

449 f"error in {qualname}: nested configs should have exactly one flag; "

450 "to disable prefixing, pass an empty string as a flag"

451 )

452 if len(flags) > 1:

453 raise TypeError(

454 f"error in {qualname}: nested configs can't have multiple flags"

455 )

456 if flags[0]:

457 if not flags[0].startswith("--"):

458 raise TypeError(

459 f"error in {qualname}: nested configs can't have a short flag"

460 )

461 try:

462 yuio.cli._check_flag(flags[0])

463 except TypeError as e:

464 raise TypeError(f"error in {qualname}: {e}") from None

465 else:

466 if not flags:

467 raise TypeError(f"{qualname} should have at least one flag")

468 for flag in flags:

469 try:

470 yuio.cli._check_flag(flag)

471 except TypeError as e:

472 raise TypeError(f"error in {qualname}: {e}") from None

473

474 default = self.default

475 if is_subconfig and default is not yuio.MISSING:

476 raise TypeError(f"error in {qualname}: nested configs can't have defaults")

477

478 parser = self.parser

479 if is_subconfig and parser is not None:

480 raise TypeError(f"error in {qualname}: nested configs can't have parsers")

481 elif not is_subconfig and parser is None:

482 try:

483 parser = yuio.parse.from_type_hint(ty_with_extras)

484 except TypeError as e:

485 raise TypeError(

486 f"can't derive parser for {qualname}:\n"

487 + textwrap.indent(str(e), " ")

488 ) from None

489 if parser is not None:

490 origin = _t.get_origin(ty)

491 args = _t.get_args(ty)

492 is_optional = (

493 default is None or _tx.is_union(origin) and types.NoneType in args

494 )

495 if is_optional and not yuio.parse._is_optional_parser(parser):

496 parser = yuio.parse.Optional(parser)

497 completer = self.completer

498 metavar = self.metavar

499 if not metavar and flags is yuio.POSITIONAL:

500 metavar = f"<{name.replace('_', '-')}>"

501 if completer is not None or metavar is not None:

502 parser = yuio.parse.WithMeta(parser, desc=metavar, completer=completer)

503

504 required = self.required

505 if is_subconfig and required:

506 raise TypeError(f"error in {qualname}: nested configs can't be required")

507 if required is None:

508 if is_subconfig:

509 required = False

510 elif allow_positionals:

511 required = default is yuio.MISSING

512 else:

513 required = False

514

515 merge = self.merge

516 if is_subconfig and merge is not None:

517 raise TypeError(

518 f"error in {qualname}: nested configs can't have merge function"

519 )

520

521 mutex_group = self.mutex_group

522 if is_subconfig and mutex_group is not None:

523 raise TypeError(

524 f"error in {qualname}: nested configs can't be a part "

525 "of a mutually exclusive group"

526 )

527 if flags is yuio.POSITIONAL and mutex_group is not None:

528 raise TypeError(

529 f"error in {qualname}: positional arguments can't appear in mutually exclusive groups"

530 )

531

532 option_ctor = self.option_ctor

533 if option_ctor is not None and is_subconfig:

534 raise TypeError(

535 f"error in {qualname}: nested configs can't have option constructors"

536 )

537

538 help: str | yuio.Disabled

539 if self.help is not None:

540 help = self.help

541 full_help = help or ""

542 elif parsed_help is not None:

543 help = full_help = parsed_help

544 if cut_help and (index := help.find("\n\n")) != -1:

545 help = help[:index]

546 else:

547 help = full_help = ""

548

549 help_group = self.help_group

550 if help_group is yuio.COLLAPSE and not is_subconfig:

551 raise TypeError(

552 f"error in {qualname}: help_group=yuio.COLLAPSE only allowed for nested configs"

553 )

554

555 usage = self.usage

556

557 default_desc = self.default_desc

558

559 show_if_inherited = self.show_if_inherited

560

561 return _Field(

562 name=name,

563 qualname=qualname,

564 default=default,

565 parser=parser,

566 env=env,

567 flags=flags,

568 is_subconfig=is_subconfig,

569 ty=ty,

570 required=required,

571 merge=merge,

572 mutex_group=mutex_group,

573 option_ctor=option_ctor,

574 help=help,

575 full_help=full_help,

576 help_group=help_group,

577 usage=usage,

578 default_desc=default_desc,

579 show_if_inherited=show_if_inherited,

580 )

581

582

583@dataclass(frozen=True, slots=True)

584class _Field:

585 name: str

586 qualname: str

587 default: _t.Any

588 parser: yuio.parse.Parser[_t.Any] | None

589 env: str | yuio.Disabled

590 flags: list[str] | yuio.Positional | yuio.Disabled

591 is_subconfig: bool

592 ty: type

593 required: bool

594 merge: _t.Callable[[_t.Any, _t.Any], _t.Any] | None

595 mutex_group: MutuallyExclusiveGroup | None

596 option_ctor: _t.Callable[[OptionSettings], yuio.cli.Option[_t.Any]] | None

597 help: str | yuio.Disabled

598 full_help: str

599 help_group: HelpGroup | yuio.Collapse | None | yuio.Missing

600 usage: yuio.Collapse | bool | None

601 default_desc: str | None

602 show_if_inherited: bool | None

603

604

605@_t.overload

606def field(

607 *,

608 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING,

609 env: str | yuio.Disabled | None = None,

610 flags: str | list[str] | yuio.Disabled | None = None,

611 required: bool | None = None,

612 mutex_group: MutuallyExclusiveGroup | None = None,

613 help: str | yuio.Disabled | None = None,

614 help_group: HelpGroup | yuio.Collapse | None | yuio.Missing = yuio.MISSING,

615 metavar: str | None = None,

616 usage: yuio.Collapse | bool | None = None,

617 default_desc: str | None = None,

618 show_if_inherited: bool | None = None,

619) -> _t.Any: ...

620@_t.overload

621def field(

622 *,

623 default: None,

624 parser: yuio.parse.Parser[T] | None = None,

625 env: str | yuio.Disabled | None = None,

626 flags: str | list[str] | yuio.Disabled | None = None,

627 required: bool | None = None,

628 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING,

629 merge: _t.Callable[[T, T], T] | None = None,

630 mutex_group: MutuallyExclusiveGroup | None = None,

631 option_ctor: OptionCtor[T] | None = None,

632 help: str | yuio.Disabled | None = None,

633 help_group: HelpGroup | yuio.Collapse | None | yuio.Missing = yuio.MISSING,

634 metavar: str | None = None,

635 usage: yuio.Collapse | bool | None = None,

636 default_desc: str | None = None,

637 show_if_inherited: bool | None = None,

638) -> T | None: ...

639@_t.overload

640def field(

641 *,

642 default: T | yuio.Missing = yuio.MISSING,

643 parser: yuio.parse.Parser[T] | None = None,

644 env: str | yuio.Disabled | None = None,

645 flags: str | list[str] | yuio.Disabled | None = None,

646 required: bool | None = None,

647 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING,

648 merge: _t.Callable[[T, T], T] | None = None,

649 mutex_group: MutuallyExclusiveGroup | None = None,

650 option_ctor: OptionCtor[T] | None = None,

651 help: str | yuio.Disabled | None = None,

652 help_group: HelpGroup | yuio.Collapse | None | yuio.Missing = yuio.MISSING,

653 metavar: str | None = None,

654 usage: yuio.Collapse | bool | None = None,

655 default_desc: str | None = None,

656 show_if_inherited: bool | None = None,

657) -> T: ...

658@_t.overload

659@_t.deprecated(

660 "prefer using positional-only function arguments instead",

661 category=yuio.YuioPendingDeprecationWarning,

662)

663def field(

664 *,

665 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING,

666 env: str | yuio.Disabled | None = None,

667 flags: yuio.Positional,

668 required: bool | None = None,

669 mutex_group: MutuallyExclusiveGroup | None = None,

670 help: str | yuio.Disabled | None = None,

671 help_group: HelpGroup | yuio.Collapse | None | yuio.Missing = yuio.MISSING,

672 metavar: str | None = None,

673 usage: yuio.Collapse | bool | None = None,

674 default_desc: str | None = None,

675 show_if_inherited: bool | None = None,

676) -> _t.Any: ...

677@_t.overload

678@_t.deprecated(

679 "passing flags=yuio.POSITIONAL is discouraged, "

680 "prefer using positional-only function arguments instead",

681 category=yuio.YuioPendingDeprecationWarning,

682)

683def field(

684 *,

685 default: None,

686 parser: yuio.parse.Parser[T] | None = None,

687 env: str | yuio.Disabled | None = None,

688 flags: yuio.Positional,

689 required: bool | None = None,

690 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING,

691 merge: _t.Callable[[T, T], T] | None = None,

692 mutex_group: MutuallyExclusiveGroup | None = None,

693 option_ctor: OptionCtor[T] | None = None,

694 help: str | yuio.Disabled | None = None,

695 help_group: HelpGroup | yuio.Collapse | None | yuio.Missing = yuio.MISSING,

696 metavar: str | None = None,

697 usage: yuio.Collapse | bool | None = None,

698 default_desc: str | None = None,

699 show_if_inherited: bool | None = None,

700) -> T | None: ...

701@_t.overload

702@_t.deprecated(

703 "passing flags=yuio.POSITIONAL is discouraged, "

704 "prefer using positional-only function arguments instead",

705 category=yuio.YuioPendingDeprecationWarning,

706)

707def field(

708 *,

709 default: T | yuio.Missing = yuio.MISSING,

710 parser: yuio.parse.Parser[T] | None = None,

711 env: str | yuio.Disabled | None = None,

712 flags: yuio.Positional,

713 required: bool | None = None,

714 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING,

715 merge: _t.Callable[[T, T], T] | None = None,

716 mutex_group: MutuallyExclusiveGroup | None = None,

717 option_ctor: OptionCtor[T] | None = None,

718 help: str | yuio.Disabled | None = None,

719 help_group: HelpGroup | yuio.Collapse | None | yuio.Missing = yuio.MISSING,

720 metavar: str | None = None,

721 usage: yuio.Collapse | bool | None = None,

722 default_desc: str | None = None,

723 show_if_inherited: bool | None = None,

724) -> T: ...

725def field(

726 *,

727 default: _t.Any = yuio.MISSING,

728 parser: yuio.parse.Parser[_t.Any] | None = None,

729 env: str | yuio.Disabled | None = None,

730 flags: str | list[str] | yuio.Positional | yuio.Disabled | None = None,

731 required: bool | None = None,

732 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING,

733 merge: _t.Callable[[_t.Any, _t.Any], _t.Any] | None = None,

734 mutex_group: MutuallyExclusiveGroup | None = None,

735 option_ctor: _t.Callable[..., _t.Any] | None = None,

736 help: str | yuio.Disabled | None = None,

737 help_group: HelpGroup | yuio.Collapse | None | yuio.Missing = yuio.MISSING,

738 metavar: str | None = None,

739 usage: yuio.Collapse | bool | None = None,

740 default_desc: str | None = None,

741 show_if_inherited: bool | None = None,

742) -> _t.Any:

743 """

744 Field descriptor, used for additional configuration of CLI options

745 and config fields.

746

747 :param default:

748 default value for the field or CLI option.

749 :param parser:

750 parser that will be used to parse config values and CLI options.

751 :param env:

752 specifies name of environment variable that will be used if loading config

753 from environment.

754

755 Pass :data:`~yuio.DISABLED` to disable loading this field form environment.

756

757 In sub-config fields, controls prefix for all environment variables within

758 this sub-config; pass an empty string to disable prefixing.

759 :param flags:

760 list of names (or a single name) of CLI flags that will be used for this field.

761

762 In configs, pass :data:`~yuio.DISABLED` to disable loading this field

763 form CLI arguments.

764

765 In sub-config fields, controls prefix for all flags withing this sub-config;

766 pass an empty string to disable prefixing.

767 :param completer:

768 completer that will be used for autocompletion in CLI. Using this option

769 is equivalent to overriding `completer` with :class:`yuio.parse.WithMeta`.

770 :param merge:

771 defines how values of this field are merged when configs are updated.

772 :param mutex_group:

773 defines mutually exclusive group for this field.

774 :param option_ctor:

775 this parameter is similar to :mod:`argparse`\\ 's ``action``: it allows

776 overriding logic for handling CLI arguments by providing a custom

777 :class:`~yuio.cli.Option` implementation.

778

779 `option_ctor` should be a callable which takes a single positional argument

780 of type :class:`~yuio.app.OptionSettings`, and returns an instance

781 of :class:`yuio.cli.Option`.

782 :param help:

783 help message that will be used in CLI option description,

784 formatted using RST or Markdown

785 (see :attr:`App.doc_format <yuio.app.App.doc_format>`).

786

787 Pass :data:`yuio.DISABLED` to remove this field from CLI help.

788 :param help_group:

789 overrides group in which this field will be placed when generating CLI help

790 message.

791

792 Pass :class:`yuio.COLLAPSE` to create a collapsed group.

793 :param metavar:

794 value description that will be used for CLI help messages. Using this option

795 is equivalent to overriding `desc` with :class:`yuio.parse.WithMeta`.

796 :param usage:

797 controls how this field renders in CLI usage section.

798

799 Pass :data:`False` to remove this field from usage.

800

801 Pass :class:`yuio.COLLAPSE` to omit this field and add a single string

802 ``<options>`` instead.

803

804 Setting `usage` on sub-config fields overrides default `usage` for all

805 fields within this sub-config.

806 :param default_desc:

807 overrides description for default value in CLI help message.

808

809 Pass an empty string to hide default value.

810 :param show_if_inherited:

811 for fields with flags, enables showing this field in CLI help message

812 for subcommands.

813 :returns:

814 a magic object that will be replaced with field's default value once a new

815 config class is created.

816 :example:

817 In apps:

818

819 .. invisible-code-block: python

820

821 import yuio.app

822

823 .. code-block:: python

824

825 @yuio.app.app

826 def main(

827 # Will be loaded from `--input`.

828 input: pathlib.Path | None = None,

829 # Will be loaded from `-o` or `--output`.

830 output: pathlib.Path | None = field(

831 default=None, flags=["-o", "--output"]

832 ),

833 ): ...

834

835 In configs:

836

837 .. code-block:: python

838

839 class AppConfig(Config):

840 model: pathlib.Path | None = field(

841 default=None,

842 help="trained model to execute",

843 )

844

845 """

846

847 if flags is yuio.POSITIONAL:

848 warnings.warn(

849 "passing flags=yuio.POSITIONAL is discouraged, "

850 "prefer using positional-only function arguments instead",

851 category=yuio.YuioPendingDeprecationWarning,

852 stacklevel=2,

853 )

854

855 return _FieldSettings(

856 default=default,

857 parser=parser,

858 env=env,

859 flags=flags,

860 completer=completer,

861 required=required,

862 merge=merge,

863 mutex_group=mutex_group,

864 option_ctor=option_ctor,

865 help=help,

866 help_group=help_group,

867 metavar=metavar,

868 usage=usage,

869 default_desc=default_desc,

870 show_if_inherited=show_if_inherited,

871 )

872

873

874def inline(

875 help: str | yuio.Disabled | None = None,

876 help_group: HelpGroup | yuio.Collapse | None | yuio.Missing = yuio.MISSING,

877 usage: yuio.Collapse | bool | None = None,

878 show_if_inherited: bool | None = None,

879) -> _t.Any:

880 """

881 A shortcut for inlining nested configs.

882

883 Equivalent to calling :func:`~yuio.app.field` with `env` and `flags`

884 set to an empty string.

885

886 """

887

888 return field(

889 env="",

890 flags="",

891 help=help,

892 help_group=help_group,

893 usage=usage,

894 show_if_inherited=show_if_inherited,

895 )

896

897

898@_t.overload

899def positional(

900 *,

901 env: str | yuio.Disabled | None = None,

902 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING,

903 help: str | yuio.Disabled | None = None,

904 metavar: str | None = None,

905 usage: yuio.Collapse | bool | None = None,

906 default_desc: str | None = None,

907) -> _t.Any: ...

908@_t.overload

909def positional(

910 *,

911 default: None,

912 parser: yuio.parse.Parser[T] | None = None,

913 env: str | yuio.Disabled | None = None,

914 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING,

915 help: str | yuio.Disabled | None = None,

916 metavar: str | None = None,

917 usage: yuio.Collapse | bool | None = None,

918 default_desc: str | None = None,

919) -> T | None: ...

920@_t.overload

921def positional(

922 *,

923 default: T | yuio.Missing = yuio.MISSING,

924 parser: yuio.parse.Parser[T] | None = None,

925 env: str | yuio.Disabled | None = None,

926 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING,

927 help: str | yuio.Disabled | None = None,

928 metavar: str | None = None,

929 usage: yuio.Collapse | bool | None = None,

930 default_desc: str | None = None,

931) -> T: ...

932

933

934@_t.deprecated(

935 "prefer using positional-only function arguments instead",

936 category=yuio.YuioPendingDeprecationWarning,

937)

938def positional(

939 *,

940 default: _t.Any = yuio.MISSING,

941 parser: yuio.parse.Parser[_t.Any] | None = None,

942 env: str | yuio.Disabled | None = None,

943 completer: yuio.complete.Completer | None | yuio.Missing = yuio.MISSING,

944 help: str | yuio.Disabled | None = None,

945 metavar: str | None = None,

946 usage: yuio.Collapse | bool | None = None,

947 default_desc: str | None = None,

948) -> _t.Any:

949 """

950 A shortcut for adding a positional argument.

951

952 Equivalent to calling :func:`field` with `flags` set to :data:`~yuio.POSITIONAL`.

953

954 """

955

956 return _FieldSettings(

957 default=default,

958 parser=parser,

959 env=env,

960 flags=yuio.POSITIONAL,

961 completer=completer,

962 help=help,

963 metavar=metavar,

964 usage=usage,

965 default_desc=default_desc,

966 )

967

968

969@_t.dataclass_transform(

970 eq_default=False,

971 order_default=False,

972 kw_only_default=True,

973 frozen_default=False,

974 field_specifiers=(field, inline, positional),

975)

976class Config:

977 """

978 Base class for configs.

979

980 Pass keyword args to set fields, or pass another config to copy it::

981

982 Config(config1, config2, ..., field1=value1, ...)

983

984 Upon creation, all fields that aren't explicitly initialized

985 and don't have defaults are considered missing.

986 Accessing them will raise :class:`AttributeError`.

987

988 .. note::

989

990 Unlike dataclasses, Yuio does not provide an option to create new instances

991 of default values upon config instantiation. This is done so that default

992 values don't override non-default ones when you update one config from another.

993

994 .. automethod:: update

995

996 .. automethod:: load_from_env

997

998 .. automethod:: load_from_json_file

999

1000 .. automethod:: load_from_yaml_file

1001

1002 .. automethod:: load_from_toml_file

1003

1004 .. automethod:: load_from_parsed_file

1005

1006 .. automethod:: to_json_schema

1007

1008 .. automethod:: to_json_value

1009

1010 """

1011

1012 @classmethod

1013 def __get_fields(cls) -> dict[str, _Field]:

1014 if cls.__fields is not None:

1015 return cls.__fields

1016

1017 docs = getattr(cls, "__yuio_pre_parsed_docs__", None)

1018 if docs is None:

1019 try:

1020 docs = _find_docs(cls)

1021 except Exception:

1022 yuio._logger.warning(

1023 "unable to get documentation for class %s.%s",

1024 cls.__module__,

1025 cls.__qualname__,

1026 )

1027 docs = {}

1028

1029 fields = {}

1030

1031 for base in reversed(cls.__mro__):

1032 if base is not cls and hasattr(base, "_Config__get_fields"):

1033 fields.update(getattr(base, "_Config__get_fields")())

1034

1035 try:

1036 types = _t.get_type_hints(cls, include_extras=True)

1037 except NameError as e:

1038 if "<locals>" in cls.__qualname__:

1039 raise NameError(

1040 f"{e}. "

1041 f"Note: forward references do not work inside functions "

1042 f"(see https://github.com/python/typing/issues/797)"

1043 ) from None

1044 raise # pragma: no cover

1045

1046 cut_help = getattr(cls, "__yuio_short_help__", False)

1047

1048 for name, field in cls.__gathered_fields.items():

1049 if not isinstance(field, _FieldSettings):

1050 field = _FieldSettings(default=field)

1051

1052 fields[name] = field._update_defaults(

1053 f"{cls.__qualname__}.{name}",

1054 name,

1055 types[name],

1056 docs.get(name),

1057 cls.__allow_positionals,

1058 cut_help,

1059 )

1060 cls.__fields = fields

1061

1062 return fields

1063

1064 def __init_subclass__(cls, _allow_positionals=None, **kwargs):

1065 super().__init_subclass__(**kwargs)

1066

1067 if _allow_positionals is not None:

1068 cls.__allow_positionals: bool = _allow_positionals

1069 cls.__fields: dict[str, _Field] | None = None

1070

1071 cls.__gathered_fields: dict[str, _FieldSettings | _t.Any] = {}

1072 for name in cls.__annotations__:

1073 if not name.startswith("_"):

1074 cls.__gathered_fields[name] = cls.__dict__.get(name, yuio.MISSING)

1075 for name, value in cls.__dict__.items():

1076 if isinstance(value, _FieldSettings) and name not in cls.__gathered_fields:

1077 qualname = f"{cls.__qualname__}.{name}"

1078 raise TypeError(

1079 f"error in {qualname}: field without annotations is not allowed"

1080 )

1081 for name, value in cls.__gathered_fields.items():

1082 if isinstance(value, _FieldSettings):

1083 value = value.default

1084 setattr(cls, name, value)

1085

1086 def __init__(self, *args: _t.Self | dict[str, _t.Any], **kwargs):

1087 for name, field in self.__get_fields().items():

1088 if field.is_subconfig:

1089 setattr(self, name, field.ty())

1090

1091 for arg in args:

1092 self.update(arg)

1093

1094 self.update(kwargs)

1095

1096 def update(self, other: _t.Self | dict[str, _t.Any], /):

1097 """

1098 Update fields in this config with fields from another config.

1099

1100 This function is similar to :meth:`dict.update`.

1101

1102 Nested configs are updated recursively.

1103

1104 :param other:

1105 data for update.

1106

1107 """

1108

1109 if not other:

1110 return

1111

1112 if isinstance(other, Config):

1113 if (

1114 self.__class__ not in other.__class__.__mro__

1115 and other.__class__ not in self.__class__.__mro__

1116 ):

1117 raise TypeError("updating from an incompatible config")

1118 ns = other.__dict__

1119 elif isinstance(other, dict):

1120 ns = other

1121 for name in ns:

1122 if name not in self.__get_fields():

1123 raise TypeError(f"unknown field: {name}")

1124 else:

1125 raise TypeError("expected a dict or a config class")

1126

1127 for name, field in self.__get_fields().items():

1128 if name in ns:

1129 if field.is_subconfig:

1130 getattr(self, name).update(ns[name])

1131 elif ns[name] is not yuio.MISSING:

1132 if field.merge is not None and name in self.__dict__:

1133 setattr(self, name, field.merge(getattr(self, name), ns[name]))

1134 else:

1135 setattr(self, name, ns[name])

1136

1137 @classmethod

1138 def load_from_env(cls, prefix: str = "") -> _t.Self:

1139 """

1140 Load config from environment variables.

1141

1142 :param prefix:

1143 if given, names of all environment variables will be prefixed with

1144 this string and an underscore.

1145 :returns:

1146 a parsed config.

1147 :raises:

1148 :class:`~yuio.parse.ParsingError`.

1149

1150 """

1151

1152 return cls.__load_from_env(prefix)

1153

1154 @classmethod

1155 def __load_from_env(cls, prefix: str = "") -> _t.Self:

1156 fields = {}

1157

1158 for name, field in cls.__get_fields().items():

1159 if field.env is yuio.DISABLED:

1160 continue

1161

1162 if prefix and field.env:

1163 env = f"{prefix}_{field.env}"

1164 else:

1165 env = f"{prefix}{field.env}"

1166

1167 if field.is_subconfig:

1168 fields[name] = field.ty.load_from_env(prefix=env)

1169 elif env in os.environ:

1170 assert field.parser is not None

1171 try:

1172 fields[name] = field.parser.parse(os.environ[env])

1173 except yuio.parse.ParsingError as e:

1174 raise yuio.parse.ParsingError(

1175 "Can't parse environment variable `%s`:\n%s",

1176 env,

1177 yuio.string.Indent(e),

1178 ) from None

1179

1180 return cls(**fields)

1181

1182 @classmethod

1183 def _build_options(cls):

1184 return cls.__build_options("", "", None, True, False)

1185

1186 @classmethod

1187 def __build_options(

1188 cls,

1189 prefix: str,

1190 dest_prefix: str,

1191 help_group: yuio.cli.HelpGroup | None,

1192 usage: yuio.Collapse | bool,

1193 show_if_inherited: bool,

1194 ) -> list[yuio.cli.Option[_t.Any]]:

1195 options: list[yuio.cli.Option[_t.Any]] = []

1196

1197 if prefix:

1198 prefix += "-"

1199

1200 for name, field in cls.__get_fields().items():

1201 if field.flags is yuio.DISABLED:

1202 continue

1203

1204 dest = dest_prefix + name

1205

1206 flags: list[str] | yuio.Positional

1207 if prefix and field.flags is not yuio.POSITIONAL:

1208 flags = [prefix + flag.lstrip("-") for flag in field.flags]

1209 else:

1210 flags = field.flags

1211

1212 field_usage = field.usage

1213 if field_usage is None:

1214 field_usage = usage

1215

1216 field_show_if_inherited = field.show_if_inherited

1217 if field_show_if_inherited is None:

1218 field_show_if_inherited = show_if_inherited

1219

1220 if field.is_subconfig:

1221 assert flags is not yuio.POSITIONAL

1222 assert issubclass(field.ty, Config)

1223 if field.help is yuio.DISABLED:

1224 subgroup = yuio.cli.HelpGroup("", help=yuio.DISABLED)

1225 elif field.help_group is yuio.MISSING:

1226 if field.full_help:

1227 lines = field.full_help.split("\n\n", 1)

1228 title = lines[0].replace("\n", " ").rstrip(".").strip() or name

1229 help = lines[1] if len(lines) > 1 else ""

1230 subgroup = yuio.cli.HelpGroup(title=title, help=help)

1231 else:

1232 subgroup = help_group

1233 elif field.help_group is yuio.COLLAPSE:

1234 if field.full_help:

1235 lines = field.full_help.split("\n\n", 1)

1236 title = lines[0].replace("\n", " ").rstrip(".").strip() or name

1237 help = lines[1] if len(lines) > 1 else ""

1238 subgroup = yuio.cli.HelpGroup(title=title, help=help)

1239 else:

1240 subgroup = yuio.cli.HelpGroup(title=field.name)

1241 subgroup.collapse = True

1242 subgroup._slug = field.name

1243 else:

1244 subgroup = field.help_group

1245 options.extend(

1246 field.ty.__build_options(

1247 flags[0],

1248 dest + ".",

1249 subgroup,

1250 field_usage,

1251 field_show_if_inherited,

1252 )

1253 )

1254 continue

1255

1256 assert field.parser is not None

1257

1258 option_ctor = field.option_ctor or _default_option

1259 option = option_ctor(

1260 OptionSettings(

1261 name=name,

1262 qualname=field.qualname,

1263 parser=field.parser,

1264 flags=flags,

1265 required=field.required,

1266 mutex_group=field.mutex_group,

1267 usage=field_usage,

1268 help=field.help,

1269 help_group=field.help_group if field.help_group else help_group,

1270 show_if_inherited=field_show_if_inherited,

1271 merge=field.merge,

1272 dest=dest,

1273 default=field.default,

1274 default_desc=field.default_desc,

1275 long_flag_prefix=prefix or "--",

1276 )

1277 )

1278 options.append(option)

1279

1280 return options

1281

1282 def __getattribute(self, item):

1283 value = super().__getattribute__(item)

1284 if value is yuio.MISSING:

1285 raise AttributeError(f"{item} is not configured")

1286 else:

1287 return value

1288

1289 # A dirty hack to hide `__getattribute__` from type checkers.

1290 locals()["__getattribute__"] = __getattribute

1291

1292 def __repr__(self):

1293 field_reprs = ", ".join(

1294 f"{name}={getattr(self, name, yuio.MISSING)!r}"

1295 for name in self.__get_fields()

1296 )

1297 return f"{self.__class__.__name__}({field_reprs})"

1298

1299 def __rich_repr__(self):

1300 for name in self.__get_fields():

1301 yield name, getattr(self, name, yuio.MISSING)

1302

1303 def __copy__(self):

1304 return type(self)(self)

1305

1306 def __deepcopy__(self, memo: dict[int, _t.Any] | None = None):

1307 return type(self)(copy.deepcopy(self.__dict__, memo))

1308

1309 @classmethod

1310 def load_from_json_file(

1311 cls,

1312 path: str | pathlib.Path,

1313 /,

1314 *,

1315 ignore_unknown_fields: bool = False,

1316 ignore_missing_file: bool = False,

1317 ) -> _t.Self:

1318 """

1319 Load config from a ``.json`` file.

1320

1321 :param path:

1322 path of the config file.

1323 :param ignore_unknown_fields:

1324 if :data:`True`, this method will ignore fields that aren't listed

1325 in config class.

1326 :param ignore_missing_file:

1327 if :data:`True`, silently ignore a missing file error. This is useful

1328 when loading a config from a home directory.

1329 :returns:

1330 a parsed config.

1331 :raises:

1332 :class:`~yuio.parse.ParsingError` if config parsing has failed

1333 or if config file doesn't exist.

1334

1335 """

1336

1337 return cls.__load_from_file(

1338 path, json.loads, ignore_unknown_fields, ignore_missing_file

1339 )

1340

1341 @classmethod

1342 def load_from_yaml_file(

1343 cls,

1344 path: str | pathlib.Path,

1345 /,

1346 *,

1347 ignore_unknown_fields: bool = False,

1348 ignore_missing_file: bool = False,

1349 ) -> _t.Self:

1350 """

1351 Load config from a ``.yaml`` file.

1352

1353 This requires `PyYaml <https://pypi.org/project/PyYAML/>`__ package

1354 to be installed.

1355

1356 :param path:

1357 path of the config file.

1358 :param ignore_unknown_fields:

1359 if :data:`True`, this method will ignore fields that aren't listed

1360 in config class.

1361 :param ignore_missing_file:

1362 if :data:`True`, silently ignore a missing file error. This is useful

1363 when loading a config from a home directory.

1364 :returns:

1365 a parsed config.

1366 :raises:

1367 :class:`~yuio.parse.ParsingError` if config parsing has failed

1368 or if config file doesn't exist. Can raise :class:`ImportError`

1369 if ``PyYaml`` is not available.

1370

1371 """

1372

1373 try:

1374 import yaml

1375 except ImportError:

1376 raise ImportError("PyYaml is not available")

1377

1378 return cls.__load_from_file(

1379 path, yaml.safe_load, ignore_unknown_fields, ignore_missing_file

1380 )

1381

1382 @classmethod

1383 def load_from_toml_file(

1384 cls,

1385 path: str | pathlib.Path,

1386 /,

1387 *,

1388 ignore_unknown_fields: bool = False,

1389 ignore_missing_file: bool = False,

1390 ) -> _t.Self:

1391 """

1392 Load config from a ``.toml`` file.

1393

1394 This requires

1395 `tomllib <https://docs.python.org/3/library/tomllib.html>`_ or

1396 `toml <https://pypi.org/project/toml/>`_ package

1397 to be installed.

1398

1399 :param path:

1400 path of the config file.

1401 :param ignore_unknown_fields:

1402 if :data:`True`, this method will ignore fields that aren't listed

1403 in config class.

1404 :param ignore_missing_file:

1405 if :data:`True`, silently ignore a missing file error. This is useful

1406 when loading a config from a home directory.

1407 :returns:

1408 a parsed config.

1409 :raises:

1410 :class:`~yuio.parse.ParsingError` if config parsing has failed

1411 or if config file doesn't exist. Can raise :class:`ImportError`

1412 if ``toml`` is not available.

1413

1414 """

1415

1416 try:

1417 import toml

1418 except ImportError:

1419 try:

1420 import tomllib as toml

1421 except ImportError:

1422 raise ImportError("toml is not available")

1423

1424 return cls.__load_from_file(

1425 path, toml.loads, ignore_unknown_fields, ignore_missing_file

1426 )

1427

1428 @classmethod

1429 def __load_from_file(

1430 cls,

1431 path: str | pathlib.Path,

1432 file_parser: _t.Callable[[str], _t.Any],

1433 ignore_unknown_fields: bool = False,

1434 ignore_missing_file: bool = False,

1435 ) -> _t.Self:

1436 path = pathlib.Path(path)

1437

1438 if ignore_missing_file and (not path.exists() or not path.is_file()):

1439 return cls()

1440

1441 try:

1442 loaded = file_parser(path.read_text())

1443 except Exception as e:

1444 raise yuio.parse.ParsingError(

1445 "Invalid config <c path>%s</c>:\n%s",

1446 path,

1447 yuio.string.Indent(e),

1448 ) from None

1449

1450 return cls.load_from_parsed_file(

1451 loaded, ignore_unknown_fields=ignore_unknown_fields, path=path

1452 )

1453

1454 @classmethod

1455 def load_from_parsed_file(

1456 cls,

1457 parsed: dict[str, object],

1458 /,

1459 *,

1460 ignore_unknown_fields: bool = False,

1461 path: str | pathlib.Path | None = None,

1462 ) -> _t.Self:

1463 """

1464 Load config from parsed config file.

1465

1466 This method takes a dict with arbitrary values that you'd get from

1467 parsing type-rich configs such as ``yaml`` or ``json``.

1468

1469 For example::

1470

1471 with open("conf.yaml") as file:

1472 config = Config.load_from_parsed_file(yaml.load(file))

1473

1474 :param parsed:

1475 data from parsed file.

1476 :param ignore_unknown_fields:

1477 if :data:`True`, this method will ignore fields that aren't listed

1478 in config class.

1479 :param path:

1480 path of the original file, used for error reporting.

1481 :returns:

1482 a parsed config.

1483 :raises:

1484 :class:`~yuio.parse.ParsingError`.

1485

1486 """

1487

1488 try:

1489 return cls.__load_from_parsed_file(

1490 yuio.parse.ConfigParsingContext(parsed), ignore_unknown_fields, ""

1491 )

1492 except yuio.parse.ParsingError as e:

1493 if path is None:

1494 raise

1495 else:

1496 raise yuio.parse.ParsingError(

1497 "Invalid config <c path>%s</c>:\n%s",

1498 path,

1499 yuio.string.Indent(e),

1500 ) from None

1501

1502 @classmethod

1503 def __load_from_parsed_file(

1504 cls,

1505 ctx: yuio.parse.ConfigParsingContext,

1506 ignore_unknown_fields: bool = False,

1507 field_prefix: str = "",

1508 ) -> _t.Self:

1509 value = ctx.value

1510

1511 if not isinstance(value, dict):

1512 raise yuio.parse.ParsingError.type_mismatch(value, dict, ctx=ctx)

1513

1514 fields = {}

1515

1516 if not ignore_unknown_fields:

1517 for name in value:

1518 if name not in cls.__get_fields() and name != "$schema":

1519 raise yuio.parse.ParsingError(

1520 "Unknown field `%s`", f"{field_prefix}{name}"

1521 )

1522

1523 for name, field in cls.__get_fields().items():

1524 if name in value:

1525 if field.is_subconfig:

1526 fields[name] = field.ty.__load_from_parsed_file(

1527 ctx.descend(value[name], name),

1528 ignore_unknown_fields,

1529 field_prefix=name + ".",

1530 )

1531 else:

1532 assert field.parser is not None

1533 fields[name] = field.parser.parse_config_with_ctx(

1534 ctx.descend(value[name], name)

1535 )

1536

1537 return cls(**fields)

1538

1539 @classmethod

1540 def to_json_schema(

1541 cls, ctx: yuio.json_schema.JsonSchemaContext

1542 ) -> yuio.json_schema.JsonSchemaType:

1543 """

1544 Create a JSON schema object based on this config.

1545

1546 The purpose of this method is to make schemas for use in IDEs, i.e. to provide

1547 autocompletion or simple error checking. The returned schema is not guaranteed

1548 to reflect all constraints added to the parser.

1549

1550 :param ctx:

1551 context for building a schema.

1552 :returns:

1553 a JSON schema that describes structure of this config.

1554

1555 """

1556

1557 return ctx.add_type(cls, _tx.type_repr(cls), lambda: cls.__to_json_schema(ctx))

1558

1559 def to_json_value(

1560 self, *, include_defaults: bool = True

1561 ) -> yuio.json_schema.JsonValue:

1562 """

1563 Convert this config to a representation suitable for JSON serialization.

1564

1565 :param include_defaults:

1566 if :data:`False`, default values will be skipped.

1567 :returns:

1568 a config converted to JSON-serializable representation.

1569 :raises:

1570 :class:`TypeError` if any of the config fields contain values that can't

1571 be converted to JSON by their respective parsers.

1572

1573 """

1574

1575 data = {}

1576 for name, field in self.__get_fields().items():

1577 if not include_defaults and name not in self.__dict__:

1578 continue

1579 if field.is_subconfig:

1580 value = getattr(self, name).to_json_value(

1581 include_defaults=include_defaults

1582 )

1583 if value:

1584 data[name] = value

1585 else:

1586 assert field.parser

1587 try:

1588 value = getattr(self, name)

1589 except AttributeError:

1590 pass

1591 else:

1592 data[name] = field.parser.to_json_value(value)

1593 return data

1594

1595 @classmethod

1596 def __to_json_schema(

1597 cls, ctx: yuio.json_schema.JsonSchemaContext

1598 ) -> yuio.json_schema.JsonSchemaType:

1599 properties: dict[str, yuio.json_schema.JsonSchemaType] = {}

1600 defaults = {}

1601

1602 properties["$schema"] = yuio.json_schema.String()

1603

1604 for name, field in cls.__get_fields().items():

1605 if field.is_subconfig:

1606 properties[name] = field.ty.to_json_schema(ctx)

1607 else:

1608 assert field.parser

1609 field_schema = field.parser.to_json_schema(ctx)

1610 if field.help and field.help is not yuio.DISABLED:

1611 field_schema = yuio.json_schema.Meta(

1612 field_schema, description=field.help

1613 )

1614 properties[name] = field_schema

1615 if field.default is not yuio.MISSING:

1616 try:

1617 defaults[name] = field.parser.to_json_value(field.default)

1618 except TypeError:

1619 pass

1620

1621 return yuio.json_schema.Meta(

1622 yuio.json_schema.Object(properties),

1623 title=cls.__name__,

1624 description=cls.__doc__,

1625 default=defaults,

1626 )

1627

1628

1629Config.__init_subclass__(_allow_positionals=False)

1630

1631

1632@dataclass(eq=False, kw_only=True)

1633class OptionSettings:

1634 """

1635 Settings for creating an :class:`~yuio.cli.Option` derived from field's type

1636 and configuration.

1637

1638 """

1639

1640 name: str | None

1641 """

1642 Name of config field or app parameter that caused creation of this option.

1643

1644 """

1645

1646 qualname: str | None

1647 """

1648 Fully qualified name of config field or app parameter that caused creation

1649 of this option. Useful for reporting errors.

1650

1651 """

1652

1653 default: _t.Any | yuio.Missing

1654 """

1655 See :attr:`yuio.cli.ValueOption.default`.

1656

1657 """

1658

1659 parser: yuio.parse.Parser[_t.Any]

1660 """

1661 Parser associated with this option.

1662

1663 """

1664

1665 flags: list[str] | yuio.Positional

1666 """

1667 See :attr:`yuio.cli.Option.flags`.

1668

1669 """

1670

1671 required: bool

1672 """

1673 See :attr:`yuio.cli.Option.required`.

1674

1675 """

1676

1677 merge: _t.Callable[[_t.Any, _t.Any], _t.Any] | None

1678 """

1679 See :attr:`yuio.cli.ValueOption.merge`.

1680

1681 """

1682

1683 mutex_group: None | MutuallyExclusiveGroup

1684 """

1685 See :attr:`yuio.cli.Option.mutex_group`.

1686

1687 """

1688

1689 dest: str

1690 """

1691 See :attr:`yuio.cli.Option.dest`. We don't provide any guarantees about `dest`\\ 's

1692 contents and recommend treating it as an opaque value.

1693

1694 """

1695

1696 help: str | yuio.Disabled

1697 """

1698 See :attr:`yuio.cli.Option.help`.

1699

1700 """

1701

1702 help_group: HelpGroup | None

1703 """

1704 See :attr:`yuio.cli.Option.help_group`.

1705

1706 """

1707

1708 usage: yuio.Collapse | bool

1709 """

1710 See :attr:`yuio.cli.Option.usage`.

1711

1712 """

1713

1714 default_desc: str | None

1715 """

1716 See :attr:`yuio.cli.Option.default_desc`.

1717

1718 """

1719

1720 show_if_inherited: bool

1721 """

1722 See :attr:`yuio.cli.Option.show_if_inherited`.

1723

1724 """

1725

1726 long_flag_prefix: str

1727 """

1728 This argument will contain prefix that was added to all :attr:`~OptionSettings.flags`.

1729 For apps and top level configs it will be ``"--"``, for nested configs it will

1730 include additional prefixes, for example ``"--nested-"``.

1731

1732 """

1733

1734

1735OptionCtor: _t.TypeAlias = _t.Callable[[OptionSettings], yuio.cli.Option[T]]

1736

1737

1738def _default_option(s: OptionSettings):

1739 if s.flags is not yuio.POSITIONAL and yuio.parse._is_bool_parser(s.parser):

1740 return bool_option()(s)

1741 elif s.parser.supports_parse_many():

1742 return parse_many_option()(s)

1743 else:

1744 return parse_one_option()(s)

1745

1746

1747def bool_option(*, neg_flags: list[str] | None = None) -> OptionCtor[bool]:

1748 """

1749 Factory for :class:`yuio.cli.BoolOption`.

1750

1751 :param neg_flags:

1752 additional set of flags that will set option's value to :data:`False`. If not

1753 given, a negative flag will be created by adding prefix ``no-`` to the first

1754 long flag of the option.

1755 :example:

1756 Boolean flag :flag:`--json` implicitly creates flag :flag:`--no-json`:

1757

1758 .. code-block:: python

1759 :emphasize-lines: 5

1760

1761 @yuio.app.app

1762 def main(

1763 json: bool = yuio.app.field(

1764 default=False,

1765 option_ctor=yuio.app.bool_option(),

1766 ),

1767 ): ...

1768

1769 Boolean flag :flag:`--json` with explicitly provided flag

1770 :flag:`--disable-json`:

1771

1772 .. code-block:: python

1773 :emphasize-lines: 5-7

1774

1775 @yuio.app.app

1776 def main(

1777 json: bool = yuio.app.field(

1778 default=False,

1779 option_ctor=yuio.app.bool_option(

1780 neg_flags=["--disable-json"],

1781 ),

1782 ),

1783 ): ...

1784

1785 """

1786

1787 def ctor(s: OptionSettings, /):

1788 if s.flags is yuio.POSITIONAL:

1789 raise TypeError(f"error in {s.qualname}: BoolOption can't be positional")

1790 if neg_flags is None:

1791 _neg_flags = []

1792 for flag in s.flags:

1793 if not yuio.cli._is_short(flag) and flag.startswith(s.long_flag_prefix):

1794 prefix = s.long_flag_prefix.strip("-")

1795 if prefix:

1796 prefix += "-"

1797 suffix = flag[len(s.long_flag_prefix) :].removeprefix("-")

1798 _neg_flags.append(f"--{prefix}no-{suffix}")

1799 break

1800 elif s.long_flag_prefix == "--":

1801 _neg_flags = neg_flags

1802 else:

1803 _neg_flags = []

1804 for flag in neg_flags:

1805 _neg_flags.append(s.long_flag_prefix + flag.lstrip("-"))

1806 return yuio.cli.BoolOption(

1807 pos_flags=s.flags,

1808 neg_flags=_neg_flags,

1809 required=s.required,

1810 mutex_group=s.mutex_group,

1811 usage=s.usage,

1812 help=s.help,

1813 help_group=s.help_group,

1814 show_if_inherited=s.show_if_inherited,

1815 dest=s.dest,

1816 parser=s.parser,

1817 merge=s.merge,

1818 default=s.default,

1819 default_desc=s.default_desc,

1820 )

1821

1822 return ctor

1823

1824

1825def parse_one_option() -> OptionCtor[_t.Any]:

1826 """

1827 Factory for :class:`yuio.cli.ParseOneOption`.

1828

1829 This option takes one argument and passes it

1830 to :meth:`Parser.parse() <yuio.parse.Parser.parse>`.

1831

1832 :example:

1833 Forcing a field which can use :func:`parse_many_option`

1834 to use :func:`parse_one_option` instead.

1835

1836 .. code-block:: python

1837 :emphasize-lines: 6

1838

1839 @yuio.app.app

1840 def main(

1841 files: list[str] = yuio.app.field(

1842 default=[],

1843 parser=yuio.parse.List(yuio.parse.Int(), delimiter=","),

1844 option_ctor=yuio.app.parse_one_option(),

1845 ),

1846 ): ...

1847

1848 This will disable multi-argument syntax:

1849

1850 .. code-block:: console

1851

1852 $ prog --files a.txt,b.txt # Ok

1853 $ prog --files a.txt b.txt # Error: `--files` takes one argument.

1854

1855 """

1856

1857 def ctor(s: OptionSettings, /):

1858 return yuio.cli.ParseOneOption(

1859 flags=s.flags,

1860 required=s.required,

1861 mutex_group=s.mutex_group,

1862 usage=s.usage,

1863 help=s.help,

1864 help_group=s.help_group,

1865 show_if_inherited=s.show_if_inherited,

1866 dest=s.dest,

1867 parser=s.parser,

1868 merge=s.merge,

1869 default=s.default,

1870 default_desc=s.default_desc,

1871 )

1872

1873 return ctor

1874

1875

1876def parse_many_option() -> OptionCtor[_t.Any]:

1877 """

1878 Factory for :class:`yuio.cli.ParseManyOption`.

1879

1880 This option takes multiple arguments and passes them

1881 to :meth:`Parser.parse_many() <yuio.parse.Parser.parse_many>`.

1882

1883 """

1884

1885 def ctor(s: OptionSettings, /):

1886 return yuio.cli.ParseManyOption(

1887 flags=s.flags,

1888 required=s.required,

1889 mutex_group=s.mutex_group,

1890 usage=s.usage,

1891 help=s.help,

1892 help_group=s.help_group,

1893 show_if_inherited=s.show_if_inherited,

1894 dest=s.dest,

1895 parser=s.parser,

1896 merge=s.merge,

1897 default=s.default,

1898 default_desc=s.default_desc,

1899 )

1900

1901 return ctor

1902

1903

1904def collect_option() -> OptionCtor[_t.Any]:

1905 """

1906 Factory for :class:`yuio.cli.ParseManyOption`.

1907

1908 This option takes single argument; it collects all arguments across all uses

1909 of this option, and passes them

1910 to :meth:`Parser.parse_many() <yuio.parse.Parser.parse_many>`.

1911

1912 :example:

1913 Forcing a field which can use :func:`parse_many_option`

1914 to collect arguments one-by-one.

1915

1916 .. code-block:: python

1917 :emphasize-lines: 5

1918

1919 @yuio.app.app

1920 def main(

1921 files: list[str] = yuio.app.field(

1922 default=[],

1923 option_ctor=yuio.app.collect_option(),

1924 flags="--file",

1925 ),

1926 ): ...

1927

1928 This will disable multi-argument syntax, but allow giving option multiple

1929 times without overriding previous value:

1930

1931 .. code-block:: console

1932

1933 $ prog --file a.txt --file b.txt # Ok

1934 $ prog --files a.txt b.txt # Error: `--file` takes one argument.

1935

1936 """

1937

1938 def ctor(s: OptionSettings, /):

1939 return yuio.cli.CollectOption(

1940 flags=s.flags,

1941 required=s.required,

1942 mutex_group=s.mutex_group,

1943 usage=s.usage,

1944 help=s.help,

1945 help_group=s.help_group,

1946 show_if_inherited=s.show_if_inherited,

1947 dest=s.dest,

1948 parser=s.parser,

1949 merge=s.merge,

1950 default=s.default,

1951 default_desc=s.default_desc,

1952 )

1953

1954 return ctor

1955

1956

1957def store_const_option(const: T) -> OptionCtor[T]:

1958 """

1959 Factory for :class:`yuio.cli.StoreConstOption`.

1960

1961 This options takes no arguments. When it's encountered amongst CLI arguments,

1962 it writes `const` to the resulting config.

1963

1964 """

1965

1966 def ctor(s: OptionSettings, /):

1967 if s.flags is yuio.POSITIONAL:

1968 raise TypeError(

1969 f"error in {s.qualname}: StoreConstOption can't be positional"

1970 )

1971

1972 return yuio.cli.StoreConstOption(

1973 flags=s.flags,

1974 required=s.required,

1975 mutex_group=s.mutex_group,

1976 usage=s.usage,

1977 help=s.help,

1978 help_group=s.help_group,

1979 show_if_inherited=s.show_if_inherited,

1980 dest=s.dest,

1981 merge=s.merge,

1982 default=s.default,

1983 default_desc=s.default_desc,

1984 const=const,

1985 )

1986

1987 return ctor

1988

1989

1990def count_option() -> OptionCtor[int]:

1991 """

1992 Factory for :class:`yuio.cli.CountOption`.

1993

1994 This option counts number of times it's encountered amongst CLI arguments.

1995

1996 Equivalent to using :func:`store_const_option` with ``const=1``

1997 and ``merge=lambda a, b: a + b``.

1998

1999 :example:

2000

2001 .. code-block:: python

2002

2003 @yuio.app.app

2004 def main(

2005 quiet: int = yuio.app.field(

2006 default=0,

2007 flags=["-q", "--quiet"],

2008 option_ctor=yuio.app.count_option(),

2009 ),

2010 ): ...

2011

2012 .. code-block:: console

2013

2014 prog -qq # quiet=2

2015

2016 """

2017

2018 def ctor(s: OptionSettings, /):

2019 if s.flags is yuio.POSITIONAL:

2020 raise TypeError(f"error in {s.qualname}: CountOption can't be positional")

2021

2022 return yuio.cli.CountOption(

2023 flags=s.flags,

2024 required=s.required,

2025 mutex_group=s.mutex_group,

2026 usage=s.usage,

2027 help=s.help,

2028 help_group=s.help_group,

2029 show_if_inherited=s.show_if_inherited,

2030 dest=s.dest,

2031 default=s.default,

2032 default_desc=s.default_desc,

2033 )

2034

2035 return ctor

2036

2037

2038def store_true_option() -> OptionCtor[bool]:

2039 """

2040 Factory for :class:`yuio.cli.StoreTrueOption`.

2041

2042 Equivalent to using :func:`store_const_option` with ``const=True``.

2043

2044 """

2045

2046 def ctor(s: OptionSettings, /):

2047 if s.flags is yuio.POSITIONAL:

2048 raise TypeError(

2049 f"error in {s.qualname}: StoreTrueOption can't be positional"

2050 )

2051

2052 return yuio.cli.StoreTrueOption(

2053 flags=s.flags,

2054 required=s.required,

2055 mutex_group=s.mutex_group,

2056 usage=s.usage,

2057 help=s.help,

2058 help_group=s.help_group,

2059 show_if_inherited=s.show_if_inherited,

2060 dest=s.dest,

2061 default=s.default,

2062 default_desc=s.default_desc,

2063 )

2064

2065 return ctor

2066

2067

2068def store_false_option() -> OptionCtor[bool]:

2069 """

2070 Factory for :class:`yuio.cli.StoreFalseOption`.

2071

2072 Equivalent to using :func:`store_const_option` with ``const=False``.

2073

2074 """

2075

2076 def ctor(s: OptionSettings, /):

2077 if s.flags is yuio.POSITIONAL:

2078 raise TypeError(

2079 f"error in {s.qualname}: StoreFalseOption can't be positional"

2080 )

2081

2082 return yuio.cli.StoreFalseOption(

2083 flags=s.flags,

2084 required=s.required,

2085 mutex_group=s.mutex_group,

2086 usage=s.usage,

2087 help=s.help,

2088 help_group=s.help_group,

2089 show_if_inherited=s.show_if_inherited,

2090 dest=s.dest,

2091 default=s.default,

2092 default_desc=s.default_desc,

2093 )

2094

2095 return ctor

Coverage for yuio / config.py: 93%

511 statements