# packages

pkgId: d9c8e9322c4241c19ff749baaf52d23c7d3f1af5898c4affda6e234e3595c5cc
name: python-hcs-cli-help
arch: noarch
version: 0.1.246
epoch: 0
release: 1
summary: Development documents and examples for hcs-cli
url: https://github.com/vmware/horizon-cloud-service-cli
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-hcs-cli-0.1.246-1.src.rpm
location_href: Packages/python-hcs-cli-help-0.1.246-1.noarch.rpm
checksum_type: sha256
description:
  [![License](https://img.shields.io/badge/License-Apache%202.0-blue)](https://github.com/vmware-labs/compliance-dashboard-for-kubernetes/blob/main/LICENSE)

  - [horizon-cloud-service-cli](#horizon-cloud-service-cli)
    - [Overview](#overview)
    - [Try it out](#try-it-out)
      - [Prerequisites](#prerequisites)
      - [Installation](#installation)
        - [Mac \& Linux](#mac--linux)
        - [Windows](#windows)
        - [Use the CLI](#use-the-cli)
    - [Documentation](#documentation)
    - [Contributing](#contributing)
    - [License](#license)



  Command line toolbox for [VMware Horizon Cloud Service (HCS) Next-Gen](https://www.vmware.com/products/horizon-cloud.html). It provides human-friendly operations based on HCS REST API.





  * Python 3.9+
  * Pip3

  Refer to [Setup Prerequisites](doc/dev-setup.md#setup-prerequisites) for more details.





  Install the tool
  ```
  brew install python3
  pip3 install hcs-cli
  ```


  Install the tool.
  ```
  pip install hcs-cli
  ```
  If you have python installed with option "Add python to path", it should be fine. Otherwise, you need to add python and it's Script directory to path.


  Use with default public HCS service.
  ```
  hcs login
  ```
  Run a command, for example, list templates:
  ```
  hcs admin template list
  ```


  * [HCS CLI - User Guide](doc/hcs-cli-user-guide.md)
  * [HCS CLI - Cheatsheet](doc/hcs-cli-cheatsheet.md)
  * [HCS CLI - Dev Guide](doc/hcs-cli-dev-guide.md)
  * [HCS Plan - template engine for HCS](doc/hcs-plan.md)
  * [Context Programming](https://github.com/nanw1103/context-programming)




  The horizon-cloud-service-cli project team welcomes contributions from the community. Before you start working with horizon-cloud-service-cli, please read and sign our Contributor License Agreement [CLA](https://cla.vmware.com/cla/1/preview). If you wish to contribute code and you have not signed our CLA, our bot will prompt you to do so when you open a Pull Request. For any questions about the CLA process, please refer to our [FAQ]([https://cla.vmware.com/faq](https://cla.vmware.com/faq)).



  Apache 2.0


pkgId: 502a37e396986aba27e1d1d733b5b75cc61c2a337e9526aca850682fc62b04c3
name: python-hda-help
arch: noarch
version: 2.20
epoch: 0
release: 1
summary: Development documents and examples for hda
url: https://github.com/ecmwf/hda
rpm_license: Apache 2.0
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-hda-2.20-1.src.rpm
location_href: Packages/python-hda-help-2.20-1.noarch.rpm
checksum_type: sha256
description:
  This package provides a fully compliant Python 3 client that can be used to search and download products using the Harmonized Data Access WEkEO API.

  HDA is a RESTful interface allowing users to search and download WEkEO datasets.

  Documentation about its usage can be found at https://www.wekeo.eu/.

  The documentation for this package can be found at: https://hda.readthedocs.io/en/latest/


  Contribute to the development of this client by creating an account on GitHub. You can follow this guidelines for a general overview of the needed steps: https://github.com/firstcontributions/first-contributions.

  Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

  Please make sure to update this README as appropriate - changes on the interface, version etc.


  Please refer to LICENSE.txt


pkgId: aa18ac92296466dc2b4c7d1a0a23068eb6d1e30d0619e992aae89f3e0f0696c5
name: python-hdf5tools-help
arch: noarch
version: 0.4.5
epoch: 0
release: 1
summary: Development documents and examples for hdf5tools
url: https://github.com/mullenkamp/hdf5tools
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-hdf5tools-0.4.5-1.src.rpm
location_href: Packages/python-hdf5tools-help-0.4.5-1.noarch.rpm
checksum_type: sha256
description:
  This git repository contains a python package with an H5 class to load and combine one or more HDF5 data files (or xarray datasets) with optional filters. The class will then export the combined data to an HDF5 file, file object, or xr.Dataset. This class is designed to be fast and safe on memory. This means that files of any size can be combined and saved even on a PC with low memory (unlike xarray).


pkgId: d5a04b402f3fd9490b2576547d4015438b5c562a3be82e0ef9cd111d923fa24b
name: python-hdmf-zarr-help
arch: noarch
version: 0.10.0
epoch: 0
release: 1
summary: Development documents and examples for hdmf-zarr
url: https://github.com/hdmf-dev/hdmf-zarr
rpm_license: BSD
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-hdmf-zarr-0.10.0-1.src.rpm
location_href: Packages/python-hdmf-zarr-help-0.10.0-1.noarch.rpm
checksum_type: sha256
description:
  The ``hdmf-zarr`` library implements a Zarr backend for HDMF as well as convenience classes for integration of Zarr with PyNWB to support writing of NWB files to Zarr.
  **Status:** The Zarr backend is **under development** and may still change. See the `overiew page <https://hdmf-zarr.readthedocs.io/en/latest/overview.html>`_ for an overview of the available features and known limitations of hdmf-zarr.


pkgId: 4a571c8cdcb58cdd005d2de6d9a9d38e2023f8a88054886f2225117fb680a853
name: python-headergen-extension-help
arch: noarch
version: 0.1.2
epoch: 0
release: 1
summary: Development documents and examples for headergen-extension
url: https://pypi.org/project/headergen-extension/
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-headergen-extension-0.1.2-1.src.rpm
location_href: Packages/python-headergen-extension-help-0.1.2-1.noarch.rpm
checksum_type: sha256
description:
  Headergen is a Jupyter Notebook extension designed to help convert undocumented notebooks into well-documented ones by generating and managing headers automatically.


  Before you begin, make sure you meet the following requirements:

  Python 3.x installed on your system.

  Jupyter Notebook version 6.x.x. Jupyter Notebook version 7.x is not supported for now.


  To install and use the headergen-extension Jupyter Notebook extension, follow the instructions below:


  To install the extension from PyPI, simply run the following command:

  `pip install headergen-extension`

  This will install the extension and all necessary dependencies directly from PyPI.


  After installing, you need to install and enable the extension in your Jupyter Notebook environment. Run the following command:

  ```bash
  jupyter nbextension install headergen_extension --py --sys-prefix
  jupyter nbextension enable headergen_extension --py --sys-prefix
  ```


  Once everything is installed and enabled, start Jupyter Notebook by running:

  `jupyter notebook`

  This will open the Jupyter interface in your browser.


pkgId: e0565636c1fb1c2599f6f18da247c4a8f1f4b46bc23b72147f31a5fcd88cd418
name: python-heagVehicleLivedataUtils-help
arch: noarch
version: 0.0.2
epoch: 0
release: 1
summary: Development documents and examples for heagVehicleLivedataUtils
url: https://pypi.org/project/heagVehicleLivedataUtils/
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-heagVehicleLivedataUtils-0.0.2-1.src.rpm
location_href: Packages/python-heagVehicleLivedataUtils-help-0.0.2-1.noarch.rpm
checksum_type: sha256
description:
  provides some tools for working with vehiclelivedata used in [this app](https://heagmobilo-live.geomobile.de/).


pkgId: d0c1a46bd9a4b36a9659f228238342b36817394cb74b17bc3cca869ff2850a9c
name: python-heatdiffs-help
arch: noarch
version: 0.1.0
epoch: 0
release: 1
summary: Development documents and examples for heatdiffs
url: https://pypi.org/project/heatdiffs/
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-heatdiffs-0.1.0-1.src.rpm
location_href: Packages/python-heatdiffs-help-0.1.0-1.noarch.rpm
checksum_type: sha256
description:
  The aim of this initial release is to demonstrate how the heat semigroup can be used as a lossy compression tool for image processing, in particular, for image corruption and restoration.

  We also demonstrate that the heat semigroup can be used for image compression, via its use as a kernel in a weighted K-Means algorithm.

  In addition, we will demonstrate that explicit diffusion equations can also be used as lossy compression for images. One can view this method as a 'learning free' denoising diffusion model. We replicate the experiments for the heat semigroup with diffusion equations, using their appropriate analogues.

  In the future, we aim to investigate further topics such as: regularised image restoration; lossless compression; and the integration of machine learning tools amongst others.



  ```bash
  pip install heatdiff
  ```



  The following notebooks demonstrate the most relevant features:

  - [Heat Semigroup](notebooks/semigroup_demo.ipynb)
  - [Diffusion Process](notebooks/diffusion_demo.ipynb)


pkgId: 47b453fdf0a6d2f74a72245fe36dae955acc9cd0434124b66d5c23049a58cf61
name: python-heatmap_cli-help
arch: noarch
version: 0.14.1
epoch: 0
release: 1
summary: Development documents and examples for heatmap_cli
url: https://pypi.org/project/heatmap_cli/
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-heatmap_cli-0.14.1-1.src.rpm
location_href: Packages/python-heatmap_cli-help-0.14.1-1.noarch.rpm
checksum_type: sha256
description:
  A console program that generates yearly calendar heatmap.



  Stable version From PyPI using `pipx`:

  ```console
  pipx install heatmap_cli
  ```

  Stable version From PyPI using `pip`:

  ```console
  python -m pip install heatmap_cli
  ```

  Upgrade to latest stable version:

  ```console
  python3 -m pip install heatmap_cli --upgrade
  ```

  Latest development version from GitHub:

  ```console
  python -m pip install -e git+https://github.com/kianmeng/heatmap_cli.git
  ```



  ```console
  heatmap -h -v
  ```

  <!--help !-->

  ```console
  usage: heatmap_cli [--demo [NUMBER_OF_COLORMAP]] [-y YEAR] [-w WEEK] [-d DATE]
                     [-O OUTPUT_DIR] [-o] [-p] [-v] [-t TITLE] [-f IMAGE_FORMAT]
                     [-c COLORMAP] [-i COLORMAP_MIN_VALUE]
                     [-x COLORMAP_MAX_VALUE] [-b]
                     [-a | --annotate | --no-annotate] [-q] [-Y] [-D] [-e] [-V]
                     [-h]
                     CSV_FILENAME

  A console program that generates yearly calendar heatmap.

    website: https://github.com/kianmeng/heatmap_cli
    changelog: https://github.com/kianmeng/heatmap_cli/blob/master/CHANGELOG.md
    issues: https://github.com/kianmeng/heatmap_cli/issues

  positional arguments:
    CSV_FILENAME
        csv filename

  options:
    --demo [NUMBER_OF_COLORMAP]
        generate number of heatmaps by colormaps (default: '182')
    -y YEAR, --year YEAR
        filter by year from the CSV file (default: '2024')
    -w WEEK, --week WEEK
        filter until week of the year from the CSV file (default: '50')
    -d DATE
        filter until the date of the year from the CSV file, will overwrite -y and -w option (default: 'None')
    -O OUTPUT_DIR, --output-dir OUTPUT_DIR
        set default output folder (default: 'output')
    -o, --open
        open the generated heatmap using default program (default: 'False')
    -p, --purge
        remove all leftover artifacts set by --output-dir folder (default: 'False')
    -v, --verbose
        show verbosity of debugging log, use -vv, -vvv for more details
    -t TITLE, --title TITLE
        set title for the heatmap (default: 'False')
    -f IMAGE_FORMAT, --format IMAGE_FORMAT
        set the default image format(default: 'png')
    -c COLORMAP, --cmap COLORMAP
        set default colormap (default: RdYlGn_r)
        Accent, Accent_r, afmhot, afmhot_r, autumn, autumn_r
        binary, binary_r, Blues, Blues_r, bone, bone_r
        BrBG, BrBG_r, brg, brg_r, BuGn, BuGn_r
        BuPu, BuPu_r, bwr, bwr_r, cividis, cividis_r
        CMRmap, CMRmap_r, cool, cool_r, coolwarm, coolwarm_r
        copper, copper_r, crest, crest_r, cubehelix, cubehelix_r
        Dark2, Dark2_r, flag, flag_r, flare, flare_r
        gist_earth, gist_earth_r, gist_gray, gist_gray_r, gist_grey, gist_heat
        gist_heat_r, gist_ncar, gist_ncar_r, gist_rainbow, gist_rainbow_r, gist_stern
        gist_stern_r, gist_yarg, gist_yarg_r, gist_yerg, GnBu, GnBu_r
        gnuplot, gnuplot2, gnuplot2_r, gnuplot_r, gray, gray_r
        Grays, Greens, Greens_r, grey, Greys, Greys_r
        hot, hot_r, hsv, hsv_r, icefire, icefire_r
        inferno, inferno_r, jet, jet_r, magma, magma_r
        mako, mako_r, nipy_spectral, nipy_spectral_r, ocean, ocean_r
        Oranges, Oranges_r, OrRd, OrRd_r, Paired, Paired_r
        Pastel1, Pastel1_r, Pastel2, Pastel2_r, pink, pink_r
        PiYG, PiYG_r, plasma, plasma_r, PRGn, PRGn_r
        prism, prism_r, PuBu, PuBu_r, PuBuGn, PuBuGn_r
        PuOr, PuOr_r, PuRd, PuRd_r, Purples, Purples_r
        rainbow, rainbow_r, RdBu, RdBu_r, RdGy, RdGy_r
        RdPu, RdPu_r, RdYlBu, RdYlBu_r, RdYlGn, RdYlGn_r
        Reds, Reds_r, rocket, rocket_r, seismic, seismic_r
        Set1, Set1_r, Set2, Set2_r, Set3, Set3_r
        Spectral, Spectral_r, spring, spring_r, summer, summer_r
        tab10, tab10_r, tab20, tab20_r, tab20b, tab20b_r
        tab20c, tab20c_r, terrain, terrain_r, turbo, turbo_r
        twilight, twilight_r, twilight_shifted, twilight_shifted_r, viridis, viridis_r
        vlag, vlag_r, winter, winter_r, Wistia, Wistia_r
        YlGn, YlGn_r, YlGnBu, YlGnBu_r, YlOrBr, YlOrBr_r
        YlOrRd, YlOrRd_r
    -i COLORMAP_MIN_VALUE, --cmap-min COLORMAP_MIN_VALUE
        set the minimum value of the colormap range (default: 'False')
    -x COLORMAP_MAX_VALUE, --cmap-max COLORMAP_MAX_VALUE
        set the maximum value of the colormap range (default: 'False')
    -b, --cbar
        show colorbar (default: 'False')
    -a, --annotate, --no-annotate
        add count to each heatmap region (default: 'True')
    -q, --quiet
        suppress all logging
    -Y, --yes
        yes to prompt
    -D, --debug
        show debugging log and stacktrace
    -e, --env
        print environment information for bug reporting
    -V, --version
        show program's version number and exit
    -h, --help
        show this help message and exit
  ```

  <!--help !-->



  Copyright (C) 2023,2024 Kian-Meng Ang

  This program is free software: you can redistribute it and/or modify it under
  the terms of the GNU Affero General Public License as published by the Free
  Software Foundation, either version 3 of the License, or (at your option) any
  later version.

  This program is distributed in the hope that it will be useful, but WITHOUT ANY
  WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
  PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

  You should have received a copy of the GNU Affero General Public License along
  with this program. If not, see <https://www.gnu.org/licenses/>.

  The fish logo used in the documentation generated by Sphinx is a public domain
  drawing of male freshwater phase [Steelhead (Oncorhynchus
  mykiss)](https://en.wikipedia.org/w/index.php?oldid=1147106962) from
  <https://commons.wikimedia.org/entity/M2787008>.


pkgId: b772be44be8b085e342e9015dd82afe9442ebe170200c014118930346cf545e6
name: python-heavyball-help
arch: noarch
version: 1.2.1
epoch: 0
release: 1
summary: Development documents and examples for heavyball
url: https://github.com/clashluke/heavyball
rpm_license: BSD
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-heavyball-1.2.1-1.src.rpm
location_href: Packages/python-heavyball-help-1.2.1-1.noarch.rpm
checksum_type: sha256
description:
  > [!IMPORTANT]
  > It's recommended to use `heavyball.utils.set_torch()` for faster training and less memory usage.

  A simple package of efficient optimizers

  The goal is not to thrive for completeness, full maintenance or abstraction, but instead to provide a simple
  largely static alternative to `torch.optim` with more and better optimizers.

  Currently (2024-12-07, 1.0.0), the recommended stable optimizer is `PrecondSchedulePaLMSOAP` (see below). The
  recommended experimental optimizer is `DelayedPSGDKron` ([tuning guide](docs/psgd_efficiency.md)).



  * **Optax-like API**: `C = heavyball.chainable; grokfast = C.ChainOpt(p, lr, C.exp_avg, C.scale_by_adam)`
  * **Stochastic Rounding**: [FP32 convergence with BF16 parameters](https://github.com/pytorch/pytorch/issues/120376)
  * **Inplace EMA**: Same math, but less memory, less compute and higher stability
  * **Foreach**: Fast multi-tensor application (turn it off to save memory via `foreach=False`)
  * **PaLM Beta2**: Fast initial
    convergence, [stable late convergence](https://x.com/_clashluke/status/1820810798693818761)
  * **ScheduleFree**: No learning rate schedule, but better convergence
  * [**Preconditioner Schedule**](https://github.com/lixilinx/psgd_torch/): Improved loss-per-step in early convergence,
    better step-per-second in late convergence (explained below)
  * **Memory-efficient storage** PSGD supports `store_triu_as_line` (default: `True`) and `q_dtype` to trade off memory
    usage for memory
    bandwidth; Other optimizers have `storage_dtype`, supporting lower-precision EMAs at no(?) performance drop via
    stochastic rounding



  ```bash
  pip install heavyball
  ```

  ```python
  import torch
  import heavyball


  model = torch.nn.Linear(16, 1)


  optimizer = heavyball.PrecondSchedulePaLMSOAP(model.parameters(), lr=1e-3)

  x = torch.randn(128, 16)
  y = torch.randn(128, 1)

  for _ in range(1000):
      optimizer.zero_grad()
      loss = torch.nn.functional.mse_loss(model(x), y)
      loss.backward()
      optimizer.step()
  ```



  | Name                          | Description                                                                                                                                                       | Advantages / Disadvantages                                                                                                                                                                                                                                                                                                            |
  |-------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | **AdamW**                     | More efficient (speed, memory) [AdamW](https://arxiv.org/abs/1711.05101)                                                                                          | + Faster than AdamW<br>+ Possibly more (numerically) stable
  | **LaProp**                    | More efficient (speed, memory) [LaProp](https://arxiv.org/abs/2002.04839)                                                                                         | + Same cost as AdamW<br>+ Marginally better converence (better proofs)<br>+ Higher hyperparameter stability<br>- Not a guaranteed win (can be neutral)<br>- No "Slingshot"                                                                                                                                                            |
  | **ADOPT**                     | More efficient (speed, memory) [ADOPT](https://arxiv.org/abs/2411.02853)                                                                                          | + Same cost as AdamW<br>+ Rigorous mathematical convergence proofs, even for challenging models (GANs)<br>- Empirically underperforms LaProp<br>- no bf16                                                                                                                                                                             |
  | **SFAdamW**                   | More efficient (speed, memory) [ScheduleFree AdamW](https://arxiv.org/abs/2405.15682)                                                                             | + Same cost as AdamW, but better eval perf<br>+ Full control over hyperparameters                                                                                                                                                                                                                                                     |
  | **PaLMSFAdamW**               | ForeachSFAdamW with [PaLM's beta2 schedule](https://arxiv.org/abs/2204.02311)                                                                                     | + Same cost as AdamW, but better eval perf<br>+ Less control, but faster early and more stable late convergence<br>+ ScheduleFree<br>- slow early convergence                                                                                                                                                                         |
  | **SOAP**                      | More efficient (speed, memory) [SOAP](https://arxiv.org/abs/2409.11321)                                                                                           | + Faster convergence (loss-at-step)<br>+ Full control over hyperparameters<br>- more memory usage<br>- more hyperparameters<br>- higher overhead than AdamW (can be ammortized; better loss-at-second)                                                                                                                                |
  | **PaLMSOAP**                  | ForeachSOAP with [PaLM's beta2 schedule](https://arxiv.org/abs/2204.02311)                                                                                        | + Faster convergence (loss-at-step)<br>+ Less control, but faster early and more stable late convergence<br>- more memory usage<br>- more hyperparameters<br>- higher overhead than AdamW (can be ammortized; better loss-at-second)                                                                                                  |
  | **SFPaLMSOAP**                | ScheduleFree PaLMForeachSOAP                                                                                                                                      | + Fast convergence (loss-at-step)<br>+ less memory usage than PaLMForeachSOAP (more tham AdamW)<br>- slower initial convergence than PaLMForeachSOAP (but allows higher LRs)<br>- higher overhead than AdamW (can be ammortized)                                                                                                      |
  | **PrecondScheduleSFPaLMSOAP** | SFPaLMForeachSOAP with [preconditioner schedule](https://github.com/lixilinx/psgd_torch/), matching the error of PrecondEvery=2 with the cost of PrecondEvery=512 | + Better initial convergence than SFPaLMForeachSOAP<br>+ Significantly faster (sec/it) later<br>+ less memory usage than PaLMForeachSOAP (more tham AdamW)<br>- slower initial convergence than PaLMForeachSOAP (but allows higher LRs)<br>- higher overhead than AdamW (can be ammortized), goes to 0 with increasing number of step |
  | **PrecondSchedulePaLMSOAP**   | PrecondScheduleSFPaLMForeachSOAP without schedule-free                                                                                                            | + Best initial convergence<br>+ Significantly faster (sec/it) later<br>+ high stability<br>- more memory usage than PrecondScheduleSFPaLMForeachSOAP<br>- higher overhead than AdamW (can be ammortized), goes to 0 with increasing number of steps                                                                                   |
  | **PrecondScheduleSOAP**       | PrecondScheduleSFPaLMForeachSOAP without PaLM's beta2 schedule                                                                                                    | + Better initial convergence<br>+ Significantly faster (sec/it) later<br>- more memory usage than PrecondScheduleSFPaLMForeachSOAP<br>- higher overhead than AdamW (can be ammortized), goes to 0 with increasing number of steps                                                                                                     |



  The default preconditioner schedule (`f`) would yield the following update intervals:

  | Steps     | Interval, `f` | Total (schedule) | Total (constant, every 2) | Total (constant, every 16) |
  |-----------|---------------|------------------|---------------------------|----------------------------|
  | 10        | 1.00005       | 10               | 5 (0.5x)                  | 0 (0.0x)                   |
  | 100       | 1.026         | 99               | 50 (0.5x)                 | 6 (0.1x)                   |
  | 1,000     | 2.0           | 738              | 500 (0.7x)                | 62 (0.1x)                  |
  | 10,000    | 14.3          | 2,168            | 5,000 (2.3x)              | 625 (0.3x)                 |
  | 100,000   | 100.2         | 4,049            | 50,000 (12.3x)            | 6,250 (1.5x)               |
  | 1,000,000 | 513           | 7,245            | 500,000 (69.0x)           | 62,500 (8.6x)              |



  Second order optimizers make it difficult to estimate memory usage, as it depends on shapes and hyperparameters. To
  estimate your memory usage, you may use `test/test_memory.py` which attempts to ensure there are no regressions.\
  Furthermore, you can find real-world memory usage of a 300M parameters video diffusion model below:
  ![img.png](assets/memory.png)



  HeavyBall offers various configurations of PSGD:

  * "PSGDKron" is the baseline, equivalent to [kron_torch](https://github.com/evanatyourservice/kron_torch/), but with
    lower compute and memory
    overhead.
  * "PurePSGD" has no momentum, further reducing memory and compute
  * "DelayedPSGD" implements SOAP/ADOPT-style off-by-one momentum, which has worse initial convergence but higher
    stability
    ![img.png](assets/delayed_psgd.png)



  To access `heavyball.utils`, you need to explicitly `import heavyball.utils`.\
  It has several handy functions:

  * `set_torch()` sets pytorch optimization settings (TF32, opt_einsum, benchmark, ...)
  * `compile_mode`, a string passed as-is to `torch.compile(mode=compile_mode)` in all compiled heavyball calls; `compile_mode=None` disables torch_compile
  * `zeroth_power_mode`, a string determining whether to use QR, newtonschulz, or svd or eigh to approximate
    the eigenvectors.


pkgId: b8c6b5ea082fec8921469544433a207e1df5b199374951499319ed457f36a61a
name: python-helical-help
arch: noarch
version: 0.0.1a16
epoch: 0
release: 1
summary: Development documents and examples for helical
url: https://github.com/helicalAI/helical
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-helical-0.0.1a16-1.src.rpm
location_href: Packages/python-helical-help-0.0.1a16-1.noarch.rpm
checksum_type: sha256
description:
  <div align="center">
    <p><a href="https://helical.readthedocs.io/"/>
    <picture>
      <source media="(prefers-color-scheme: dark)" srcset="docs/assets/logo_and_text_v2_white.png">
      <source media="(prefers-color-scheme: light)" srcset="docs/assets/logo_and_text_v2.png">
      <img alt="Helical Logo" src="docs/assets/logo_and_text_v2_white.png" width="300">
    </picture>
    </a></p>
  </div>




  Helical provides a framework for state-of-the-art pre-trained bio foundation models on genomics and transcriptomics modalities.

  Helical simplifies the entire application lifecycle when building with bio foundation models. You will be able to:
  - Leverage the latest bio foundation models through our easy-to-use python package
  - Run example notebooks on key downstream tasks from examples

  We will update this repo on a regular basis with new models, benchmarks, modalities and functions - so stay tuned.
  Let’s build the most exciting AI-for-Bio community together!
  <div align="center">

  ![Workflow](https://github.com/helicalAI/helical/actions/workflows/release.yml/badge.svg) &nbsp;
  ![Workflow](https://github.com/helicalAI/helical/actions/workflows/github-code-scanning/codeql/badge.svg) &nbsp;
  [![Docs](https://img.shields.io/badge/docs-available-brightgreen)](https://helical.readthedocs.io/) &nbsp;
  [![PyPI version](https://badge.fury.io/py/helical.svg)](https://badge.fury.io/py/helical) &nbsp;
  ![GitHub contributors](https://img.shields.io/github/contributors/helicalAI/helical) &nbsp;

  </div>



  We’re thrilled to announce the release of our first-ever mRNA Bio Foundation Model, designed to:

  1) Be Efficient, handling long sequence lengths effortlessly
  2) Balance Diversity & Specificity, leveraging a 2-step pre-training approach
  3) Deliver High-Resolution, using single nucleotides as a resolution

  Check out our <a href="https://www.helical-ai.com/blog/helix-mrna-v0" target="_blank">blog post</a> to learn more about our approach and read the <a href="https://helical.readthedocs.io/en/latest/model_cards/helix_mrna/" target="_blank">model card</a> to get started.



  We recommend installing Helical within a conda environment with the commands below (run them in your terminal) - this step is optional:
  ```
  conda create --name helical-package python=3.11.8
  conda activate helical-package
  ```

  To install the latest pip release of our Helical package, you can run the command below:
  ```
  pip install helical
  ```

  To install the latest Helical package, you can run the command below:
  ```
  pip install --upgrade git+https://github.com/helicalAI/helical.git
  ```

  Alternatively, clone the repo and install it:
  ```
  git clone https://github.com/helicalAI/helical.git
  pip install .
  ```

  [Optional] To install mamba-ssm and causal-conv1d use the command below:
  ```
  pip install helical[mamba-ssm]
  ```
  or in case you're installing from the Helical repo cloned locally:
  ```
  pip install .[mamba-ssm]
  ```

  Note: make sure your machine has GPU(s) and Cuda installed. Currently this is a requirement for the packages mamba-ssm and causal-conv1d.


  If you desire to run your code in a singularity file, you can use the [singularity.def](./singularity.def) file and build an apptainer with it:
  ```
  apptainer build --sandbox singularity/helical singularity.def
  ```

  and then shell into the sandbox container (use the --nv flag if you have a GPU available):
  ```
  apptainer shell --nv --fakeroot singularity/helical/
  ```


  - [Helix-mRNA](https://helical.readthedocs.io/en/latest/model_cards/helix_mrna/)
  - [Mamba2-mRNA](https://helical.readthedocs.io/en/latest/model_cards/mamba2_mrna/)
  - [Geneformer](https://helical.readthedocs.io/en/latest/model_cards/geneformer/)
  - [scGPT](https://helical.readthedocs.io/en/latest/model_cards/scgpt/)
  - [Universal Cell Embedding (UCE)](https://helical.readthedocs.io/en/latest/model_cards/uce/)


  - [HyenaDNA](https://helical.readthedocs.io/en/latest/model_cards/hyenadna/)
  - [Caduceus](https://helical.readthedocs.io/en/latest/model_cards/caduceus/)




  To run examples, be sure to have installed the Helical package (see Installation) and that it is up-to-date.

  You can look directly into the example folder above and download the script of your choice, look into our [documentation](https://helical.readthedocs.io/) for step-by-step guides or directly clone the repository using:
  ```
  git clone https://github.com/helicalAI/helical.git
  ```
  Within the `examples/notebooks` folder, open the notebook of your choice. We recommend starting with `Quick-Start-Tutorial.ipynb`



  | Example | Description | Colab |
  | ----------- | ----------- |----------- |
  |[Quick-Start-Tutorial.ipynb](./examples/notebooks/Quick-Start-Tutorial.ipynb)| A tutorial to quickly get used to the helical package and environment. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/helicalAI/helical/blob/main/examples/notebooks/Quick-Start-Tutorial.ipynb)|
  |[Helix-mRNA.ipynb](./examples/notebooks/Helix-mRNA.ipynb)|An example of how to use the Helix-mRNA model.|[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/helicalAI/helical/blob/main/examples/notebooks/Helix-mRNA.ipynb) |
  |[Geneformer-vs-UCE.ipynb](./examples/notebooks/Geneformer-vs-UCE.ipynb) | Zero-Shot Reference Mapping with Geneformer & UCE and compare the outcomes. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/helicalAI/helical/blob/main/examples/notebooks/Geneformer-vs-UCE.ipynb) |
  |[Hyena-DNA-Inference.ipynb](./examples/notebooks/Hyena-DNA-Inference.ipynb)|An example how to do probing with HyenaDNA by training a neural network on 18 downstream classification tasks.|[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/helicalAI/helical/blob/main/examples/notebooks/Hyena-Dna-Inference.ipynb) |
  |[Cell-Type-Annotation.ipynb](./examples/notebooks/Cell-Type-Annotation.ipynb)|An example how to do probing with scGPT by training a neural network to predict cell type annotations.|[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/helicalAI/helical/blob/main/examples/notebooks/Cell-Type-Annotation.ipynb) |
  |[Cell-Type-Classification-Fine-Tuning.ipynb](./examples/notebooks/Cell-Type-Classification-Fine-Tuning.ipynb)|An example how to fine-tune different models on classification tasks.|[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/helicalAI/helical/blob/main/examples/notebooks/Cell-Type-Classification-Fine-Tuning.ipynb) |
  |[HyenaDNA-Fine-Tuning.ipynb](./examples/notebooks/HyenaDNA-Fine-Tuning.ipynb)|An example of how to fine-tune the HyenaDNA model on downstream benchmarks.|[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/helicalAI/helical/blob/main/examples/notebooks/HyenaDNA-Fine-Tuning.ipynb) |
  | Coming Soon | New models such as SCimilarity, scVI; benchmarking scripts; new use cases; others |


  We are eager to help you and interact with you. Reach out via support@helical-ai.com.
  You can also open github issues here.


  If you are (or plan to) working with bio foundation models s.a. Geneformer or UCE on RNA and DNA data, Helical will be your best buddy! We provide and improve on:
  - Up-to-date model library
  - A unified API for all models
  - User-facing abstractions tailored to computational biologists, researchers & AI developers
  - Innovative use case and application examples and ideas
  - Efficient data processing & code-base

  We will continuously upload the latest model, publish benchmarks and make our code more efficient.




  A lot of our models have been published by talend authors developing these exciting technologies. We sincerely thank the authors of the following open-source projects:

  - [scGPT](https://github.com/bowang-lab/scGPT/)
  - [Geneformer](https://huggingface.co/ctheodoris/Geneformer)
  - [UCE](https://github.com/snap-stanford/UCE)
  - [HyenaDNA](https://github.com/HazyResearch/hyena-dna)
  - [anndata](https://github.com/scverse/anndata)
  - [scanpy](https://github.com/scverse/scanpy)
  - [transformers](https://github.com/huggingface/transformers)
  - [scikit-learn](https://github.com/scikit-learn/scikit-learn)



  You can find the Licenses for each model implementation in the model repositories:

  - [Helix-mRNA](https://github.com/helicalAI/helical/blob/release/helical/models/helix_mrna/LICENSE)
  - [Mamba2-mRNA](https://github.com/helicalAI/helical/blob/release/helical/models/mamba2_mrna/LICENSE)
  - [scGPT](https://github.com/helicalAI/helical/blob/release/helical/models/scgpt/LICENSE)
  - [Geneformer](https://github.com/helicalAI/helical/blob/release/helical/models/geneformer/LICENSE)
  - [UCE](https://github.com/helicalAI/helical/blob/release/helical/models/uce/LICENSE)
  - [HyenaDNA](https://github.com/helicalAI/helical/blob/release/helical/models/hyena_dna/LICENSE)



  Please use this BibTeX to cite this repository in your publications:

  ```bibtex
  @software{allard_2024_13135902,
    author       = {Helical Team},
    title        = {helicalAI/helical: v0.0.1a14},
    month        = nov,
    year         = 2024,
    publisher    = {Zenodo},
    version      = {0.0.1a14},
    doi          = {10.5281/zenodo.13135902},
    url          = {https://doi.org/10.5281/zenodo.13135902}
  }
  ```

rpm_license:
[cell repeats an earlier one in this file: 33880 characters]


pkgId: 8bbbaa51361640dba28e7f1c561f1915d1cc6556a5a73d29b5332802245d70c4
name: python-helix-platform-api-public-client-help
arch: noarch
version: 0.0.126
epoch: 0
release: 1
summary: Development documents and examples for helix-platform-api-public-client
url: https://pypi.org/project/helix-platform-api-public-client/
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-helix-platform-api-public-client-0.0.126-1.src.rpm
location_href: Packages/python-helix-platform-api-public-client-help-0.0.126-1.noarch.rpm
checksum_type: sha256
description:
      _OpenAPI specification for the **Platform API** of **Helix**, the nomination platform of ElecLink_  ---  Additional documentation available in the API&#39;s [GitHub repository](https://github.com/eleclink-helix/platform-api)


pkgId: cb7ca146bbbfda6d70ddaf9086ca4750107df107c877c519a72ae793041db03d
name: python-helix.personmatching-help
arch: noarch
version: 1.0.52
epoch: 0
release: 1
summary: Development documents and examples for helix.personmatching
url: https://github.com/icanbwell/helix.personmatching
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-helix.personmatching-1.0.52-1.src.rpm
location_href: Packages/python-helix.personmatching-help-1.0.52-1.noarch.rpm
checksum_type: sha256
description:
  <p align="left">
    <a href="https://github.com/icanbwell/helix.personmatching/actions">
      <img src="https://github.com/icanbwell/helix.personmatching/workflows/Build%20and%20Test/badge.svg"
           alt="Continuous Integration">
    </a>
    <a href="https://github.com/icanbwell/helix.personmatching/blob/main/LICENSE">
      <img src="https://img.shields.io/badge/license-Apache%202-blue"
           alt="GitHub license">
    </a>
  </p>


  [helix_personmatching/matchers/matcher.py](helix_personmatching/matchers/matcher.py)


  This code takes in:
  1. A source Patient/Person resource or a Bundle of Patient/Person resources
  2. A target Patient/Person resource or a Bundle of Patient/Person resources
  3. A set of rule options/weights
  4. Whether to return only successful matches or all match results
  5. (Optional) Matching threshold (between 0 and 1).  Can also be set as an environment variable: `PERSON_MATCH_THRESHOLD`
  6. (Optional) Average score boost (between 0 and 1).  Can also be set as an environment variable: `PERSON_MATCH_AVERAGE_SCORE_BOOST`



  1. A list of match results where id_source is id of source record and id_target is id of matched record.


  1. The code runs through each rule and calculates a uniqueness probability (between 0 and 1) if the rule matches.
  2. The code then picks the rule with the highest uniqueness probability and uses that probability
  3. The code then calculates the average of all the other rule probabilities and boosts the uniqueness probability by it.
  4. The code then sees if any other boosting rules were requested and boosts the probability by those


  There are five cases when matching fields of two records in a rule and the weights that are applied for each case:
  1. Field present in both sides and match -> `exact_match`
  2. Field present in both sides and partial match -> `partial_match`
  3. Field missing in both sides -> `missing`
  4. Field missing in one side -> `missing`
  5. Field present in both sides but do not match -> 0.0

  For boosting rules, there is an additional `boost` weight.


pkgId: 1f236471c5e5e28fb38a5c83b2856f6a3c44a4318609ed9eb603361f8b16491a
name: python-hello-ridwaanhall-help
arch: noarch
version: 0.3.0
epoch: 0
release: 1
summary: Development documents and examples for hello-ridwaanhall
url: https://github.com/ridwaanhall/hello_ridwaanhall
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-hello-ridwaanhall-0.3.0-1.src.rpm
location_href: Packages/python-hello-ridwaanhall-help-0.3.0-1.noarch.rpm
checksum_type: sha256
description:
  A package that says hello to Ridwaan.



  ```sh
  pip install hello_ridwaanhall
  ```



  ```python
  from hello_ridwaanhall import hello, me

  print(hello.hi())
  print(hello.love())

  print(me.username())
  print(me.nim())
  print(me.name())
  print(me.email())
  print(me.college())
  ```


pkgId: c98261da2fd298a46b6e4a43ede1229dc8cec652095c54e3468b4085fe47803b
name: python-helloworld-printer-help
arch: noarch
version: 0.1
epoch: 0
release: 1
summary: Development documents and examples for helloworld-printer
url: https://github.com/kenchou2006/helloworld-print-py
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-helloworld-printer-0.1-1.src.rpm
location_href: Packages/python-helloworld-printer-help-0.1-1.noarch.rpm
checksum_type: sha256
description:
  `helloworld-printer` is a simple Python package that allows you to print custom messages in the command line. It enables you to easily output any message or text in your Python program.



  - Easily print any message from your Python program.
  - Provides a command-line tool that allows users to print messages directly from the terminal.





  You can import and use the `helloworld` package in your Python program:
  ```python
  from helloworld import *


  helloworld("print")
  ```

  This will output:
  ```plaintext
  print
  ```



  After installation, you can also use the `helloworld` command directly from the terminal to print a message:

  ```bash
  helloworld "print"
  ```

  This will output:
  ```plaintext
  print
  ```


pkgId: a792c6dab45163e1a5ad17036f42e0a61eb59504b6499dfb863efa4b5739afef
name: python-help-me-bob-help
arch: noarch
version: 0.1.2
epoch: 0
release: 1
summary: Development documents and examples for help-me-bob
url: https://github.com/rafasacaan/help-me-bob
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-help-me-bob-0.1.2-1.src.rpm
location_href: Packages/python-help-me-bob-help-0.1.2-1.noarch.rpm
checksum_type: sha256
description:
  A Python package that serves as a perfect companion, delivering random song lyrics whenever you need them right at your CLI.



  ```bash
  pip install help-me-bob


pkgId: d4114cd0d1c57f0c6834b275ab250c6aca1762fced98206c9b0d0750fff231a1
name: python-heracless-help
arch: noarch
version: 0.3.1
epoch: 0
release: 1
summary: Development documents and examples for heracless
url: https://heracless.io
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-heracless-0.3.1-1.src.rpm
location_href: Packages/python-heracless-help-0.3.1-1.noarch.rpm
checksum_type: sha256
description:
  A simple and minimalistic Config Manager using YAML.
  [![GitHub](https://img.shields.io/badge/GitHub-Repository-blue?logo=github)](https://github.com/felixscode/heracless)
  [![PyPi](https://img.shields.io/badge/PyPi-Package-blue?logo=pypi)](https://pypi.org/project/heracless/)
  [![Documentation](https://img.shields.io/badge/Documentation-Read-green?logo=readthedocs)](https://heracless.io/docs)



  - [Description](#description)
  - [Installation](#installation)
  - [Setup](#setup)
  - [CLI Tool](#cli-tool)
  - [Helper Functions in Heracless](#helper-functions-in-heracless)
    - [mutate_config](#mutate_config)
    - [as_dict](#as_dict)
    - [from_dict](#from_dict)
  - [Version](#version)
  - [Future](#future)
  - [Author](#author)



  Heracless aims to make working with config files in Python easy. It parses a config file into a dataclass and creates types as a Python stub file (.pyi) which can be used for type hints. Generated types also make autocompletion dreamy!



  Heracless is available as a pip package:

  ```bash
  pip install heracless
  ```

  If you want to build from source, run:

  ```bash
  git clone https://github.com/felixscode/heracless.git
  cd heracless
  pip install -e .
  ```



  First, create a `config.yaml` file in a desired location and add desired configs. Make a new Python file called `load_config.py` and put it somewhere into your project.
  Here is an example project structure:
  ```
  ├── src
  │   └── your_module
  │       ├── main.py
  │       └──utils
  │           └── load_config.py
  ├── data
  │   └── config.yaml
  ├── README.md
  ├── pyproject.toml
  └── .gitignore

  ```


  Paste the following code into your `load_config.py`:

  ```python
  from pathlib import Path
  from typing import Type, TypeVar

  from heracless import load_config as _load_config



  CONFIG_YAML_PATH = None

  Config = TypeVar("Config")

  def load_config(frozen: bool = True) -> Config:
      """
      Load the configuration from the specified directory and return a Config object.

      Args:
          frozen (bool, optional): Whether the configuration should be frozen. Defaults to True.

      Returns:
          Config: The loaded configuration object.

      Raises:
          FileNotFoundError: If the configuration file does not exist.
          yaml.YAMLError: If there is an error parsing the YAML configuration file.

      Note:
          CONFIG_YAML_PATH is a global variable that sets the path of your YAML config file.
      """

      file_path = Path(__file__).resolve()
      yaml_config_path = CONFIG_YAML_PATH
      return _load_config(file_path, yaml_config_path, frozen=frozen)
  ```

  After creating the `load_config.py` file, set the `CONFIG_YAML_PATH` variable to the path of your `config.yaml` file. For example:

  ```python
  CONFIG_YAML_PATH = "/path/to/your/config.yaml"
  ```




  This document describes the helper functions in the `helper.py` module of the Heracless project.



  This function takes a `Config` object, a name, and a value, and returns a new `Config` object with the value at the name updated.

  ```python
  from your_project import load_config

  from heracless.utils.helper import mutate_config

  config = load_config()
  new_config = mutate_config(config, "name", "new_value")
  ```



  This function converts a `Config` object to a dictionary.

  ```python
  from your_project import load_config

  from heracless.utils.helper import as_dict

  config = load_config()
  config_dict = as_dict(config)
  ```



  This function creates a `Config` object from a dictionary. You can specify whether the `Config` object should be frozen.

  ```python
  from heracless.utils.helper import from_dict

  config_dict = {...}  # A dictionary representing the configuration
  config = from_dict(config_dict, frozen=True)
  ```



  Heracless 0.4 <br>
  Written in Python 3.11



  - [ ] Add config variants
  - [ ] Add None type support
  - [x] Web app



  Felix Schelling<br>
  GitHub: [felixscode](https://github.com/felixscode)<br>
  Website: [heracless.io](https://heracless.io)<br>
  Personal website: [felixschelling.de](https://felixschelling.de)<br>
  Written with ❤️ in Mexico


pkgId: e7120d5651ce91fc1ed89a3701d6336e1f069bbd7ebc7f0a473d63f470487829
name: python-hermes-cai-help
arch: noarch
version: 0.0.96
epoch: 0
release: 1
summary: Development documents and examples for hermes-cai
url: https://github.com/character-tech/chat-stack
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-hermes-cai-0.0.96-1.src.rpm
location_href: Packages/python-hermes-cai-help-0.0.96-1.noarch.rpm
checksum_type: sha256
description:
  ```
  import datetime
  from hermes_cai import build_structured_prefix
  from hermes_cai.structured_prefix import StructuredPrefix, ChatContextMessage








  persona_definition = None
  username = user.name
  if persona_character:
      persona_definition = (
          persona_character.get("sanitized_definition")
          if persona_character.get("sanitized_definition")
          else persona_character.get("definition")
      )
      if persona_character.get("name") != "My Persona":
          username = persona_cahracter.get("name")
  chat_context_messages = []
  for turn in turns:
      candidate = candidates.get(turn["primary_candidate_id"])
      ccm = ChatContextMessage(
          author=turn.author_name,
          text=candidate.raw_content,
          is_pinned=turn.is_pinned,
          type=0,  # unused
      )
      chat_context_messages.append(ccm)

  raw_prompt_data = {
      "character": character,
      "chat_type": chat.chat_type,
      "user_id": user.id,
      "character_id": character.id,
      "chat_id": chat.id,
      "persona_definition": persona_definition,
      "username": username,
  }

  structured_prefix = StructuredPrefix(
      
      "reply_prompt": f"{character.get('name')}:",
      "timestamp": datetime.utcnow(),
      "space_added": True,
      "use_hermes_generation": True,
      "hermes_generation_template_name": "production_raw.yml.j2",
      "token_limit": TOKEN_LIMIT,  # IMPORTANT: this must not result in mismatch on model server otherwise it will fallback to legacy prompt.
      
      "raw_prompt_data_dict": raw_prompt_data,
      "chat_context_messages": chat_context_messages,
  )
  hermes_structured_prefix = build_structured_prefix(
      contextual_logger=logging.getLogger(__name__),  # Provides contextual logging.
      structured_prefix=structured_prefix,
      close_last_message=True,
  )


pkgId: d95d1006fe8670f9616ec7ec2b76c2efcd51807914a4cc9b286d5db48d1d807d
name: python-hg-setup-help
arch: noarch
version: 0.1.0
epoch: 0
release: 1
summary: Development documents and examples for hg-setup
url: https://pypi.org/project/hg-setup/
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-hg-setup-0.1.0-1.src.rpm
location_href: Packages/python-hg-setup-help-0.1.0-1.noarch.rpm
checksum_type: sha256
description:
  [![Latest version](https://badge.fury.io/py/hg-setup.svg)](https://pypi.python.org/pypi/hg-setup/)
  ![Supported Python versions](https://img.shields.io/pypi/pyversions/hg-setup.svg)
  [![Heptapod CI](https://foss.heptapod.net/fluiddyn/hg-setup/badges/branch/default/pipeline.svg)](https://foss.heptapod.net/fluiddyn/hg-setup/-/pipelines)
  [![Github Actions](https://github.com/fluiddyn/hg-setup/actions/workflows/ci.yml/badge.svg?branch=branch/default)](https://github.com/fluiddyn/hg-setup/actions)



  Mercurial is a Python application using C and Rust extensions. It is extendable with
  Mercurial extensions and two Python packages provide very useful Mercurial extensions
  that most users should use : hg-git (Mercurial extension hggit) and hg-evolve (Mercurial
  extensions topic and evolve).

  These things are packaged in 3 PyPI packages (associated with their equivalent
  conda-forge packages): mercurial, hg-git, hg-evolve.

  To use Mercurial extensions, one has to write few lines in a configuration file
  (~/.hgrc).

  Mercurial with hg-git and hg-evolve is great but it is a bit difficult to setup. hg-setup
  is there to help people to start with Mercurial and finalize its installation.



  Currently, only installation from source works.

  ```sh
  pipx install hg-setup@hg+https://foss.heptapod.net/fluiddyn/hg-setup
  ```

  For development installation, see the file [CONTRIBUTING.md](./CONTRIBUTING.md).



  I hope that ultimately hg-setup can be installed automatically with Mercurial when
  running install commands like:

  ```sh

  pipx install mercurial[full]
  uv tool install mercurial[full]

  conda-app install mercurial

  pixi global install mercurial[full]
  ```

  However, we are not yet there. Nevertheless, this will work very soon (once hg-setup is
  on PyPI)

  ```sh
  pipx install mercurial
  pipx inject mercurial hg-git hg-evolve hg-setup

  uv tool install mercurial --with hg-git --with hg-evolve --with hg-setup
  ```

  and this will work soon (once hg-setup is on conda-forge and mercurial-app has been
  modified).

  ```sh
  conda-app install mercurial
  pixi global install mercurial-app
  pixi global install mercurial --with hg-git --with hg-evolve --with hg-setup
  ```

  Note that this will soon work too, independently of how Mercurial is installed:

  ```sh
  uvx hg-setup init
  ```



  The ~/.hgrc file and shell completion for bash and zsh can be initialized with a simple
  Terminal User Interface (TUI):

  ```sh
  hg-setup init
  ```

  We can also avoid the TUI with

  ```sh
  hg-setup init --name "Alice Lastname" --email alice.lastname@proton.me --auto
  ```

  The shell completion for bash and zsh can be initialized with:

  ```sh
  hg-setup init-shell-completion bash
  hg-setup init-shell-completion zsh
  ```


pkgId: 60f5c9f39d7596412e116801fb9e2faac4c9e3d0e1a8167363717e4e1e6fde92
name: python-hh-applicant-tool-help
arch: noarch
version: 0.5.2
epoch: 0
release: 1
summary: Development documents and examples for hh-applicant-tool
url: https://pypi.org/project/hh-applicant-tool/
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-hh-applicant-tool-0.5.2-1.src.rpm
location_href: Packages/python-hh-applicant-tool-help-0.5.2-1.noarch.rpm
checksum_type: sha256
description:
  ![Publish to PyPI](https://github.com/s3rgeym/hh-applicant-tool/actions/workflows/publish.yml/badge.svg)
  [![PyPi Version](https://img.shields.io/pypi/v/hh-applicant-tool)]()
  [![Python Versions](https://img.shields.io/pypi/pyversions/hh-applicant-tool.svg)]()
  [![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/s3rgeym/hh-applicant-tool)]()
  [![PyPI - Downloads](https://img.shields.io/pypi/dm/hh-applicant-tool)]()
  [![Total Downloads](https://static.pepy.tech/badge/hh-applicant-tool)]()

  <div align="center">
    <img src="https://github.com/user-attachments/assets/29d91490-2c83-4e3f-a573-c7a6182a4044" width="500">
  </div>



  Утилита для успешных волчат, служащая для автоматизации действий на HH.RU таких как рассылка откликов на подходящие вакансии и обновление всех резюме. Но данная утилита больше чем просто спамилка отзывами, вы так же выступаете в роли тайного агента, и если в списке подходящих вакансий встречается отказ, она возвращает ссылку на обсуждение работодателя в группе [Отзывы о работодателях с HH.RU](https://t.me/otzyvy_headhunter). Там вы можете написать отзыв о работодателе и почитать чужие. Для этого собираются данные о работодателях и их вакансиях (персональные данные пользователя не передаются ни в каком виде). Отправку данных на сервер разработчика можно отключить, но тогда вы не получите ссылку на обсуждение, а так же не сможете пожаловаться на неадекватного мудака, выкатившего отказ после "небольшого" тестового задания на недельку. Через сайты на таких жаловаться бесполезно: владелец сайта за деньги или после угроз судом удаляют отзывы. Единственное место где можно написать отзыв — это **Telegram**.

  Работает с Python >= 3.10. Нужную версию Python можно поставить через
  asdf/pyenv/conda и что-то еще.

  Данная утилита написана для Linux, но будет работать и на Ga..Mac OS, и в Windows, но с WSL не будет, так как для авторизации требуются оконный сервер X11 либо Wayland — только прямая установка пакета через pip в Windows. После авторизации вы можете перенести конфиг на сервер и запускать утилиту через systemd или cron. Столь странный процесс связан с тем, что на странице авторизации запускается море скриптов, которые шифруют данные на клиенте перед отправкой на сервер, а так же выполняется куча запросов чтобы проверить не бот ли ты. Хорошо, что после авторизации никаких проверок по факту нет, даже айпи не проверяется на соответсвие тому с какого была авторизация. В этой лапше мне лень разбираться. Так же при наличии рутованного телефона можно вытащить `access` и `refresh` токены из официального приложения и добавить их в конфиг.

  Пример работы:

  ![image](https://github.com/user-attachments/assets/a0cce1aa-884b-4d84-905a-3bb207eba4a3)

  > Если в веб-интерфейсе выставить фильтры, то они будут применяться в скрипте при отклике на подходящие



  Долгое время я делал массовые заявки с помощью консоли браузера:

  ```js
  $$('[data-qa="vacancy-serp__vacancy_response"]').forEach((el) => el.click());
  ```

  Оно работает, хоть и не идеально. Я даже пробовал автоматизировать рассылки через `p[yu]ppeeter`, пока не прочитал [документацию](https://github.com/hhru/api). И не обнаружил, что **API** (интерфейс) содержит все необходимые мне методы. Headhunter позволяет создать свое приложение, но там ручная модерация, и наврядли кто-то разрешит мне создать приложение для спама заявками. Я [декомпилировал](https://gist.github.com/s3rgeym/eee96bbf91b04f7eb46b7449f8884a00) официальное приложение для **Android** и получил **CLIENT_ID** и **CLIENT_SECRET**, необходимые для работы через **API**.



  ```bash


  $ pipx install 'hh-applicant-tool[qt]'


  $ pipx install git+https://github.com/s3rgeym/hh-applicant-tool


  $ pipx upgrade hh-applicant-tool
  ```

  Отдельно я распишу процесс установки в **Windows** в подробностях:

  * Для начала поставьте последнюю версию **Python 3** любым удобным способом.
  * Запустите **Terminal** или **PowerShell** от Администратора и выполните:
    ```ps
    Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy Unrestricted
    ```
    Данная политика разрешает текущему пользователю (от которого зашли) запускать скрипты. Без нее не будут работать виртуальные окружения.
  * Создайте и активируйте виртуальное окружение:
    ```ps
    PS> python -m pip venv hh-applicant-venv
    PS> .\hh-applicant-venv\Scripts\activate
    ```
  * Поставьте все пакеты в виртуальное окружение `hh-applicant-venv`:
    ```ps
    (hh-applicant-venv) PS> pip install hh-applicant-tool[qt]
    ```
  * Проверьте работает ли оно:
    ```ps
    (hh-applicant-venv) PS> hh-applicant-tool -h
    ```
  * В случае неудачи вернитесь к первому шагу.
  * Для последующих запусков сначала активируйте виртуальное окружение.



  ```bash
  $ hh-applicant-tool -vv authorize
  ```

  ![image](https://github.com/user-attachments/assets/88961e31-4ea3-478f-8c43-914d6785bc3b)

  > В Windows не забудьте разрешить доступ к сети (Allow access) в всплывающем окне.

  Проверка авторизации:

  ```bash
  $ hh-applicant-tool whoami
  {
    "auth_type": "applicant",
    "counters": {
      "new_resume_views": 1488,
      "resumes_count": 1,
      "unread_negotiations": 228
    },
    "email": "vasya.pupkin@gmail.com",
    "employer": null,
    "first_name": "Вася",
    "id": "1234567890",
    "is_admin": false,
    "is_anonymous": false,
    "is_applicant": true,
    "is_application": false,
    "is_employer": false,
    "is_in_search": true,
    "last_name": "Пупкин",
    "manager": null,
    "mid_name": null,
    "middle_name": null,
    "negotiations_url": "https://api.hh.ru/negotiations",
    "personal_manager": null,
    "phone": "79012345678",
    "profile_videos": {
      "items": []
    },
    "resumes_url": "https://api.hh.ru/resumes/mine"
  }
  ```

  В случае успешной авторизации токены будут сохранены в `config.json`:

  ```json
  {
    "token": {
      "access_token": "...",
      "created_at": 1678151427,
      "expires_in": 1209599,
      "refresh_token": "...",
      "token_type": "bearer"
    }
  }
  ```

  Токен доступа выдается на две недели. После его нужно обновить:

  ```bash
  $ hh-applicant-tool refresh-token
  ```



  | OS                         | Путь                                                                |
  |----------------------------|---------------------------------------------------------------------|
  | **Windows**                | `C:\Users\%username%\AppData\Roaming\hh-applicant-tool\config.json` |
  | **macOS**                  | `~/Library/Application Support/hh-applicant-tool/config.json`       |
  | **Linux**                  | `~/.config/hh-applicant-tool/config.json`                           |


  Через конфиг можно задать дополнительные настройки:

  | Имя атрибута | Описание |
  | --- | --- |
  | `user_agent` | Кастомный юзерагент, передаваемый при кажом запросе, например, `Mozilla/5.0 YablanBrowser` |
  | `proxy_url` | Прокси, используемый для всех запросов, например, `socks5h://127.0.0.1:9050` |
  | `reply_message` | Сообщение для ответа работодателю при отклике на вакансии, см. формат сообщений |



  ```bash
  $ hh-applicant-tool [ GLOBAL_FLAGS ] [ OPERATION [ OPERATION_FLAGS  ] ]


  $ hh-applicant-tool -h


  $ hh-applicant-tool apply-similar -h


  $ hh-applicant-tool authorize


  $ hh-applicant-tool apply-similar


  $ hh-applicant-tool update-resumes


  $ hh-applicant-tool clear-negotiations --blacklist-discard
  ```

  Можно вызвать любой метод API:

  ```bash
  $ hh-applicant-tool call-api /employers text="IT" only_with_vacancies=true | jq -r '.items[].alternate_url'
  https://hh.ru/employer/1966364
  https://hh.ru/employer/4679771
  https://hh.ru/employer/8932785
  https://hh.ru/employer/9451699
  https://hh.ru/employer/766478
  https://hh.ru/employer/4168187
  https://hh.ru/employer/9274777
  https://hh.ru/employer/1763330
  https://hh.ru/employer/5926815
  https://hh.ru/employer/1592535
  https://hh.ru/employer/9627641
  https://hh.ru/employer/4073857
  https://hh.ru/employer/2667859
  https://hh.ru/employer/4053700
  https://hh.ru/employer/5190600
  https://hh.ru/employer/607484
  https://hh.ru/employer/9386615
  https://hh.ru/employer/80660
  https://hh.ru/employer/6078902
  https://hh.ru/employer/1918903
  ```

  Данная возможность полезна для написания Bash-скриптов.

  Глобальные флаги:

  - `-v` используется для вывода отладочной информации. Два таких флага, например, выводят запросы к **API**.
  - `-c <path>` можно создать путь до конфига. С помощью этого флага можно одновременно использовать несколько профилей.

  | Операция               | Описание                                                                                            |
  | ---------------------- | --------------------------------------------------------------------------------------------------- |
  | **authorize**          | Открывает сайт hh.ru для авторизации и перехватывает перенаправление на `hhadnroid://oauthresponse` |
  | **whoami**             | Выводит информацию об авторизованном пользователе                                                   |
  | **list-resumes**       | Список резюме                                                                                       |
  | **update-resumes**     | Обновить все резюме. Аналогично нажатию кнопки «Обновить дату».                                     |
  | **apply-similar**      | Откликнуться на все подходящие вакансии. Лимит = 200 в день. На HH есть спам-фильтры, так что лучше не рассылайте отклики со ссылками. |
  | **reply-employers** | Ответить во все чаты с работодателями, где нет ответа либо не прочитали ваш предыдущий ответ |
  | **clear-negotiations** | Удаляет отказы и отменяет заявки, которые долго висят                                               |
  | **call-api**           | Вызов произвольного метода API с выводом результата.                                                |
  | **refresh-token**      | Обновляет access_token.                                                                             |
  | **get-employer-contacts** | Получить список контактов работодателя, даже если тот не высылал приглашения. Это  функционал для избранных, но в группе есть бесплатный бот с тем же функционалом. |



  Команда `apply-similar` поддерживает специальный формат сообщений.

  Так же в сообщении можно использовать плейсхолдеры:

  - **`s`**: Название вакансии.
  - **`s`**: Название работодателя.
  - **`s`**: Имя пользователя.
  - **`s`**: Фамилия пользователя.
  - **`s`**: Email пользователя.
  - **`s`**: Телефон пользователя.

  Эти плейсхолдеры могут быть использованы в сообщениях для отклика на вакансии, чтобы динамически подставлять соответствующие данные в текст сообщения. Например:

  ```
  Меня заинтересовала ваша вакансия s. Прошу рассмотреть мою кандидатуру. С уважением, s s.
  ```

  Так же можно делать текст уникальным с помощью `{}`. Внутри них через `|` перечисляются варианты, один из которых будет случайно выбран:

  ```
  {Здоров|Привет}, {как {ты|сам}|что делаешь}?
  ```

  В итоге получится что-то типа:

  ```
  Привет, как ты?
  ```



  Утилита использует систему плагинов. Все они лежат в [operations](https://github.com/s3rgeym/hh-applicant-tool/tree/main/hh_applicant_tool/operations). Модули расположенные там автоматически добавляются как доступные операции. За основу для своего плагина можно взять [whoami.py](https://github.com/s3rgeym/hh-applicant-tool/tree/main/hh_applicant_tool/operations/whoami.py).

  Отдельные замечания у меня к API HH. Оно пиздец какое кривое. Например, при создании заявки возвращается пустой ответ либо редирект, хотя по логике должен возвраться созданный объект. Так же в ответах сервера нет `Content-Length`. Из-за этого нельзя узнать есть тело у ответа сервера нужно его пробовать прочитать. Я так понял там какой-то прокси оборачивает все запросы и отдает всегда `Transfer-Encoding: Chunked`. А еще он возвращает 502 ошибку, когда бекенд на Java падает либо долго отвечает (таймаут)? А вот [язык запросов](https://hh.ru/article/1175) мне понравился. Можно что-то типа этого использовать `NOT (!ID:123 OR !ID:456 OR !ID:789)` что бы отсеить какие-то вакансии.

  Для создания своих плагинов прочитайте документацию:

  * [HH.RU OpenAPI](https://api.hh.ru/openapi/redoc)

  Для тестирования запросов к API используйте команду `call-api` и `jq` для вывода JSON в удобочитаемом формате.


pkgId: fe08892de00c0201b9a4f28f9930dd85532f5f05736a050d7d71a503b0b55a78
name: python-hhoppe-tools-help
arch: noarch
version: 1.5.0
epoch: 0
release: 1
summary: Development documents and examples for hhoppe-tools
url: https://pypi.org/project/hhoppe-tools/
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-hhoppe-tools-1.5.0-1.src.rpm
location_href: Packages/python-hhoppe-tools-help-1.5.0-1.noarch.rpm
checksum_type: sha256
description:
  Library of Python tools -- [Hugues Hoppe](https://hhoppe.com).

  [**[API docs]**](https://hhoppe.github.io/hhoppe-tools/)


pkgId: edf8a0a1d28ed19638476b108bc96f6137456e3ad471b5783b111457339d8bf0
name: python-hid-tools-help
arch: noarch
version: 0.9
epoch: 0
release: 1
summary: Development documents and examples for hid-tools
url: http://gitlab.freedesktop.org/libevdev/hid-tools
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-hid-tools-0.9-1.src.rpm
location_href: Packages/python-hid-tools-help-0.9-1.noarch.rpm
checksum_type: sha256
description:
  hid-tools is a set of tools to interact with the kernel's HID subsystem.

  It can be run directly from the git repository or installed via `pip3
  install hid-tools`.



  The `hid-tools` repository does not need to be installed, all tools and
  kernel tests can be run straight from the git repository, for example the
  following commands clone the repository and run the `hid-recorder` tool.

  ```
  $ git clone https://gitlab.freedesktop.org/libevdev/hid-tools
  $ cd hid-tools
  $ sudo ./hid-recorder
  ```

  Where the tools need to be installed, it is recommended to use `pip`:

  ```
  $ sudo pip3 install .
  ```

  This installs all tools into the system-wide Python path. hid-tools needs
  root access to the `/dev/hidraw` nodes, an installation in the user-specific
  paths will not usually work without further commandline tweaking
  configuration.  Removal of the tools works with `pip` as well:

  ```
  $ pip3 uninstall hid-tools
  ```





  `hid-recorder` prints the HID Report Descriptor from a `/dev/hidraw` device
  node and any HID reports coming from that device. The output format can be
  used with `hid-replay` for debugging. When run without any arguments, the
  tool prints a list of available devices.

  ```
  $ sudo hid-recorder
  ```



  `hid-replay` takes the output from `hid-recorder` and replays it through a
  virtual HID device that looks exactly like the one recorded. `hid-replay`
  requires UHID support so make sure `pyudev` is installed.

  ```
  $ sudo hid-replay recording-file.hid
  ```



  `hid-decode` takes a HID Report Descriptor and prints a human-readable
  version of it. `hid-decode` takes binary report descriptors, strings of
  bytes, and other formats.

  ```
  $ hid-decode /sys/class/input/event5/device/device/report_descriptor
  ```



  The `hid-tools` repository contains a number of tests exercising the kernel
  HID subsystem. The tests are not part of the `pip3` module and must be run
  from the git repository. The most convenient invocation of the tests is by
  simply calling `pytest`. The test suite requires UHID support so make sure
  `pyudev` is installed.

  ```
  $ git clone https://gitlab.freedesktop.org/libevdev/hid-tools
  $ cd hid-tools
  ```

  **Note** If your testing system is running X, please follow the steps
  below to let X drivers ignore uhid test devices. Otherwise, the X driver
  will recognize and handle the test devices, which would interfere with
  the kernel tests and the running session.

  ```
  $ sudo cp tests/91-hid-tools-uhid-test.conf /etc/X11/xorg.conf.d/
  ```

  Restart your X server
  **End of Note**

  ```
  $ sudo pytest-3
  ```

  See the `pytest` documentation for information on how to run a subset of
  tests.



  Technical limitations require that `hid-tools` ships with a Python module
  called `hidtools`. This module is **not** to be used by external
  applications.

  **The hidtools python module does not provide any API stability guarantee.
  It may change at any time**



  `hid-tools` is licensed under the GPLv2+.


pkgId: 3827f767ab58ae05c21d618f790152a734462ade4ade826ad4849fa187ab29a5
name: python-hiddifypanel-help
arch: noarch
version: 10.80.11
epoch: 0
release: 1
summary: Development documents and examples for hiddifypanel
url: https://hiddify.com
rpm_license: LICENSE.md
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-hiddifypanel-10.80.11-1.src.rpm
location_href: Packages/python-hiddifypanel-help-10.80.11-1.noarch.rpm
checksum_type: sha256
description:
   hiddifypanel to create multi proxy using xray mtproxy and others



  <div align=center>

  [![inlang status badge](https://inlang.com/badge?url=github.com/hiddify/hiddifypanel)](https://inlang.com/editor/github.com/hiddify/hiddifypanel?ref=badge)
  </div>

  Improve existing languages or add new ones by manually editing the JSON files located in `/assets/translations` or by using the [Inlang online editor](https://inlang.com/editor/github.com/hiddify/hiddifypanel).


  Please visit https://github.com/hiddify/hiddify-manager/wiki for the installation.


  <!--


  From source:

  ```bash
  git clone https://github.com/hiddify/HiddifyPanel hiddifypanel
  cd hiddifypanel
  make install
  ```

  From pypi:

  ```bash
  pip install hiddifypanel
  ```



  This application has a CLI interface that extends the Flask CLI.

  Just run:

  ```bash
  $ hiddifypanel
  ```

  or

  ```bash
  $ python -m hiddifypanel
  ```

  To see the help message and usage instructions.



  ```bash
  hiddifypanel init-db   # run once
  echo localhost:9000/$(hiddifypanel admin-path)
  hiddifypanel run
  ```


  > **Note**: You can also use `flask run` to run the application.

  -->


pkgId: 968fb95a5443732323b5819dd7100b7d0bf86cb4be630a9679597b01f74edfc3
name: python-hier-config-cli-help
arch: noarch
version: 0.1.0
epoch: 0
release: 1
summary: Development documents and examples for hier-config-cli
url: https://pypi.org/project/hier-config-cli/
rpm_license: Apache 2.0
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-hier-config-cli-0.1.0-1.src.rpm
location_href: Packages/python-hier-config-cli-help-0.1.0-1.noarch.rpm
checksum_type: sha256
description:
  hier-config-cli is a command-line interface tool built on top of the [Hier Config](https://github.com/netdevops/hier_config) library. It enables network engineers to analyze, remediate, and predict configurations for network devices across multiple platforms.



  ```python
  hier-config-cli [COMMAND] --platform PLATFORM --running-config PATH --generated-config PATH
  ```



  * remediation
  * rollback
  * future



  Contributions are welcome! Please open an issue or submit a pull request for features, bug fixes, or documentation improvements.


pkgId: bb6c3d8cbc9cd4bdc1c2c2a622d3760ba94bc88c0cae37a5a6385b0f2dd1b2d1
name: python-hierarchicalforecast-help
arch: noarch
version: 1.0.0
epoch: 0
release: 1
summary: Development documents and examples for hierarchicalforecast
url: https://github.com/Nixtla/hierarchicalforecast/
rpm_license: Apache Software License 2.0
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-hierarchicalforecast-1.0.0-1.src.rpm
location_href: Packages/python-hierarchicalforecast-help-1.0.0-1.noarch.rpm
checksum_type: sha256
description:
  <div align="center">
  <img src="https://raw.githubusercontent.com/Nixtla/neuralforecast/main/nbs/imgs_indx/logo_mid.png">
  <h1 align="center">Hierarchical Forecast 👑</h1>
  <h3 align="center">Probabilistic hierarchical forecasting with statistical and econometric methods</h3>

  [![CI](https://github.com/Nixtla/hierarchicalforecast/actions/workflows/ci.yml/badge.svg)](https://github.com/Nixtla/hierarchicalforecast/actions/workflows/ci.yml)
  [![Python](https://img.shields.io/pypi/pyversions/hierarchicalforecast)](https://pypi.org/project/hierarchicalforecast/)
  [![PyPi](https://img.shields.io/pypi/v/hierarchicalforecast?color=blue)](https://pypi.org/project/hierarchicalforecast/)
  [![conda-nixtla](https://img.shields.io/conda/vn/conda-forge/hierarchicalforecast?color=seagreen&label=conda)](https://anaconda.org/conda-forge/hierarchicalforecast)
  [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://github.com/Nixtla/hierarchicalforecast/blob/main/LICENSE)

  **HierarchicalForecast** offers a collection of reconciliation methods, including `BottomUp`, `TopDown`, `MiddleOut`, `MinTrace` and `ERM`. And Probabilistic coherent predictions including `Normality`, `Bootstrap`, and `PERMBU`.
  </div>


  A vast amount of time series datasets are organized into structures with different levels or hierarchies of aggregation. Examples include categories, brands, or geographical groupings. Coherent forecasts across levels are necessary for consistent decision-making and planning. Hierachical Forecast offers different reconciliation methods that render coherent forecasts across hierachies.
  Until recent, this methods were mainly avaiable in the R ecosystem. This Python-based framework aims to bridge the gap between statistical modeling and Machine Learning in the time series field.



  * Classic reconciliation methods:
      - `BottomUp`: Simple addition to the upper levels.
      - `TopDown`: Distributes the top levels forecasts trough the hierarchies.
  * Alternative reconciliation methods:
      - `MiddleOut`: It anchors the base predictions in a middle level. The levels above the base predictions use the bottom-up approach, while the levels below use a top-down.
      - `MinTrace`: Minimizes the total forecast variance of the space of coherent forecasts, with the Minimum Trace reconciliation.
      - `ERM`: Optimizes the reconciliation matrix minimizing an L1 regularized objective.
  * Probabilistic coherent methods:
      - `Normality`: Uses MinTrace variance-covariance closed form matrix under a normality assumption.
      - `Bootstrap`: Generates distribution of hierarchically reconciled predictions using Gamakumara's bootstrap approach.
      - `PERMBU`: Reconciles independent sample predictions by reinjecting multivariate dependence with estimated rank permutation copulas, and performing a Bottom-Up aggregation.

  Missing something? Please open an issue here or write us in [![Slack](https://img.shields.io/badge/Slack-4A154B?&logo=slack&logoColor=white)](https://join.slack.com/t/nixtlaworkspace/shared_invite/zt-135dssye9-fWTzMpv2WBthq8NK0Yvu6A)



  **Short**: We want to contribute to the ML field by providing reliable baselines and benchmarks for hierarchical forecasting task in industry and academia. Here's the complete [paper](https://arxiv.org/abs/2207.03517).

  **Verbose**: `HierarchicalForecast` integrates publicly available processed datasets, evaluation metrics, and a curated set of standard statistical baselines. In this library we provide usage examples and references to extensive experiments where we showcase the baseline's use and evaluate the accuracy of their predictions. With this work, we hope to contribute to Machine Learning forecasting by bridging the gap to statistical and econometric modeling, as well as providing tools for the development of novel hierarchical forecasting algorithms rooted in a thorough comparison of these well-established models. We intend to continue maintaining and increasing the repository, promoting collaboration across the forecasting community.



  You can install `HierarchicalForecast`'s the Python package index [pip](https://pypi.org) with:

  ```python
  pip install hierarchicalforecast
  ```

  You can also can install `HierarchicalForecast`'s from [conda](https://anaconda.org) with:

  ```python
  conda install -c conda-forge hierarchicalforecast
  ```




  The following example needs `statsforecast` and `datasetsforecast` as additional packages. If not installed, install it via your preferred method, e.g. `pip install statsforecast datasetsforecast`.
  The `datasetsforecast` library allows us to download hierarhical datasets and we will use `statsforecast` to compute the base forecasts to be reconciled.

  You can open a complete example in Colab [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/nixtla/hierarchicalforecast/blob/main/nbs/examples/TourismSmall.ipynb)

  Minimal Example:
  ```python

  import numpy as np
  import pandas as pd


  from datasetsforecast.hierarchical import HierarchicalData


  from statsforecast.core import StatsForecast
  from statsforecast.models import AutoARIMA, Naive


  from hierarchicalforecast.core import HierarchicalReconciliation
  from hierarchicalforecast.evaluation import evaluate
  from hierarchicalforecast.methods import BottomUp, TopDown, MiddleOut
  from utilsforecast.losses import mse



  Y_df, S, tags = HierarchicalData.load('./data', 'TourismSmall')
  Y_df['ds'] = pd.to_datetime(Y_df['ds'])
  S = S.reset_index(names="unique_id")


  Y_test_df  = Y_df.groupby('unique_id').tail(4)
  Y_train_df = Y_df.drop(Y_test_df.index)


  fcst = StatsForecast(df=Y_train_df,
                       models=[AutoARIMA(season_length=4), Naive()],
                       freq='Q', n_jobs=-1)
  Y_hat_df = fcst.forecast(h=4)


  reconcilers = [
      BottomUp(),
      TopDown(method='forecast_proportions'),
      MiddleOut(middle_level='Country/Purpose/State',
                top_down_method='forecast_proportions')
  ]
  hrec = HierarchicalReconciliation(reconcilers=reconcilers)
  Y_rec_df = hrec.reconcile(Y_hat_df=Y_hat_df, Y_df=Y_train_df,
                            S=S, tags=tags)
  ```


  Assumes you have a test dataframe.

  ```python
  df = Y_rec_df.merge(Y_test_df, on=['unique_id', 'ds'])
  evaluation = evaluate(df = df,
                        tags = tags,
                        train_df = Y_train_df,
                        metrics = [mse],
                        benchmark = "Naive")
  ```


  Here is a link to the [documentation](https://nixtla.github.io/hierarchicalforecast/).


  This project is licensed under the MIT License - see the [LICENSE](https://github.com/Nixtla/neuralforecast/blob/main/LICENSE) file for details.


  In the R ecosystem, we recommend checking out [fable](http://fable.tidyverts.org/), and the now-retired [hts](https://github.com/earowang/hts).
  In Python we want to acknowledge the following libraries [hiere2e](https://github.com/rshyamsundar/gluonts-hierarchical-ICML-2021), [sktime](https://github.com/sktime/sktime-tutorial-pydata-berlin-2022), [darts](https://github.com/unit8co/darts), [pyhts](https://github.com/AngelPone/pyhts), [scikit-hts](https://github.com/carlomazzaferro/scikit-hts).


  This work is highly influenced by the fantastic work of previous contributors and other scholars who previously proposed the reconciliation methods presented here. We want to highlight the work of Rob Hyndman, George Athanasopoulos, Shanika L. Wickramasuriya, Souhaib Ben Taieb, and Bonsoo Koo. For a full reference link, please visit the Reference section of this [paper](https://arxiv.org/pdf/2207.03517.pdf).
  We encourage users to explore this [literature review](https://otexts.com/fpp3/hierarchical-reading.html).


  If you enjoy or benefit from using these Python implementations, a citation to this [hierarchical forecasting reference paper](https://arxiv.org/abs/2207.03517) will be greatly appreciated.
  ```bibtex
  @article{olivares2024hierarchicalforecastreferenceframeworkhierarchical,
        title={HierarchicalForecast: A Reference Framework for Hierarchical Forecasting in Python},
        author={Kin G. Olivares and Azul Garza and David Luo and Cristian Challú and Max Mergenthaler and Souhaib Ben Taieb and Shanika L. Wickramasuriya and Artur Dubrawski},
        year={2024},
        eprint={2207.03517},
        archivePrefix={arXiv},
        primaryClass={stat.ML},
        url={https://arxiv.org/abs/2207.03517},
  }
  ```


pkgId: 7651314625f5786bce102b644b673ec036031113a089e03ecba28e10e3cc98b9
name: python-hikerapi-help
arch: noarch
version: 1.6.0
epoch: 0
release: 1
summary: Development documents and examples for hikerapi
url: https://hikerapi.com
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-hikerapi-1.6.0-1.src.rpm
location_href: Packages/python-hikerapi-help-1.6.0-1.noarch.rpm
checksum_type: sha256
description:
  ```
  pip install hikerapi
  ```



  Create a token https://hikerapi.com/tokens and copy "Access key"

  ```python
  from hikerapi import Client

  cl = Client(token="<ACCESS_KEY>")
  user = cl.user_by_username_v2("instagram")
  print(user)
  ```

  ```python
  from hikerapi import AsyncClient

  cl = AsyncClient(token="<ACCESS_KEY>")
  user = await cl.user_by_username_v2("instagram")
  print(user)
  ```



  ```
  HIKERAPI_TOKEN=<token> pytest -v tests.py

  HIKERAPI_TOKEN=<token> pytest -v tests.py::test_search_music
  ```


pkgId: 774e598f920f7482d728de427b6c46b8e9362a2cfc6784403865ee1bf7332da3
name: python-hirundo-help
arch: noarch
version: 0.1.9
epoch: 0
release: 1
summary: Development documents and examples for hirundo
url: https://github.com/Hirundo-io/hirundo-client
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-hirundo-0.1.9-1.src.rpm
location_href: Packages/python-hirundo-help-0.1.9-1.noarch.rpm
checksum_type: sha256
description:
  This package exposes access to Hirundo APIs for dataset optimization for Machine Learning.

  Dataset optimization is currently available for datasets labelled for classification and object detection.


  Support dataset storage configs include:
     - Google Cloud (GCP) Storage
     - Amazon Web Services (AWS) S3
     - Git LFS (Large File Storage) repositories (e.g. GitHub or HuggingFace)

  Optimizing a classification dataset
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  Currently ``hirundo`` requires a CSV file with the following columns (all columns are required):
     - ``image_path``: The location of the image within the dataset ``root``
     - ``label``: The label of the image, i.e. which the class that was annotated for this image

  And outputs a CSV with the same columns and:
     - ``suspect_level``: mislabel suspect level
     - ``suggested_label``: suggested label
     - ``suggested_label_conf``: suggested label confidence

  Optimizing an object detection (OD) dataset
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  Currently ``hirundo`` requires a CSV file with the following columns (all columns are required):
     - ``image_path``: The location of the image within the dataset ``root``
     - ``bbox_id``: The index of the bounding box within the dataset. Used to indicate label suspects
     - ``label``: The label of the image, i.e. which the class that was annotated for this image
     - ``x1``, ``y1``, ``x2``, ``y2``: The bounding box coordinates of the object within the image

  And outputs a CSV with the same columns and:
     - ``suspect_level``: object mislabel suspect level
     - ``suggested_label``: suggested object label
     - ``suggested_label_conf``: suggested object label confidence

  Note: This Python package must be used alongside a Hirundo server, either the SaaS platform, a custom VPC deployment or an on-premises installation.




  You can install the codebase with a simple `pip install hirundo` to install the latest version of this package. If you prefer to install from the Git repository and/or need a specific version or branch, you can simply clone the repository, check out the relevant commit and then run `pip install .` to install that version. A full list of dependencies can be found in `requirements.txt`, but these will be installed automatically by either of these commands.



  Classification example:
  ```python
  from hirundo import (
      HirundoCSV,
      LabelingType,
      OptimizationDataset,
      StorageGCP,
      StorageConfig,
      StorageTypes,
  )

  gcp_bucket = StorageGCP(
      bucket_name="cifar100bucket",
      project="Hirundo-global",
      credentials_json=json.loads(os.environ["GCP_CREDENTIALS"]),
  )
  test_dataset = OptimizationDataset(
      name="TEST-GCP cifar 100 classification dataset",
      labeling_type=LabelingType.SINGLE_LABEL_CLASSIFICATION,
      storage_config=StorageConfig(
          name="cifar100bucket",
          type=StorageTypes.GCP,
          gcp=gcp_bucket,
      ),
      data_root_url=gcp_bucket.get_url(path="/pytorch-cifar/data"),
      labeling_info=HirundoCSV(
          csv_url=gcp_bucket.get_url(path="/pytorch-cifar/data/cifar100.csv"),
      ),
      classes=cifar100_classes,
  )

  test_dataset.run_optimization()
  results = test_dataset.check_run()
  print(results)
  ```


  Object detection example:

  ```python
  from hirundo import (
      GitRepo,
      HirundoCSV,
      LabelingType,
      OptimizationDataset,
      StorageGit,
      StorageConfig,
      StorageTypes,
  )

  git_storage = StorageGit(
      repo=GitRepo(
          name="BDD-100k-validation-dataset",
          repository_url="https://git@hf.co/datasets/hirundo-io/bdd100k-validation-only.git",
      ),
      branch="main",
  )
  test_dataset = OptimizationDataset(
      name="TEST-HuggingFace-BDD-100k-validation-OD-validation-dataset",
      labeling_type=LabelingType.OBJECT_DETECTION,
      storage_config=StorageConfig(
          name="BDD-100k-validation-dataset",
          type=StorageTypes.GIT,
          git=git_storage,
      ),
      data_root_url=git_storage.get_url(path="/BDD100K Val from Hirundo.zip/bdd100k"),
      labeling_info=HirundoCSV(
          csv_url=git_storage.get_url(
              path="/BDD100K Val from Hirundo.zip/bdd100k/bdd100k.csv"
          ),
      ),
      classes=[
          "traffic light",
          "traffic sign",
          "car",
          "pedestrian",
          "bus",
          "truck",
          "rider",
          "bicycle",
          "motorcycle",
          "train",
          "other vehicle",
          "other person",
          "trailer",
      ],
  )

  test_dataset.run_optimization()
  results = test_dataset.check_run()
  print(results)
  ```

  Note: Currently we only support the main CPython release 3.9, 3.10 and 3.11. PyPy support may be introduced in the future.



  To learn more about how to use this library, please visit the [http://docs.hirundo.io/](documentation) or see the Google Colab examples.

rpm_license:
  MIT License  Copyright (c) 2024, Hirundo  Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:  The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.


pkgId: c256efe86ddb8c7b8d6916f75ca26bb44e9ca6b38dd5b693036db82006ddb877
name: python-hjimi-tools-help
arch: noarch
version: 0.0.5
epoch: 0
release: 1
summary: Development documents and examples for hjimi-tools
url: https://github.com/zidanewenqsh/hjimi_utils
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-hjimi-tools-0.0.5-1.src.rpm
location_href: Packages/python-hjimi-tools-help-0.0.5-1.noarch.rpm
checksum_type: sha256
description:
  This Python class, `CProjectGenerator`, is a utility for automatically setting up a new C project structure. It creates a directory structure for the project, as well as several important files such as `main.c`, `Makefile` or `CMakeLists.txt` depending on the chosen build system.

  The `CProjectGenerator` class is initialized with the following parameters:
  - `project_name`: The name of the project to set up. This is a required argument.
  - `modules`: The names of the modules to be created. This is an optional argument and defaults to an empty string.
  - `build_system`: The build system to use. This can be either 'make' or 'cmake'. Defaults to 'make'.
  Example:
  ```python
  gen = CProjectGenerator("my_project", "module1 module2", "cmake")
  ```

  The following directory structure will be created:
  ```
  my_project/
  ├── src/
  │   └── main.c
  ├── include/
  ├── lib/
  ├── bin/
  ├── obj/
  ├── test/
  └── docs/
  ```

  The class will generate a `main.c` file in the `src/` directory as well as a `Makefile` or `CMakeLists.txt` in the project root directory depending on the chosen build system. If modules are specified, corresponding `.c` and `.h` files will be created in the `src/` and `include/` directories respectively.

  - `project_name`: (str) The name of the project to set up. This is a required argument.
  - `-b` or `--build`: (str, optional) The build system to use. It can either be 'make' or 'cmake'. The default is 'make'.
  - `-m` or `--modules`: (str, optional) A comma-separated list of module names. These modules will have corresponding `.c` and `.h` files created. If not specified, no modules will be created.

  To use this class to generate a project, you simply need to create an instance of the class and call the `generate_project` method. For example:
  ```python
  gen = CProjectGenerator("my_project", "module1,module2", "cmake")
  gen.generate_project()
  ```
  This will create a new project with the specified name, modules and build system.

  The FileCollector class is a utility designed to collect files with specific extensions from a list of given paths, recursively traverse directories to find these files, and write their contents into a Markdown file formatted with code blocks. It supports multiple file types, automatically determining the appropriate syntax highlighting for each based on the file extension, such as Python for .py files or JavaScript for .js files. The class handles multiple root directories and organizes the output by grouping files under their respective project root directories, using top-level headings for each project and secondary headings for each file. It writes the relative paths of the files, followed by their content in a code block, allowing users to document code and other text files in a structured and easily readable format. Additionally, it gracefully handles potential encoding errors when reading files by including an error message in the output. The entire process is executed through the run method, which first collects the files and then generates the Markdown output, ensuring the resulting file is ready for documentation or sharing purposes.

  - `paths`: List of file or directory paths to collect files from.
  - `extensions`: List of file extensions to collect.
  - `output_path`: Path to the output markdown file.

  ```python
  from hjimi_tools import FileCollector

  collector = FileCollector(paths=['/path/to/project'], extensions=['.py', '.md', '.txt'], output_path="output.md")

  collector.run()
  ```


pkgId: 8c1ed422790133f5e80054bc5956acea4d4d311c0fb0ffec4a62a1124142ab41
name: python-hk-cdp-help
arch: noarch
version: 1.0.24
epoch: 0
release: 1
summary: Development documents and examples for hk-cdp
url: http://gitlab.tdtech.gao7.com/newfire/hk_cap/hk_cdp/server/hk_cdp.git
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-hk-cdp-1.0.24-1.src.rpm
location_href: Packages/python-hk-cdp-help-1.0.24-1.noarch.rpm
checksum_type: sha256
description:
  To make it easy for you to get started with GitLab, here's a list of recommended next steps.

  Already a pro? Just edit this README.md and make it your own. Want to make it easy? [Use the template at the bottom](#editing-this-readme)!



  - [ ] [Create](https://docs.gitlab.com/ee/user/project/repository/web_editor.html#create-a-file) or [upload](https://docs.gitlab.com/ee/user/project/repository/web_editor.html#upload-a-file) files
  - [ ] [Add files using the command line](https://docs.gitlab.com/ee/gitlab-basics/add-file.html#add-a-file-using-the-command-line) or push an existing Git repository with the following command:

  ```
  cd existing_repo
  git remote add origin http://gitlab.tdtech.gao7.com/newfire/hk_cap/hk_cdp/server/hk_cdp.git
  git branch -M master
  git push -uf origin master
  ```



  - [ ] [Set up project integrations](http://gitlab.tdtech.gao7.com/newfire/hk_cap/hk_cdp/server/hk_cdp/-/settings/integrations)



  - [ ] [Invite team members and collaborators](https://docs.gitlab.com/ee/user/project/members/)
  - [ ] [Create a new merge request](https://docs.gitlab.com/ee/user/project/merge_requests/creating_merge_requests.html)
  - [ ] [Automatically close issues from merge requests](https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically)
  - [ ] [Enable merge request approvals](https://docs.gitlab.com/ee/user/project/merge_requests/approvals/)
  - [ ] [Automatically merge when pipeline succeeds](https://docs.gitlab.com/ee/user/project/merge_requests/merge_when_pipeline_succeeds.html)



  Use the built-in continuous integration in GitLab.

  - [ ] [Get started with GitLab CI/CD](https://docs.gitlab.com/ee/ci/quick_start/index.html)
  - [ ] [Analyze your code for known vulnerabilities with Static Application Security Testing(SAST)](https://docs.gitlab.com/ee/user/application_security/sast/)
  - [ ] [Deploy to Kubernetes, Amazon EC2, or Amazon ECS using Auto Deploy](https://docs.gitlab.com/ee/topics/autodevops/requirements.html)
  - [ ] [Use pull-based deployments for improved Kubernetes management](https://docs.gitlab.com/ee/user/clusters/agent/)
  - [ ] [Set up protected environments](https://docs.gitlab.com/ee/ci/environments/protected_environments.html)

  ***



  When you're ready to make this README your own, just edit this file and use the handy template below (or feel free to structure it however you want - this is just a starting point!). Thank you to [makeareadme.com](https://www.makeareadme.com/) for this template.


  Every project is different, so consider which of these sections apply to yours. The sections used in the template are suggestions for most open source projects. Also keep in mind that while a README can be too long and detailed, too long is better than too short. If you think your README is too long, consider utilizing another form of documentation rather than cutting out information.


  Choose a self-explaining name for your project.


  Let people know what your project can do specifically. Provide context and add a link to any reference visitors might be unfamiliar with. A list of Features or a Background subsection can also be added here. If there are alternatives to your project, this is a good place to list differentiating factors.


  On some READMEs, you may see small images that convey metadata, such as whether or not all the tests are passing for the project. You can use Shields to add some to your README. Many services also have instructions for adding a badge.


  Depending on what you are making, it can be a good idea to include screenshots or even a video (you'll frequently see GIFs rather than actual videos). Tools like ttygif can help, but check out Asciinema for a more sophisticated method.


  Within a particular ecosystem, there may be a common way of installing things, such as using Yarn, NuGet, or Homebrew. However, consider the possibility that whoever is reading your README is a novice and would like more guidance. Listing specific steps helps remove ambiguity and gets people to using your project as quickly as possible. If it only runs in a specific context like a particular programming language version or operating system or has dependencies that have to be installed manually, also add a Requirements subsection.


  Use examples liberally, and show the expected output if you can. It's helpful to have inline the smallest example of usage that you can demonstrate, while providing links to more sophisticated examples if they are too long to reasonably include in the README.


  Tell people where they can go to for help. It can be any combination of an issue tracker, a chat room, an email address, etc.


  If you have ideas for releases in the future, it is a good idea to list them in the README.


  State if you are open to contributions and what your requirements are for accepting them.

  For people who want to make changes to your project, it's helpful to have some documentation on how to get started. Perhaps there is a script that they should run or some environment variables that they need to set. Make these steps explicit. These instructions could also be useful to your future self.

  You can also document commands to lint the code or run tests. These steps help to ensure high code quality and reduce the likelihood that the changes inadvertently break something. Having instructions for running tests is especially helpful if it requires external setup, such as starting a Selenium server for testing in a browser.


  Show your appreciation to those who have contributed to the project.


  For open source projects, say how it is licensed.


  If you have run out of energy or time for your project, put a note at the top of the README saying that development has slowed down or stopped completely. Someone may choose to fork your project or volunteer to step in as a maintainer or owner, allowing your project to keep going. You can also make an explicit request for maintainers.


pkgId: 67cb505ae92fd7b5da0e06667a83979062e28a4d4074573f392815b4aaf477e0
name: python-hkfdb-help
arch: noarch
version: 4.88
epoch: 0
release: 1
summary: Development documents and examples for hkfdb
url: https://www.hkfdb.net
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-hkfdb-4.88-1.src.rpm
location_href: Packages/python-hkfdb-help-4.88-1.noarch.rpm
checksum_type: sha256
description:
  To install the latest version:
  ```
  pip install hkfdb
  ```

  Before you start, please make sure you have done the following:
  * **subscribe** the data from our website
  * get the personal authToken
  * create an client object from class Database
  * have fun!

  create client object with authToken and class Database:
  ```
  import hkfdb

  authToken = 'personal_authToken'
  client = hkfdb.Database(authToken)
  ```

  **Major Functions**

  Price Data:
  * get_hk_stock_ohlc()
  * get_us_stock_ohlc()
  * get_hk_fut_ohlc()
  * get_hk_deri_daily_market_report()

  CCASS Data:
  * get_ccass_all_id()
  * get_ccass_by_code()
  * get_ccass_holding_rank()
  * get_ccass_by_id()
  * get_ccass_by_id_change()

  Index list:
  * get_spx_index_const()
  * get_hk_index_const()
  * get_hk_stock_plate_const()
  * get_all_hk_index_name()
  * get_all_hk_stock_plate_name()

  Earning Calendar:
  * get_us_earning_calendar_by_date()
  * get_us_earning_calendar_by_code()
  * get_hk_earning_calendar_by_code()
  * get_hk_earning_calendar_by_date()

  Other:
  * get_hk_market_cap_hist()
  * get_hk_ipo_hist()
  * get_north_water()
  * get_market_highlight()
  * get_hk_buyback_by_date()
  * get_hk_buyback_by_code()
  * get_basic_hk_stock_info()


pkgId: 117eff7698a03483547d5cad0930835bab2d82224cfb39f489115f18d38b5955
name: python-hlink-help
arch: noarch
version: 3.8.0
epoch: 0
release: 1
summary: Development documents and examples for hlink
url: https://github.com/ipums/hlink
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-hlink-3.8.0-1.src.rpm
location_href: Packages/python-hlink-help-3.8.0-1.noarch.rpm
checksum_type: sha256
description:
  [![HLink Docker CI](https://github.com/ipums/hlink/actions/workflows/docker-build.yml/badge.svg)](https://github.com/ipums/hlink/actions/workflows/docker-build.yml)



  hlink is a Python package that provides a flexible, configuration-driven solution to probabilistic record linking at scale. It provides a high-level API for python as well as a standalone command line interface for running linking jobs with little to no programming. hlink supports the linking process from beginning to end, including preprocessing, filtering, training, model exploration, blocking, feature generation and scoring.

  It is used at [IPUMS](https://www.ipums.org/) to link U.S. historical census data, but can be applied to any record linkage job.
  A paper on the creation and applications of this program on historical census data can be found at <https://www.tandfonline.com/doi/full/10.1080/01615440.2021.1985027>.


  Wellington, J., R. Harper, and K.J. Thompson. 2022. "hlink." https://github.com/ipums/hlink: Institute for Social Research and Data Innovation, University of Minnesota.



  hlink requires

  - Python 3.10, 3.11, or 3.12
  - Java 8 or greater for integration with PySpark

  You can install the newest version of the Python package directly from PyPI with pip:
  ```
  pip install hlink
  ```

  We do our best to make hlink compatible with Python 3.10-3.12. If you have a
  problem using hlink on one of these versions of Python, please open an issue
  through GitHub. Versions of Python older than 3.10 are not supported.

  Note that PySpark 3.5 does not yet officially support Python 3.12. If you
  encounter PySpark-related import errors while running hlink on Python 3.12, try

  - Installing the setuptools package. The distutils package was deleted from the
    standard library in Python 3.12, but some versions of PySpark still import
    it. The setuptools package provides a hacky stand-in distutils library which
    should fix some import errors in PySpark. We install setuptools in our
    development and test dependencies so that our tests work on Python 3.12.

  - Downgrading Python to 3.10 or 3.11. PySpark officially supports these
    versions of Python. So you should have better chances getting PySpark to work
    well on Python 3.10 or 3.11.



  hlink has optional support for two additional machine learning algorithms,
  [XGBoost](https://xgboost.readthedocs.io/en/stable/index.html) and
  [LightGBM](https://lightgbm.readthedocs.io/en/latest/index.html). Both of these
  algorithms are highly performant gradient boosting libraries, each with its own
  characteristics. These algorithms are not implemented directly in Spark, so
  they require some additional dependencies. To install the required Python
  dependencies, run

  ```
  pip install hlink[xgboost]
  ```

  for XGBoost or

  ```
  pip install hlink[lightgbm]
  ```

  for LightGBM. If you would like to install both at once, you can run

  ```
  pip install hlink[xgboost,lightgbm]
  ```

  to get the Python dependencies for both. Both XGBoost and LightGBM also require
  libomp, which will need to be installed separately if you don't already have it.

  After installing the dependencies for one or both of these algorithms, you can
  use them as model types in training and model exploration. You can read more
  about these models in the hlink documentation [here](https://hlink.docs.ipums.org/models.html).

  *Note: The XGBoost-PySpark integration provided by the xgboost Python package is
  currently unstable. So the hlink xgboost support is experimental and may change
  in the future.*



  The documentation site can be found at [hlink.docs.ipums.org](https://hlink.docs.ipums.org).
  This includes information about installation and setting up your configuration files.

  An example script and configuration file can be found in the `examples` directory.



  The main class in the library is LinkRun, which represents a complete linking job. It provides access to each of the link tasks and their steps. Here is an example script that uses LinkRun to do some linking.

  ```python
  from hlink.linking.link_run import LinkRun
  from hlink.spark.factory import SparkFactory
  from hlink.configs.load_config import load_conf_file


  factory = SparkFactory()
  spark = factory.create()





  config = load_conf_file("./my_conf.toml")

  lr = LinkRun(spark, config)



  prep_steps = lr.preprocessing.get_steps()
  for (i, step) in enumerate(prep_steps):
      print(f"Step {i}:", step)
      print("Required input tables:", step.input_table_names)
      print("Generated output tables:", step.output_table_names)


  lr.preprocessing.run_all_steps()


  lr.matching.run_step(0)
  lr.matching.run_step(1)


  matches = lr.get_table("potential_matches")

  assert matches.exists()


  matches_df = matches.df()
  ```

  An example configuration file:

  ```toml




  id_column = "id"




  [datasource_a]
  alias = "a"
  file = "data/A.csv"

  [datasource_b]
  alias = "b"
  file = "data/B.csv"




  [[column_mappings]]
  column_name = "NAMEFRST"
  transforms = [
      {type = "lowercase_strip"}
  ]

  [[column_mappings]]
  column_name = "NAMELAST"
  transforms = [
      {type = "lowercase_strip"}
  ]

  [[column_mappings]]
  column_name = "AGE"
  transforms = [
      {type = "add_to_a", value = 10}
  ]

  [[column_mappings]]
  column_name = "SEX"









  [[blocking]]
  column_name = "SEX"

  [[blocking]]
  column_name = "AGE_2"
  dataset = "a"
  derived_from = "AGE"
  expand_length = 2
  explode = true








  [[comparison_features]]
  alias = "NAMEFRST_JW"
  column_name = "NAMEFRST"
  comparison_type = "jaro_winkler"

  [[comparison_features]]
  alias = "NAMELAST_JW"
  column_name = "NAMELAST"
  comparison_type = "jaro_winkler"







  [comparisons]
  operator = "AND"

  [comparisons.comp_a]
  comparison_type = "threshold"
  feature_name = "NAMEFRST_JW"
  threshold = 0.79

  [comparisons.comp_b]
  comparison_type = "threshold"
  feature_name = "NAMELAST_JW"
  threshold = 0.84
  ```

rpm_license:
  Mozilla Public License Version 2.0 ==================================  1. Definitions --------------  1.1. "Contributor" means each individual or legal entity that creates, contributes to the creation of, or owns Covered Software.  1.2. "Contributor Version" means the combination of the Contributions of others (if any) used by a Contributor and that particular Contributor's Contribution.  1.3. "Contribution" means Covered Software of a particular Contributor.  1.4. "Covered Software" means Source Code Form to which the initial Contributor has attached the notice in Exhibit A, the Executable Form of such Source Code Form, and Modifications of such Source Code Form, in each case including portions thereof.  1.5. "Incompatible With Secondary Licenses" means  (a) that the initial Contributor has attached the notice described in Exhibit B to the Covered Software; or  (b) that the Covered Software was made available under the terms of version 1.1 or earlier of the License, but not also under the terms of a Secondary License.  1.6. "Executable Form" means any form of the work other than Source Code Form.  1.7. "Larger Work" means a work that combines Covered Software with other material, in a separate file or files, that is not Covered Software.  1.8. "License" means this document.  1.9. "Licensable" means having the right to grant, to the maximum extent possible, whether at the time of the initial grant or subsequently, any and all of the rights conveyed by this License.  1.10. "Modifications" means any of the following:  (a) any file in Source Code Form that results from an addition to, deletion from, or modification of the contents of Covered Software; or  (b) any new file in Source Code Form that contains any Covered Software.  1.11. "Patent Claims" of a Contributor means any patent claim(s), including without limitation, method, process, and apparatus claims, in any patent Licensable by such Contributor that would be infringed, but for the grant of the License, by the making, using, selling, offering for sale, having made, import, or transfer of either its Contributions or its Contributor Version.  1.12. "Secondary License" means either the GNU General Public License, Version 2.0, the GNU Lesser General Public License, Version 2.1, the GNU Affero General Public License, Version 3.0, or any later versions of those licenses.  1.13. "Source Code Form" means the form of the work preferred for making modifications.  1.14. "You" (or "Your") means an individual or a legal entity exercising rights under this License. For legal entities, "You" includes any entity that controls, is controlled by, or is under common control with You. For purposes of this definition, "control" means (a) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (b) ownership of more than fifty percent (50%) of the outstanding shares or beneficial ownership of such entity.  2. License Grants and Conditions --------------------------------  2.1. Grants  Each Contributor hereby grants You a world-wide, royalty-free, non-exclusive license:  (a) under intellectual property rights (other than patent or trademark) Licensable by such Contributor to use, reproduce, make available, modify, display, perform, distribute, and otherwise exploit its Contributions, either on an unmodified basis, with Modifications, or as part of a Larger Work; and  (b) under Patent Claims of such Contributor to make, use, sell, offer for sale, have made, import, and otherwise transfer either its Contributions or its Contributor Version.  2.2. Effective Date  The licenses granted in Section 2.1 with respect to any Contribution become effective for each Contribution on the date the Contributor first distributes such Contribution.  2.3. Limitations on Grant Scope  The licenses granted in this Section 2 are the only rights granted under this License. No additional rights or licenses will be implied from the distribution or licensing of Covered Software under this License. Notwithstanding Section 2.1(b) above, no patent license is granted by a Contributor:  (a) for any code that a Contributor has removed from Covered Software; or  (b) for infringements caused by: (i) Your and any other third party's modifications of Covered Software, or (ii) the combination of its Contributions with other software (except as part of its Contributor Version); or  (c) under Patent Claims infringed by Covered Software in the absence of its Contributions.  This License does not grant any rights in the trademarks, service marks, or logos of any Contributor (except as may be necessary to comply with the notice requirements in Section 3.4).  2.4. Subsequent Licenses  No Contributor makes additional grants as a result of Your choice to distribute the Covered Software under a subsequent version of this License (see Section 10.2) or under the terms of a Secondary License (if permitted under the terms of Section 3.3).  2.5. Representation  Each Contributor represents that the Contributor believes its Contributions are its original creation(s) or it has sufficient rights to grant the rights to its Contributions conveyed by this License.  2.6. Fair Use  This License is not intended to limit any rights You have under applicable copyright doctrines of fair use, fair dealing, or other equivalents.  2.7. Conditions  Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted in Section 2.1.  3. Responsibilities -------------------  3.1. Distribution of Source Form  All distribution of Covered Software in Source Code Form, including any Modifications that You create or to which You contribute, must be under the terms of this License. You must inform recipients that the Source Code Form of the Covered Software is governed by the terms of this License, and how they can obtain a copy of this License. You may not attempt to alter or restrict the recipients' rights in the Source Code Form.  3.2. Distribution of Executable Form  If You distribute Covered Software in Executable Form then:  (a) such Covered Software must also be made available in Source Code Form, as described in Section 3.1, and You must inform recipients of the Executable Form how they can obtain a copy of such Source Code Form by reasonable means in a timely manner, at a charge no more than the cost of distribution to the recipient; and  (b) You may distribute such Executable Form under the terms of this License, or sublicense it under different terms, provided that the license for the Executable Form does not attempt to limit or alter the recipients' rights in the Source Code Form under this License.  3.3. Distribution of a Larger Work  You may create and distribute a Larger Work under terms of Your choice, provided that You also comply with the requirements of this License for the Covered Software. If the Larger Work is a combination of Covered Software with a work governed by one or more Secondary Licenses, and the Covered Software is not Incompatible With Secondary Licenses, this License permits You to additionally distribute such Covered Software under the terms of such Secondary License(s), so that the recipient of the Larger Work may, at their option, further distribute the Covered Software under the terms of either this License or such Secondary License(s).  3.4. Notices  You may not remove or alter the substance of any license notices (including copyright notices, patent notices, disclaimers of warranty, or limitations of liability) contained within the Source Code Form of the Covered Software, except that You may alter any license notices to the extent required to remedy known factual inaccuracies.  3.5. Application of Additional Terms  You may choose to offer, and to charge a fee for, warranty, support, indemnity or liability obligations to one or more recipients of Covered Software. However, You may do so only on Your own behalf, and not on behalf of any Contributor. You must make it absolutely clear that any such warranty, support, indemnity, or liability obligation is offered by You alone, and You hereby agree to indemnify every Contributor for any liability incurred by such Contributor as a result of warranty, support, indemnity or liability terms You offer. You may include additional disclaimers of warranty and limitations of liability specific to any jurisdiction.  4. Inability to Comply Due to Statute or Regulation ---------------------------------------------------  If it is impossible for You to comply with any of the terms of this License with respect to some or all of the Covered Software due to statute, judicial order, or regulation then You must: (a) comply with the terms of this License to the maximum extent possible; and (b) describe the limitations and the code they affect. Such description must be placed in a text file included with all distributions of the Covered Software under this License. Except to the extent prohibited by statute or regulation, such description must be sufficiently detailed for a recipient of ordinary skill to be able to understand it.  5. Termination --------------  5.1. The rights granted under this License will terminate automatically if You fail to comply with any of its terms. However, if You become compliant, then the rights granted under this License from a particular Contributor are reinstated (a) provisionally, unless and until such Contributor explicitly and finally terminates Your grants, and (b) on an ongoing basis, if such Contributor fails to notify You of the non-compliance by some reasonable means prior to 60 days after You have come back into compliance. Moreover, Your grants from a particular Contributor are reinstated on an ongoing basis if such Contributor notifies You of the non-compliance by some reasonable means, this is the first time You have received notice of non-compliance with this License from such Contributor, and You become compliant prior to 30 days after Your receipt of the notice.  5.2. If You initiate litigation against any entity by asserting a patent infringement claim (excluding declaratory judgment actions, counter-claims, and cross-claims) alleging that a Contributor Version directly or indirectly infringes any patent, then the rights granted to You by any and all Contributors for the Covered Software under Section 2.1 of this License shall terminate.  5.3. In the event of termination under Sections 5.1 or 5.2 above, all end user license agreements (excluding distributors and resellers) which have been validly granted by You or Your distributors under this License prior to termination shall survive termination.  ************************************************************************ *                                                                      * *  6. Disclaimer of Warranty                                           * *  -------------------------                                           * *                                                                      * *  Covered Software is provided under this License on an "as is"       * *  basis, without warranty of any kind, either expressed, implied, or  * *  statutory, including, without limitation, warranties that the       * *  Covered Software is free of defects, merchantable, fit for a        * *  particular purpose or non-infringing. The entire risk as to the     * *  quality and performance of the Covered Software is with You.        * *  Should any Covered Software prove defective in any respect, You     * *  (not any Contributor) assume the cost of any necessary servicing,   * *  repair, or correction. This disclaimer of warranty constitutes an   * *  essential part of this License. No use of any Covered Software is   * *  authorized under this License except under this disclaimer.         * *                                                                      * ************************************************************************  ************************************************************************ *                                                                      * *  7. Limitation of Liability                                          * *  --------------------------                                          * *                                                                      * *  Under no circumstances and under no legal theory, whether tort      * *  (including negligence), contract, or otherwise, shall any           * *  Contributor, or anyone who distributes Covered Software as          * *  permitted above, be liable to You for any direct, indirect,         * *  special, incidental, or consequential damages of any character      * *  including, without limitation, damages for lost profits, loss of    * *  goodwill, work stoppage, computer failure or malfunction, or any    * *  and all other commercial damages or losses, even if such party      * *  shall have been informed of the possibility of such damages. This   * *  limitation of liability shall not apply to liability for death or   * *  personal injury resulting from such party's negligence to the       * *  extent applicable law prohibits such limitation. Some               * *  jurisdictions do not allow the exclusion or limitation of           * *  incidental or consequential damages, so this exclusion and          * *  limitation may not apply to You.                                    * *                                                                      * ************************************************************************  8. Litigation -------------  Any litigation relating to this License may be brought only in the courts of a jurisdiction where the defendant maintains its principal place of business and such litigation shall be governed by laws of that jurisdiction, without reference to its conflict-of-law provisions. Nothing in this Section shall prevent a party's ability to bring cross-claims or counter-claims.  9. Miscellaneous ----------------  This License represents the complete agreement concerning the subject matter hereof. If any provision of this License is held to be unenforceable, such provision shall be reformed only to the extent necessary to make it enforceable. Any law or regulation which provides that the language of a contract shall be construed against the drafter shall not be used to construe this License against a Contributor.  10. Versions of the License ---------------------------  10.1. New Versions  Mozilla Foundation is the license steward. Except as provided in Section 10.3, no one other than the license steward has the right to modify or publish new versions of this License. Each version will be given a distinguishing version number.  10.2. Effect of New Versions  You may distribute the Covered Software under the terms of the version of the License under which You originally received the Covered Software, or under the terms of any subsequent version published by the license steward.  10.3. Modified Versions  If you create software not governed by this License, and you want to create a new license for such software, you may create and use a modified version of this License if you rename the license and remove any references to the name of the license steward (except to note that such modified license differs from this License).  10.4. Distributing Source Code Form that is Incompatible With Secondary Licenses  If You choose to distribute Source Code Form that is Incompatible With Secondary Licenses under the terms of this version of the License, the notice described in Exhibit B of this License must be attached.  Exhibit A - Source Code Form License Notice -------------------------------------------  This Source Code Form is subject to the terms of the Mozilla Public License, v. 2.0. If a copy of the MPL was not distributed with this file, You can obtain one at http://mozilla.org/MPL/2.0/.  If it is not possible or desirable to put the notice in a particular file, then You may include the notice in a location (such as a LICENSE file in a relevant directory) where a recipient would be likely to look for such a notice.  You may add additional accurate notices of copyright ownership.  Exhibit B - "Incompatible With Secondary Licenses" Notice ---------------------------------------------------------  This Source Code Form is "Incompatible With Secondary Licenses", as defined by the Mozilla Public License, v. 2.0.


pkgId: 6f1913424222ea5b035091683ad3f50bb45a90af963a3d5c9fc2245e2be9c5f1
name: python-hlvox-help
arch: noarch
version: 0.8.5
epoch: 0
release: 1
summary: Development documents and examples for hlvox
url: https://gitlab.com/bhagen/hlvox
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-hlvox-0.8.5-1.src.rpm
location_href: Packages/python-hlvox-help-0.8.5-1.noarch.rpm
checksum_type: sha256
description:
  [![pipeline status](https://gitlab.com/bhagen/hlvox/badges/master/pipeline.svg)](https://gitlab.com/bhagen/hlvox/commits/master)
  [![coverage report](https://gitlab.com/bhagen/hlvox/badges/master/coverage.svg)](https://gitlab.com/bhagen/hlvox/commits/master)
  [![PyPI version](https://badge.fury.io/py/hlvox.svg)](https://badge.fury.io/py/hlvox)

  Originally intended to create sentences from the word snippets for the
  Half Life 1 Vox, this project can take a folder of word audio files and piece
  them into sentences.
  This repo when used standalone (ie by running `voice.py` directly) will output
  the sentences as audio files.



  To use this project, you will first need a folder full of voice files.
  If you have Half Life 1 installed, the voices can be extracted from the VPK
  files using something like GCFScape.



  Check pipenv file for Python package requirements.

  An installation of FFMpeg is required. On Ubuntu, just run the following:

  ```
  apt update
  apt install ffmpeg
  ```

  For persistence, a local `sqlite` database can be used, or a `postgresql` database. These are not exposed through the basic CLI.



  How to set up a basic dev environment:

  Pull this repo

  ```
  git pull https://gitlab.com/bhagen/hlvox.git
  ```

  Change into the new repo directory

  ```
  cd hlvox
  ```

  Install requirements using pipenv
  ```
  pipenv install
  ```

  Enter pipenv
  ```
  pipenv shell
  ```



  If you want to jump right into generating a sentence, you can point `voice.py`
  at a folder of voice audio files!

  ```
  python hlvox/voice.py --voice-dir ~/my_folder_of_voice_files "the sentence I want to generate"
  ```

  Your sentence will be exported in the same directory
  that you ran the file from.


  After entering pipenv shell:
  ```
  pip install --editable .
  ```



  Just run `pytest` from the base directory of the repo



  I used [Sphinx](http://www.sphinx-doc.org) to create autodoc documentation
  from docstrings in the modules.

  Change into doc directory
  ```
  cd doc
  ```

  (If you've modified the source code, grab the new apidocs
  ```
  sphinx-apidoc -f -o ./source ../hlvox
  ```


  Generate documentation
  ```
  make html
  ```




  * **Blair Hagen** - *Initial work* - [bhagen](https://github.com/bhagen)




  This project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md) file for details



  * Thanks to Valve for making such an amazing voice


pkgId: 405272ac22979ccf896fb947b37159eefcf1cee5d7cd01543d8afefbeb106a0f
name: python-hmdriver2-help
arch: noarch
version: 1.4.1
epoch: 0
release: 1
summary: Development documents and examples for hmdriver2
url: https://pypi.org/project/hmdriver2/
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-hmdriver2-1.4.1-1.src.rpm
location_href: Packages/python-hmdriver2-help-1.4.1-1.noarch.rpm
checksum_type: sha256
description:
  - [API Documents](#api-documents)
    - [初始化Driver](#初始化driver)
    - [App管理](#app管理)
      - [安装App](#安装app)
      - [卸载App](#卸载app)
      - [启动App](#启动app)
      - [停止App](#停止app)
      - [清除App数据](#清除app数据)
      - [获取App详情](#获取app详情)
    - [设备操作](#设备操作)
      - [获取设备信息](#获取设备信息)
      - [获取设备分辨率](#获取设备分辨率)
      - [获取设备旋转状态](#获取设备旋转状态)
      - [设置设备旋转](#设置设备旋转)
      - [Home](#home)
      - [返回](#返回)
      - [亮屏](#亮屏)
      - [息屏](#息屏)
      - [屏幕解锁](#屏幕解锁)
      - [Key Events](#key-events)
      - [执行 HDC 命令](#执行-hdc-命令)
      - [打开URL (schema)](#打开url-schema)
      - [文件操作](#文件操作)
      - [屏幕截图](#屏幕截图)
      - [屏幕录屏](#屏幕录屏)
      - [Device Touch](#device-touch)
        - [单击](#单击)
        - [双击](#双击)
        - [长按](#长按)
        - [滑动](#滑动)
        - [滑动 ext](#滑动-ext)
        - [输入](#输入)
        - [复杂手势](#复杂手势)
    - [控件操作](#控件操作)
      - [常规选择器](#常规选择器)
        - [控件查找](#控件查找)
        - [控件信息](#控件信息)
        - [控件数量](#控件数量)
        - [控件点击](#控件点击)
        - [控件双击](#控件双击)
        - [控件长按](#控件长按)
        - [控件拖拽](#控件拖拽)
        - [控件缩放](#控件缩放)
        - [控件输入](#控件输入)
        - [文本清除](#文本清除)
      - [XPath选择器](#xpath选择器)
        - [xpath控件是否存在](#xpath控件是否存在)
        - [xpath控件点击](#xpath控件点击)
        - [xpath控件双击](#xpath控件双击)
        - [xpath控件长按](#xpath控件长按)
        - [xpath控件输入](#xpath控件输入)
    - [获取控件树](#获取控件树)
    - [获取Toast](#获取toast)

  ```python
  from hmdriver2.driver import Driver
  d = Driver()

  ```
  参数`serial` 通过`hdc list targets` 命令获取；如果不传serial，则默认读取`hdc list targets`的第一个设备
  初始化driver后，下面所有的操作都是调用dirver实现


  ```python
  d.install_app("/Users/develop/harmony_prj/demo.hap")
  ```

  ```python
  d.uninstall_app("com.kuaishou.hmapp")
  ```
  传入的参数是`package_name`，可通过hdc命令获取`hdc shell bm dump -a`

  ```python
  d.start_app("com.kuaishou.hmapp", "EntryAbility")
  ```
  传入的两个参数分别是`package_name`, `page_name`，可以通过hdc命令获取`hdc shell aa dump -l`

  ```python
  d.stop_app("com.kuaishou.hmapp")
  ```

  ```python
  d.clear_app("com.kuaishou.hmapp")
  ```
  该方法表示清除App数据和缓存

  ```python
  d.get_app_info("com.kuaishou.hmapp")
  ```
  输出的数据结构是Dict, 内容如下
  ```bash
  {
      "appId": "com.kuaishou.hmapp_BIS88rItfUAk+V9Y4WZp2HgIZ/JeOgvEBkwgB/YyrKiwrWhje9Xn2F6Q7WKFVM22RdIR4vFsG14A7ombgQmIIxU=",
      "appIdentifier": "5765880207853819885",
      "applicationInfo": {
          "bundleName": "com.kuaishou.hmapp",
          "codePath": "/data/app/el1/bundle/public/com.kuaishou.hmapp",
          "compileSdkType": "HarmonyOS",
          "compileSdkVersion": "4.1.0.73",
          "cpuAbi": "arm64-v8a",
          "deviceId": "PHONE-001",
          "vendor": "快手",
          "versionCode": 999999,
          "versionName": "12.2.40"
      },
      "compatibleVersion": 40100011,
      "cpuAbi": "",
      "hapModuleInfos": [
      ],
      "reqPermissions": [
          "ohos.permission.ACCELEROMETER",
          "ohos.permission.GET_NETWORK_INFO",
          "ohos.permission.GET_WIFI_INFO",
          "ohos.permission.INTERNET",
      ],
      "vendor": "快手",
      "versionCode": 999999,
      "versionName": "12.2.40"
  }
  ```


  ```python
  from hmdriver2.proto import DeviceInfo
  info: DeviceInfo = d.device_info
  ```
  输入内容如下
  ```bash
  DeviceInfo(productName='HUAWEI Mate 60 Pro', model='ALN-AL00', sdkVersion='12', sysVersion='ALN-AL00 5.0.0.60(SP12DEVC00E61R4P9log)', cpuAbi='arm64-v8a', wlanIp='172.31.125.111', displaySize=(1260, 2720), displayRotation=<DisplayRotation.ROTATION_0: 0>)
  ```
  然后就可以获取你想要的值, 比如
  ```python
  info.productName
  info.model
  info.wlanIp
  info.sdkVersion
  info.sysVersion
  info.cpuAbi
  info.displaySize
  info.displayRotation
  ```

  ```python
  w, h = d.display_size

  ```

  ```python
  from hmdriver2.proto import DisplayRotation
  rotation = d.display_rotation

  ```
  设备旋转状态包括：
  ```python
  ROTATION_0 = 0    # 未旋转
  ROTATION_90 = 1  # 顺时针旋转90度
  ROTATION_180 = 2  # 顺时针旋转180度
  ROTATION_270 = 3  # 顺时针旋转270度
  ```

  ```python
  from hmdriver2.proto import DisplayRotation

  d.set_display_rotation(DisplayRotation.ROTATION_180)
  ```

  ```python
  d.go_home()
  ```

  ```python
  d.go_back()
  ```

  ```python
  d.screen_on()
  ```

  ```python
  d.screen_off()
  ```

  ```python
  d.unlock()
  ```

  ```python
  from hmdriver2.proto import KeyCode
  d.press_key(KeyCode.POWER)
  ```
  详细的Key code请参考 [harmony key code](https://github.com/codematrixer/hmdriver2/blob/4d7bceaded947bd63d737de180064679ad4c77b8/hmdriver2/proto.py#L133)

  ```python
  data = d.shell("ls -l /data/local/tmp")
  print(data.output)
  ```
  这个方法等价于执行  `hdc shell ls -l /data/local/tmp`
  Notes: `HDC`详细的命令解释参考：[awesome-hdc](https://github.com/codematrixer/awesome-hdc)

  ```python
  d.open_url("http://www.baidu.com")
  d.open_url("kwai://myprofile")
  ```

  ```python

  d.pull_file(rpath, lpath)

  d.push_file(lpath, rpath)
  ```
  参数`rpath`表示手机端文件路径，`lpath`表示本地电脑文件路径

  ```python
  d.screenshot(path)
  ```
  参数`path`表示截图保存在本地电脑的文件路径

  方式一
  ```python

  d.screenrecord.start("test.mp4")

  time.sleep(5)

  d.screenrecord.stop()
  ```
  上述方式如果录屏过程中，脚本出现异常时，`stop`无法被调用，导致资源泄漏，需要加上try catch
  【推荐】方式二  ⭐️⭐️⭐️⭐️⭐️
  ```python
  with d.screenrecord.start("test2.mp4"):
      
      time.sleep(5)
  ```
  通过上下文语法，在录屏结束时框架会自动调用`stop` 清理资源
  Notes: 使用屏幕录屏需要依赖`opencv-python`
  ```bash
  pip3 install -U "hmdriver[opencv-python]"
  ```


  ```python
  d.click(x, y)

  d.click(200, 300)
  d.click(0.4, 0.6)
  ```
  参数`x`, `y`表示点击的坐标，可以为绝对坐标值，也可以为相当坐标（屏幕百分比）

  ```python
  d.double_click(x, y)

  d.double_click(500, 1000)
  d.double_click(0.5, 0.4)
  ```

  ```python
  d.long_click(x, y)

  d.long_click(500, 1000)
  d.long_click(0.5, 0.4)
  ```

  ```python
  d.swipe(x1, y1, x2, y2, spped)

  d.swipe(600, 2600, 600, 1200, speed=2000)  # 上滑
  d.swipe(0.5, 0.8, 0.5, 0.4, speed=2000)
  ```
  - `x1`, `y1`表示滑动的起始点，`x2`, `y2`表示滑动的终点
  - `speed`为滑动速率, 范围:200~40000, 不在范围内设为默认值为2000, 单位: 像素点/秒

  ```python
  d.swipe_ext("up")  # 向上滑动，"left", "right", "up", "down"
  d.swipe_ext("right", scale=0.8)  # 向右滑动，滑动距离为屏幕宽度的80%
  d.swipe_ext("up", box=(0.2, 0.2, 0.8, 0.8))  # 在屏幕 (0.2, 0.2) -> (0.8, 0.8) 这个区域上滑

  from hmdriver2.proto import SwipeDirection
  d.swipe_ext(SwipeDirection.DOWN)  # 向下滑动
  ```
  - `direction`表示滑动方向，可以为`up`, `down`, `left`, `right`, 也可以为`SwipeDirection`的枚举值
  - `scale`表示滑动距离百分比，范围:0.1~1.0, 默认值为0.8
  - `box`表示滑动区域，格式为`(x1, y1, x2, y2)`, 表示滑动区域的左上角和右下角的坐标，可以为绝对坐标值，也可以为相当坐标（屏幕百分比）
  Notes: `swipe_ext`和`swipe`的区别在于swipe_ext可以指定滑动区域，并且可以指定滑动方向，更简洁灵活

  复杂手势就是手指按下`start`，移动`move`，暂停`pause`的集合，最后运行`action`
  ```python
  g = d.gesture
  g.start(x1, y1, interval=0.5)
  g.move(x2, y2)
  g.pause(interval=1)
  g.move(x3, y3)
  g.action()
  ```
  也支持链式调用（推荐）
  ```python
  d.gesture.start(x1, y1, interval=.5).move(x2, y2).pause(interval=1).move(x3, y3).action()
  ```
  参数`x`, `y`表示坐标位置，可以为绝对坐标值，也可以为相当坐标（屏幕百分比），`interval`表示手势持续的时间，单位秒。
  如果只有start手势，则等价于点击：
  ```python
  d.gesture.start(x, y).action() # click

  d.click(x, y)
  ```
  *如下是一个复杂手势的效果展示*
  ![Watch the gif](https://i.ibb.co/PC76PRD/gesture.gif)

  ```python
  d.input_text(text)

  d.input_text("adbcdfg")
  ```
  参数`x`, `y`表示输入的位置，`text`表示输入的文本


  控件查找支持这些`by`属性
  - `id`
  - `key`
  - `text`
  - `type`
  - `description`
  - `clickable`
  - `longClickable`
  - `scrollable`
  - `enabled`
  - `focused`
  - `selected`
  - `checked`
  - `checkable`
  - `isBefore`
  - `isAfter`
  Notes: 获取控件属性值可以配合 [UI inspector](https://github.com/codematrixer/ui-viewer) 工具查看
  **普通定位**
  ```python
  d(text="tab_recrod")
  d(id="drag")

  d(type="Button", index=0)
  ```
  Notes：当同一界面有多个属性相同的元素时，`index`属性非常实用
  **模糊定位TODO**
  **组合定位**
  指定多个`by`属性进行元素定位
  ```python

  d(type="Button", text="tab_recrod")
  ```
  **相对定位**
  ```python

  d(text="showToast", isAfter=True)

  d(id="drag", isBefore=True)
  ```

  结合上面讲的控件选择器，就可以进行元素的查找
  ```python
  d(text="tab_recrod").exists()
  d(type="Button", text="tab_recrod").exists()
  d(text="tab_recrod", isAfter=True).exists()

  d(text="tab_recrod").find_component()

  ```

  ```python
  d(text="tab_recrod").info

  {
      "id": "",
      "key": "",
      "type": "Button",
      "text": "tab_recrod",
      "description": "",
      "isSelected": False,
      "isChecked": False,
      "isEnabled": True,
      "isFocused": False,
      "isCheckable": False,
      "isClickable": True,
      "isLongClickable": False,
      "isScrollable": False,
      "bounds": {
          "left": 539,
          "top": 1282,
          "right": 832,
          "bottom": 1412
      },
      "boundsCenter": {
          "x": 685,
          "y": 1347
      }
  }
  ```
  也可以单独调用对应的属性
  ```python
  d(text="tab_recrod").id
  d(text="tab_recrod").key
  d(text="tab_recrod").type
  d(text="tab_recrod").text
  d(text="tab_recrod").description
  d(text="tab_recrod").isSelected
  d(text="tab_recrod").isChecked
  d(text="tab_recrod").isEnabled
  d(text="tab_recrod").isFocused
  d(text="tab_recrod").isCheckable
  d(text="tab_recrod").isClickable
  d(text="tab_recrod").isLongClickable
  d(text="tab_recrod").isScrollable
  d(text="tab_recrod").bounds
  d(text="tab_recrod").boundsCenter
  ```

  ```python
  d(type="Button").count   # 输出当前页面`type`为Button的元素数量

  len(d(type="Button"))
  ```

  ```python
  d(text="tab_recrod").click()
  d(type="Button", text="tab_recrod").click()
  d(text="tab_recrod").click_if_exists()
  ```
  以上两个方法有一定的区别
  - `click` 如果元素没找到，会报错`ElementNotFoundError`
  - `click_if_exists` 即使元素没有找到，也不会报错，相当于跳过

  ```python
  d(text="tab_recrod").double_click()
  d(type="Button", text="tab_recrod").double_click()
  ```

  ```python
  d(text="tab_recrod").long_click()
  d(type="Button", text="tab_recrod").long_click()
  ```

  ```python
  from hmdriver2.proto import ComponentData
  componentB: ComponentData = d(type="ListItem", index=1).find_component()

  d(type="ListItem").drag_to(componentB)
  ```
  `drag_to`的参数`component`为`ComponentData`类型

  ```python

  d(text="tab_recrod").pinch_in(scale=0.5)

  d(text="tab_recrod").pinch_out(scale=2)
  ```
  其中`scale`参数为放大和缩小比例

  ```python
  d(text="tab_recrod").input_text("abc")
  ```

  ```python
  d(text="tab_recrod").clear_text()
  ```

  xpath选择器基于标准的xpath规范，也可以使用`//*[@属性="属性值"]`的样式（xpath lite）
  ```python
  d.xpath('//root[1]/Row[1]/Column[1]/Row[1]/Button[3]')
  d.xpath('//*[@text="showDialog"]')
  ```
  控件xpath路径获取可以配合 [UI inspector](https://github.com/codematrixer/ui-viewer) 工具查看

  ```python
  d.xpath('//*[@text="showDialog"]').exists()   # 返回True/False
  d.xpath('//root[1]/Row[1]/Column[1]/Row[1]/Button[3]').exists()
  ```

  ```python
  d.xpath('//*[@text="showDialog"]').click()
  d.xpath('//root[1]/Row[1]/Column[1]/Row[1]/Button[3]').click_if_exists()
  ```
  以上两个方法有一定的区别
  - `click` 如果元素没找到，会报错`XmlElementNotFoundError`
  - `click_if_exists` 即使元素没有找到，也不会报错，相当于跳过

  ```python
  d.xpath('//*[@text="showDialog"]').double_click()
  ```

  ```python
  d.xpath('//*[@text="showDialog"]').long_click()
  ```

  ```python
  d.xpath('//*[@text="showDialog"]').input_text("adb")
  ```

  ```python
  d.dump_hierarchy()
  ```
  输出控件树格式参考 [hierarchy.json](/docs/hierarchy.json)

  ```python

  d.toast_watcher.start()

  d(text="xx").click()

  toast = d.toast_watcher.get_toast()

  ```

  See [DEVELOP.md](/docs/DEVELOP.md)

  [hmdriver2 发布：开启鸿蒙 NEXT 自动化新时代](https://testerhome.com/topics/40667)

  [Contributors](https://github.com/codematrixer/hmdriver2/graphs/contributors)

  - https://developer.huawei.com/consumer/cn/doc/harmonyos-guides-V5/ut-V5
  - https://github.com/codematrixer/awesome-hdc
  - https://github.com/openatx/uiautomator2
  - https://github.com/mrx1203/hmdriver


pkgId: d49096c60be279a58a1d4c777e164a0be5809f3de42fb6d8f6b61e61e1ee28bc
name: python-hmp-help
arch: noarch
version: 0.5.0
epoch: 0
release: 1
summary: Development documents and examples for hmp
url: https://github.com/GWeindel/hmp
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-hmp-0.5.0-1.src.rpm
location_href: Packages/python-hmp-help-0.5.0-1.noarch.rpm
checksum_type: sha256
description:
  ![](plots/general_illustration.png)
  > [!NOTE]
  > The paper on HMP was accepted at Imaging Neuroscience, see the latest version on BiorXiv: [Trial-by-trial detection of cognitive events in neural time-series](https://www.biorxiv.org/content/10.1101/2024.02.13.580102v3)
  HMP is an open-source Python package to analyze neural time-series (e.g. EEG) to estimate Hidden Multivariate Patterns (HMP).  HMP is described in Weindel, van Maanen & Borst (2024, [see the preprint on biorXiv](https://www.biorxiv.org/content/10.1101/2024.02.13.580102v1)) and is a generalized and simplified version of the HsMM-MVPA method developed by Anderson, Zhang, Borst, & Walsh  ([2016](https://psycnet.apa.org/doi/10.1037/rev0000030)).
  As a summary of the method, an HMP model parses the reaction time into a number of successive events determined based on patterns in a neural time-serie. Hence any reaction time can then be described by a number of cognitive events and the duration between them estimated using HMP. The important aspect of HMP is that it is a whole-brain analysis (or whole scalp analysis) that estimates the onset of events on a single-trial basis. These by-trial estimates allow you then to further dig into any aspect you are interested in a signal:
  - Describing an experiment or a clinical sample in terms of events detected in the EEG signal
  - Describing experimental effects based on the time onset of a particular event
  - Estimating the effect of trial-wise manipulations on the identified event presence and time occurrence (e.g. the by-trial variation of stimulus strength or the effect of time-on-task)
  - Time-lock EEG signal to the onset of a given event and perform classical ERPs or time-frequency analysis based on the onset of a new event
  - And many more (e.g. evidence accumulation models, classification based on the number of events in the signal,...)

  The package is available through *pip*.
  A way of using the package is to use a conda environment (see [anaconda](https://www.anaconda.com/products/distribution>) for how to install conda):
  ```bash
      conda create -n hmp
      conda activate hmp
      conda install pip #if not already installed
      pip install hmp #or cloning from github, see below
  ```
  For the cutting edge version you can clone the repository using *git*
  Open a terminal and type:
      $ git clone https://github.com/gweindel/hmp.git
  Then move to the clone repository and run
      $ pip install -e .
  After either installation path you can import hmp in your favorite python IDE through:
  ```python
      import hmp
  ```

  To get started with the code:
  - Check the demo below
  - Inspect the tutorials in the tutorials repository
      - [Load EEG data ](tutorials/Data_loading.ipynb)
      - [General aspects on HMP (tutorial 1)](tutorials/1-Background_on_HMP.ipynb)
      - [Estimating a model (tutorial 2)](tutorials/2-Estimating_a_model.ipynb)
      - [Test for the best number of events (tutorial 3)](tutorials/3-Testing_the_number_of_events.ipynb)
      - [Looking at condition differences (tutorial 4)](tutorials/4-Analyzing_condition_differences.ipynb)

  The following section will quickly walk you through an example usage in simulated data (using [MNE](https://mne.tools/dev/auto_examples/simulation/index.html)'s simulation functions and tutorials)
  First we load the packages necessary for the demo on simulated data

  ```python

  import numpy as np
  import matplotlib.pyplot as plt
  from scipy.stats import gamma
  import seaborn as sns
  from hmp import simulations#that module is only to simulate things, no need to import when analyzing data

  import hmp
  ```

  In the following code block we simulate 200 trials with four HMP events defined as the activation of four neural sources (in the source space of MNE's sample participant). This is not code you would need for your own analysis except if you'd want to simulate and test properties of HMP models. All four sources are defined by a location in sensor space, an activation amplitude and a distribution in time (here a gamma with shape and scale parameters) for the onsets of the events on each trial. The simulation functions are based on this [MNE tutorial ](https://mne.tools/stable/auto_examples/simulation/simulated_raw_data_using_subject_anatomy.html).
  [!IMPORTANT] If you're running this for the first time a 1.65 G file will be downloaded in order to perform the simulation but this will be done only once (alternatively you can just download the corresponding simulation file and place it in the same folder from where you are running this notebook)
  ```python
  cpus = 1 # For multiprocessing, usually a good idea to use multiple CPUs as long as you have enough RAM
  n_trials = 200 #Number of trials to simulate, we use 200 to get nice ERPs but you can reduce for speed

  sfreq = 500#sampling frequency of the signal
  n_events = 4
  frequency = 10.#Frequency of the event defining its duration, half-sine of 10Hz = 50ms
  amplitude = .25e-7 #Amplitude of the event in nAm, defining signal to noise ratio
  shape = 2 #shape of the gamma distribution
  scales = np.array([50, 150, 200, 250, 100])/shape #Mean duration of the time between each event in ms
  names = simulations.available_sources()[[22,33,55,44,0]]#Which source to activate for each event (see atlas when calling simulations.available_sources())
  sources = []
  for source in zip(names, scales):#One source = one frequency/event width, one amplitude and a given by-trial variability distribution
      sources.append([source[0], frequency, amplitude, gamma(shape, scale=source[1])])

  file = simulations.simulate(sources, n_trials, cpus, 'dataset_README',  overwrite=False, sfreq=sfreq, seed=1)

  positions = simulations.simulation_positions()
  ```
      /home/gweindel/owncloud/projects/RUGUU/main_hmp/hmp/simulations.py:190: UserWarning: ./dataset_README_raw.fif exists no new simulation performed
        warn(f'{subj_file} exists no new simulation performed', UserWarning)

  To recover the data we need to create the event structure based on the triggers created during simulation. This is the same as analyzing real EEG data and recovering events in the stimulus channel. In our case 1 signal the onset of the stimulus and 6 the moment of the response. Hence a trial is defined as the times occuring between the triggers 1 and 6.
  ```python

  generating_events = np.load(file[1])
  resp_trigger = int(np.max(np.unique(generating_events[:,2])))#Resp trigger is the last source in each trial
  event_id = {'stimulus':1}#trigger 1 = stimulus
  resp_id = {'response':resp_trigger}

  events = generating_events[(generating_events[:,2] == 1) | (generating_events[:,2] == resp_trigger)]#only retain stimulus and response triggers

  import mne
  raw = mne.io.read_raw_fif(file[0], preload=False, verbose=False)
  raw.pick_types(eeg=True).plot(scalings=dict(eeg=1e-5), events=events, block=True);
  ```
  ![png](README_files/README_7_1.png)

  To see how well HMP does at recovering by-trial events, first we get the ground truth from our simulation. Unfortunately, with an actual dataset you don’t have access to this, of course.
  ```python
  %matplotlib inline

  sim_event_times = np.reshape(np.ediff1d(generating_events[:,0],to_begin=0)[generating_events[:,2] > 1], \
             (n_trials, n_events+1))
  sim_event_times_cs = np.cumsum(sim_event_times, axis=1)
  ```

  First we read the EEG data as we would for a single participant
  ```python

  eeg_data = hmp.utils.read_mne_data(file[0], event_id=event_id, resp_id=resp_id, sfreq=sfreq,
              events_provided=events, verbose=False)
  ```
      Processing participant ./dataset_README_raw.fif's continuous eeg
      /home/gweindel/miniconda3/lib/python3.12/site-packages/mne/epochs.py:2986: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise in a future error of pandas. Value '' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
        metadata.iloc[:, 0] = ""
      200 trials were retained for participant ./dataset_README_raw.fif
  HMP uses [xarray](https://docs.xarray.dev/en/stable/) named dimension matrices, allowing to directly manipulate the data using the name of the dimensions:
  ```python

  print(eeg_data)
  eeg_data.sel(channels=['EEG 001','EEG 002','EEG 003'], samples=range(400))\
      .data.groupby('samples').mean(['participant','epochs']).plot.line(hue='channels');
  ```
      <xarray.Dataset> Size: 87MB
      Dimensions:      (participant: 1, epochs: 200, channels: 59, samples: 917)
      Coordinates:
        * epochs       (epochs) int64 2kB 0 1 2 3 4 5 6 ... 194 195 196 197 198 199
        * channels     (channels) <U7 2kB 'EEG 001' 'EEG 002' ... 'EEG 059' 'EEG 060'
        * samples      (samples) int64 7kB 0 1 2 3 4 5 6 ... 911 912 913 914 915 916
          event_name   (epochs) object 2kB 'stimulus' 'stimulus' ... 'stimulus'
          rt           (epochs) float64 2kB 0.394 0.926 0.802 ... 1.082 0.312 0.376
        * participant  (participant) <U2 8B 'S0'
      Data variables:
          data         (participant, epochs, channels, samples) float64 87MB 4.786e...
      Attributes:
          sfreq:             500.0
          offset:            0
          lowpass:           40.0
          highpass:          0.10000000149011612
          lower_limit_RT:    0
          upper_limit_RT:    5.002
          reject_threshold:  inf
          n_trials:          200
  ![png](README_files/README_12_1.png)
  Next we transform the data by applying a spatial principal components analysis (PCA) to reduce the dimensionality of the data.
  In this case we choose 5 principal components for commodity and speed.
  Note that if the number of components to retain is not specified, the scree plot of the PCA is displayed and a prompt ask how many PCs should be retained (in this case we specify it as building the README does not allow for prompts)
  ```python
  hmp_data = hmp.utils.transform_data(eeg_data, apply_standard=False, n_comp=5)
  ```

  Once the data is in the expected format, we can initialize an HMP object; note that no estimation is performed yet.
  ```python
  init = hmp.models.hmp(hmp_data, eeg_data, cpus=1, )#Initialization of the model
  ```

  We can directly fit an HMP model without giving any info on the number of events (see tutorial 2 for a detailed explanation of the following cell:
  ```python
  estimates = init.fit()
  ```
        0%|          | 0/388 [00:00<?, ?it/s]
      Transition event 1 found around sample 25
      Transition event 2 found around sample 111
      Transition event 3 found around sample 212
      Transition event 4 found around sample 341
      /home/gweindel/miniconda3/lib/python3.12/site-packages/tqdm/std.py:465: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
        if rate and total else datetime.utcfromtimestamp(0))
      All events found, refitting final combination.
      Estimating 4 events model
      parameters estimated for 4 events model

  In the previous cell we initiated an HMP model looking for default 50ms half-sine in the EEG signal. The method discovered four events, and therefore five gamma-distributed time distributions with a fixed shape of 2 and an estimated scale. We can now inspect the results of the fit.
  We can directly take a look to the topologies and latencies of the events by calling ```hmp.visu.plot_topo_timecourse```
  ```python
  hmp.visu.plot_topo_timecourse(eeg_data, estimates, #Data and estimations
                                 positions, init,#position of the electrodes and initialized model
                                 magnify=1, sensors=False)#Display control parameters
  ```
  ![png](README_files/README_20_0.png)
  This shows us the electrode activity on the scalp as well as the average time of occurrence of the events.
  But HMP doesn't provide point estimate but location probability of each of the detected event and that for each trial:
  ```python
  hmp.visu.plot_distribution(estimates.eventprobs.sel(trial_x_participant=('S0',1)),
                              xlims=(0,np.percentile(sim_event_times.sum(axis=1), q=90)));
  ```
  ![png](README_files/README_22_0.png)
  This then shows the likeliest event onset location in time for the last simulated trial!

  As we simulated the data we have access to the ground truth of the underlying generative events. We can then compare the average time of simulated event onset to the one estimated by HMP.
  ```python
  hmp.visu.plot_topo_timecourse(eeg_data, estimates, positions, init, magnify=1, sensors=False, figsize=(13,1), title='Actual vs estimated event onsets',
          times_to_display = np.mean(np.cumsum(sim_event_times,axis=1),axis=0))
  ```
  ![png](README_files/README_24_0.png)
  We see that the HMP recovered the exact average location of the bumps defined in the simulated data.
  We can further test how well the method did by comparing the generated single trial onsets with those estimated from the HMP model
  ```python
  colors = sns.color_palette(None, n_events+1)
  fig, ax = plt.subplots(n_events,2, figsize=(10,3*n_events), dpi=100, sharex=False)
  i = 0
  ax[0,0].set_title('Point-wise')
  ax[0,1].set_title('Distribution')
  ax[-1,0].set_xlabel(f'Simulated by-trial event time {i+1}')
  estimated_times = init.compute_times(init, estimates, duration=False, mean=False, add_rt=False).T
  for event in estimated_times:
      event_locations = sim_event_times_cs[:,i].T
      sns.regplot(x=event_locations, y=event, ax=ax[i,0], color=colors[i])
      ax[i,0].plot([np.min(event), np.max(event)], [np.min(event), np.max(event)],'--', color='k')
      ax[i,0].set_ylabel(f'Estimated by-trial event onset {i+1}')
      sns.kdeplot(event, ax=ax[i,1], color='orange')
      sns.kdeplot(event_locations, ax=ax[i,1], color='k')
      i+= 1
  plt.tight_layout();
  ```
  ![png](README_files/README_26_0.png)
  We see that every events gets nicely recovered even on a by-trial basis!

  HMP just helped us estimate at which time point single-trial events occured. There are plenty of things to do with this information.
  As an example consider how average curve of traditional event related potentials (ERP) in EEG are a bad representation of the underlying single-trial event. In the following cell we do the average ERP electrode activity for 4 subselected electrodes in the classical way (stimulus centered) and in a new way that HMP allows, **by-trial resampled signal**
  ```python
  fig, ax = plt.subplots(1,2, figsize=(6,2), sharey=True, sharex=True, dpi=200)
  colors = iter([plt.cm.tab10(i) for i in range(10)])
  for channel in  ['EEG 031', 'EEG 039', 'EEG 040', 'EEG 048']:
      c = next(colors)
      fakefit = init.fit_single(2, maximization=False, verbose=False)#Just to get the stim ERP in the same format
      BRP_times = init.compute_times(init, fakefit, fill_value=0, add_rt=True)
      times = BRP_times.sel(event=[0,3])#Stim and response only
      times['event'] = [0,1]
      erp_data = hmp.visu.erp_data(eeg_data.stack(trial_x_participant=["participant","epochs"]), times, channel)
      hmp.visu.plot_erp(times, erp_data, c, ax[0], upsample=2, label=channel)
      BRP_times = init.compute_times(init, estimates, fill_value=0, add_rt=True)#Real estimate
      erp_data = hmp.visu.erp_data(eeg_data.stack(trial_x_participant=["participant","epochs"]), BRP_times, channel,100)
      hmp.visu.plot_erp(BRP_times, erp_data, c, ax[1], upsample=2)
  ev_colors = iter(['red', 'purple','brown','black',])
  for event in range(4):
      c = next(ev_colors)
      ax[1].vlines(sim_event_times_cs[:,event].mean()*2, ymin=-3e-6, ymax=3e-6, color=c, alpha=.75)
  ax[0].set_xlabel('Time (ms) from stimulus')
  ax[1].set_xlabel('(Resampled) Time (ms) from stimulus')
  plt.xlim(0,800)
  ax[0].legend(bbox_to_anchor=(2.9,.85))
  plt.show()
  ```
  ![png](README_files/README_30_0.png)
  In the right panel the signal between each event is resampled to have the same duration as the average between event times (in order to be represented along with the classical ERP on the left). Do you see the gain in signal-to-noise ratio? This is because **prior to averaging we centered all trials around the most likely event onset time detected by HMP!**
  Now in order to display both ERPs side-by-side we had to transform the signal by quite a bit. Instead of doing this resampling procedure we can also just look at the activity of those four electrodes in the vincinity of the detected events:
  ```python
  import pandas as pd
  fig, ax = plt.subplots(1,4, figsize=(9,2), sharey=True, sharex=True, dpi=200)
  colors = iter([plt.cm.tab10(i) for i in range(10)])
  data = eeg_data.stack({'trial_x_participant':['participant','epochs']}).data.dropna('trial_x_participant', how="all")
  times = init.compute_times(init, estimates, fill_value=0, add_rt=True)

  baseline, n_samples = -50, 50#Take 50 samples on both sides, i.e. 100ms in a 500Hz signal
  ev_colors = iter(['red', 'purple','brown','black',])
  for i, event in enumerate(times.event[1:-1]):
      c = next(ev_colors)
      centered = hmp.utils.centered_activity(data, times, ['EEG 031',  'EEG 040', 'EEG 048'], event=event, baseline=baseline, n_samples=n_samples)
      ax[i].plot(centered.samples*2, centered.data.unstack().mean(['trials', 'channel', 'participant']).data, color=c)
      ax[i].set(title=f"Event {event.values}", ylim=(-5.5e-6, 5.5e-6), xlabel=f'Time (ms) around {event.values}')
      if i == 0:
          ax[i].set_ylabel("Voltage")
  plt.xlim(-100,100);
  ```
  ![png](README_files/README_32_0.png)
  Compared to the traditional ERP representation we see that HMP provides a much better view of the underlying single-trial activities than the tradition ERP.
  Now HMP is not merely a method to look at ERPs or by-trial times. A lot can be done once the estimation has been made, for example:
  - Connectivity analysis starting from each event onset
  - Time/frequency decomposition
  - Single-trial signal analysis with, e.g. a single-trial covariate
  - and so on

  For examples on how to use the package on real data, or to compare event time onset across conditions see the tutorial notebooks:
  - [Load EEG data ](tutorials/Data_loading.ipynb)
  - [General aspects on HMP (tutorial 1)](tutorials/1-Background_on_HMP.ipynb)
  - [Estimating a model (tutorial 2)](tutorials/2-Estimating_a_model.ipynb)
  - [Test for the best number of events (tutorial 3)](tutorials/3-Testing_the_number_of_events.ipynb)
  - [Looking at condition differences (tutorial 4)](tutorials/4-Analyzing_condition_differences.ipynb)

  HMP is very flexible regarding what type of pattern, type of distribution and data to fit HMP to. Readers interested in the original HsMM-MVPA method (which can be seen as a specific implementation of HMP) can take a look at the paper by Anderson, Zhang, Borst, & Walsh  ([2016](https://psycnet.apa.org/doi/10.1037/rev0000030)) as well as the book chapter by Borst & Anderson ([2021](http://jelmerborst.nl/pubs/ACTR_HMP_MVPA_BorstAnderson_preprint.pdf)). The following list contains a non-exhaustive list of papers published using the original HsMM-MVPA method:
  - Anderson, J.R., Borst, J.P., Fincham, J.M., Ghuman, A.S., Tenison, C., & Zhang, Q. (2018). The Common Time Course of Memory Processes Revealed. Psychological Science 29(9), pp. 1463-1474. [link](https://doi.org/10.1177/0956797618774526)
  - Berberyan, H.S., Van Rijn, H., & Borst, J.P. (2021). Discovering the brain stages of lexical decision: Behavioral effects originate from a single neural decision process. Brain and Cognition 153: 105786. [link](https://www.sciencedirect.com/science/article/pii/S0278262621001068)
  - Berberyan, H. S., van Maanen, L., van Rijn, H., & Borst, J. (2021). EEG-based identification of evidence accumulation stages in decision-making. Journal of Cognitive Neuroscience, 33(3), 510-527. [link](https://doi.org/10.1162/jocn_a_01663)
  - Van Maanen, L., Portoles, O., & Borst, J. P. (2021). The discovery and interpretation of evidence accumulation stages. Computational brain & behavior, 4(4), 395-415. [link](https://link.springer.com/article/10.1007/s42113-021-00105-2)
  - Portoles, O., Blesa, M., van Vugt, M., Cao, M., & Borst, J. P. (2022). Thalamic bursts modulate cortical synchrony locally to switch between states of global functional connectivity in a cognitive task. PLoS computational biology, 18(3), e1009407. [link](https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1009407)
  - Portoles, O., Borst, J.P., & Van Vugt, M.K. (2018). Characterizing synchrony patterns across cognitive task stages of associative recognition memory. European Journal of Neuroscience 48, pp. 2759-2769. [link](http://onlinelibrary.wiley.com/doi/10.1111/ejn.13817/epdf)
  - Zhang, Q., van Vugt, M.K., Borst, J.P., & Anderson, J.R. (2018). Mapping Working Memory Retrieval in Space and in Time: A Combined Electroencephalography and Electrocorticography Approach. NeuroImage 174, pp. 472-484. [link](https://www.sciencedirect.com/science/article/pii/S1053811918302490)

rpm_license:
  BSD 3-Clause License  Copyright (c) 2022, Gabriel Weindel, Leendert van Maanen, Jelmer Borst All rights reserved.  Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:  1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.  2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.  3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.


pkgId: 4ec16b16c0d1ebfb7ccf17e584a774e5733bf66876c9b51a54c629c5faf810ee
name: python-hoardy-web-help
arch: noarch
version: 0.20.0
epoch: 0
release: 1
summary: Development documents and examples for hoardy-web
url: https://oxij.org/software/hoardy-web/
rpm_license: GPL-3.0-or-later
rpm_group: Unspecified
rpm_buildhost: localhost
rpm_sourcerpm: python-hoardy-web-0.20.0-1.src.rpm
location_href: Packages/python-hoardy-web-help-0.20.0-1.noarch.rpm
checksum_type: sha256
description:
  `hoardy-web` is a tool to inspect, search, organize, programmatically extract values and generate static website mirrors from, archive, view, and replay `HTTP` archives/dumps in `WRR` ("Web Request+Response", produced by the [`Hoardy-Web` Web Extension browser add-on](https://github.com/Own-Data-Privateer/hoardy-web/tree/master/), also [there](https://oxij.org/software/hoardy-web/tree/master/)) and [`mitmproxy`](https://github.com/mitmproxy/mitmproxy) (`mitmdump`) file formats.



  The top part of this `README` file (from here to ["Usage"](#usage)) is designed to be read in a linear fashion, not piece-meal.

  The ["Usage"](#usage) section can be read and referenced to in arbitrary order.





  - Install `Python 3`:

    - On Windows: [Download and install Python from the official website](https://www.python.org/downloads/windows/).
    - On a conventional POSIX system like most GNU/Linux distros and MacOS X: Install `python3` via your package manager. Realistically, it probably is installed already.



  - On a Windows system with unconfigured `PATH`, install with:

    ``` bash
    pip install hoardy-web
    ```
    and run as
    ``` bash
    python3 -m hoardy_web --help
    ```

  - On a conventional POSIX system or on a Windows system with configured `PATH` environment variable, install it with:

    ``` bash
    pip install hoardy-web
    ```
    and run as
    ``` bash
    hoardy-web --help
    ```

  - Alternatively, on a POSIX system, run without installing:

    ```bash
    alias hoardy-web="python3 -m hoardy_web"
    hoardy-web --help
    ```

  - Alternatively, on a system with [Nix package manager](https://nixos.org/nix/)

    ``` bash
    nix-env -i -f ./default.nix
    hoardy-web --help
    ```

    Though, in this case, you'll probably want to do the first command from the parent directory, to install everything all at once.



  Install the [`Hoardy-Web` extension](../extension/) and get some archive data by browsing some websites.



  You can then use your archived data to generate a local offline static website mirror that can be opened in a web browser without accessing the Internet, similar to what `wget -mpk` does.

  The invocation is slightly different depending on if the data was exported via `saveAs` by the [`Hoardy-Web` extension](../extension/) itself, saved via the [`hoardy-web-sas` simple archiving server](../simple_server/), or via `hoardy-web serve --archive-to` (see below):

  ```bash

  hoardy-web mirror --to ~/hoardy-web/mirror1 ~/Downloads/Hoardy-Web-export-*


  hoardy-web mirror --to ~/hoardy-web/mirror1 ../simple_server/pwebarc-dump ~/hoardy-web/raw
  ```

  You can then, e.g. `rsync`/copy `~/hoardy-web/mirror1` to your e-book reader/phone before hopping on a plane or going on a deep-sea dive, and still be able to read all those pages.

  The default settings should work for most simple websites, but a [section below](#mirror) contains more info and more usage examples.



  You can also view your archived pages by running `hoardy-web` in web server mode:

  ```bash


  hoardy-web serve \
    ~/hoardy-web/raw \
    ../simple_server/pwebarc-dump \
    ~/Downloads/Hoardy-Web-export-* \
    mitmproxy.*.dump
  ```

  You can then navigate to

  - <http://127.0.0.1:3210/web/*/*> to see the list of all available URLs and their versions (visits), or to
  - something like <http://127.0.0.1:3210/web/2/https://archiveofourown.org/works/3733123> to view the latest archived version of that URL, or to
  - something like <http://127.0.0.1:3210/web/*/https://archiveofourown.org/works/3733123> to view the list of all visits to this URL,
  - which also works with glob patterns <http://127.0.0.1:3210/web/*/https://archiveofourown.org/works/[0-9]*>.

  This is very reminiscent of the [Wayback Machine](https://web.archive.org/) by design, yes.

  You can also use `hoardy-web serve` to replace `hoardy-web-sas` simple archiving server by combining both archival and replay:

  ```bash
  hoardy-web serve --implicit \
    --archive-to ~/hoardy-web/raw \
    ../simple_server/pwebarc-dump \
    ~/Downloads/Hoardy-Web-export-* \
    mitmproxy.*.dump
  ```

  See [below](#serve) for more info.



  If you are not interested searching, organizing, and/or manipulating your archives in various ways, then, at the moment, that's essentially everything the `hoardy-web` tool can do for you.
  See the [TODO list](../CHANGELOG.md#todo) for a list of planned features.

  If you are interested, continue reading.



  - A *`reqres`* (`Reqres` when a Python type) is an instance of a structure representing `HTTP` request+response pair with some additional metadata.

  - [*`WARC`*](https://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1-annotated/) is an ISO web archiving file format used by the [Wayback Machine](https://web.archive.org/) ([heritrix](https://github.com/internetarchive/heritrix3)) and many other tools.

  - [`mitmproxy`](https://github.com/mitmproxy/mitmproxy) is a tool stripping TLS from a connection, dumping and/or modifying the traffic going through it, and re-adding TLS back.
    I.e. a Man-In-The-Middle proxy.

    In the context of this project, *`mitmproxy`* is also a file format produced by the `mitmdump` tool.

  - [*`WRR`*](../doc/data-on-disk.md) is a native archiving format used by `Hoardy-Web` project.

    It is very much inspired by `mitmproxy` in that it stores a raw `HTTP` request+response pairs (instead of encoding `GET` documents like `WARC` does), but, unlike, `mitmproxy`, `WRR` is a [CBOR (RFC8949)](https://datatracker.ietf.org/doc/html/rfc8949) encoding of `HTTP` request+response pairs, not some custom binary encoding.

  - *`WRR` file* is a file with a single `WRR` dump in it.
    Typically, these use `.wrr` file extension.

    When you use the [`Hoardy-Web` extension](../extension/) together with the [`hoardy-web-sas` archiving server](../simple_server/) or [`hoardy-web serve`](#serve), the latter two write `WRR` dumps the extension generates, one dump per file, into separate `.wrr` files in its dumping directory.

    The situation is similar if you instead use the `Hoardy-Web` extension with `Export via 'saveAs'` option enabled but `Export via 'saveAs' > Bundle dumps` option disabled.
    The only difference is that `WRR` files get written to your `~/Downloads` or similar.

    ```bash
    ls ~/Downloads/Hoardy-Web-export-*
    ```

  - *`WRR` bundle* is a file containing a concatenation of a bunch of plain uncompressed `WRR` dumps, which are then optionally compressed with `GZip`.
    Typically, these use `.wrrb` file extension.

    When you use the `Hoardy-Web` extension together with both `Export via 'saveAs'` and bundling options enabled, it archives your data by generating `WRR` bundles, which then get written to your `~/Downloads` or similar.

  - *`HAR`* ([abandoned W3C spec](https://w3c.github.io/web-performance/specs/HAR/Overview.html), a [nicer spec](http://www.softwareishard.com/blog/har-12-spec/)) is an archiving file format used by the "Network Monitor" tools of most modern browsers.

    It is similar `mitmproxy` and `WRR` in that it, too, stores `HTTP` request+response pairs, but it uses a very inefficient `JSON` encoding with body data encoded as `base64` and a lot of the metadata duplicated multiple times across the structure.

  - [*`PCAP`*](https://en.wikipedia.org/wiki/Pcap) is a file format used by many raw packet capture tools.



  At the moment `hoardy-web` tool supports

  - `WRR` files (both compressed and not),
  - `WRR` bundles (similarly), and
  - `mitmproxy` dumps.

  `WARC` and `HAR` support will be added [soon-ish](../CHANGELOG.md#todo), `PCAP` support will be added eventually.

  All sub-commands of `hoardy-web` except for

  - `organize` when run with `--move`, `--hardlink`, or `--symlink` (i.e. with anything other than `--copy`),
  - `get`, and
  - `run`

  can take all supported file formats as inputs.
  So, most examples described below will work fine with any mix of inputs as arguments.

  You can, however, force `hoardy-web` to use a specific loader for all given inputs, e.g.:

  ```
  hoardy-web mirror --to ~/hoardy-web/mirror1 \
    --load-mitmproxy mitmproxy.*.dump
  ```

  This is slightly faster than the default `--load-any` and, for most loaders, produces more specific errors that explain exactly what failed to parse, instead of simply saying that all tried parsers failed to work.





  To use `hoardy-web organize`, `get`, and `run` sub-commands on data stored in file formats other than separate `WRR` files, you will have to import them first:

  ```bash
  hoardy-web import bundle --to ~/hoardy-web/raw ~/Downloads/Hoardy-Web-export-*
  hoardy-web import mitmproxy --to ~/hoardy-web/mitmproxy ~/mitmproxy/mitmproxy.*.dump
  ```

  Note that `.wrr` files can be parsed as single-dump `.wrrb` files, so the first command above will work even when some of the exported dumps were exported as separate `.wrr` files by the `Hoardy-Web` extension (because you configured it to do that, because it exported a bucket with a single dump as a separate file, because it exported a dump that was larger than set maximum bundle size as a separate file, etc).
  So, essentially, the first command above command is equivalent to

  ```bash
  hoardy-web organize --copy --to ~/hoardy-web/raw ~/Downloads/Hoardy-Web-export-*.wrr
  hoardy-web import bundle --to ~/hoardy-web/raw ~/Downloads/Hoardy-Web-export-*.wrrb
  ```

  In fact, internally, `hoardy-web import bundle` is actually an alias for `hoardy-web organize --copy --load-wrrb --defer-number 0`.



  You can search your archive directory by using `hoardy-web find` sub-command, that prints paths to those of its inputs which match given conditions.
  For example, to list reqres from `~/hoardy-web/raw` that contain complete `GET` requests with `200 OK` responses, you can run:

  ```bash
  hoardy-web find --method GET --status-re .200C ~/hoardy-web/raw
  ```

  To limit the above to responses containing `text/html` bodies with a (whole) word "Potter" in them:

  ```bash
  hoardy-web find --method GET --method DOM --status-re .200C --response-mime text/html \
    --response-body-grep-re "\bPotter\b" ~/hoardy-web/raw
  ```

  Most other sub-commands also accept the same filtering options.
  So, for instance, you can pretty-print or generate a static mirror from such files instead:

  ```bash
  hoardy-web pprint --method GET --method DOM --status-re .200C --response-mime text/html \
    --response-body-grep-re "\bPotter\b" \
    ~/hoardy-web/raw


  hoardy-web mirror --method GET --method DOM --response-mime text/html \
    --response-body-grep-re "\bPotter\b" \
    --to ~/hoardy-web/mirror-potter ~/hoardy-web/raw
  ```

  Or, say, you want a list of all domains you ever visited that use CloudFlare:

  ```bash
  hoardy-web stream --format=raw -ue hostname \
    --response-headers-grep-re '^server: cloudflare' \
    ~/hoardy-web/raw | sort | uniq
  ```

  Or, say, you want to get all responses from a certain host with `JSON`s, except when they were fetched from CloudFlare and encoded with `br`, and then feed them to a script:

  ```bash
  hoardy-web find -z --url-re 'https://example\.org/.*' --response-mime text/json \
    --not-response-headers-and-grep-re '^server: cloudflare' \
    --not-response-headers-and-grep-re '^content-encoding: br' \
    ~/hoardy-web/raw > found-paths
  xargs -0 my-example-org-json-parser < found-paths
  ```

  See the ["Usage"](#usage) section below for all possible filtering options.

  In principle, the possibilities are limitless since `hoardy-web` has a tiny expression language which you can use to do things not directly supported by the command-line options:

  ```bash
  hoardy-web find --and "response.body|eb|len|> 10240" ~/hoardy-web/raw
  ```

  and, if you are a developer, you can easily add your own custom functions [into there](./hoardy_web/linst.py).



  To merge multiple input directories into one you can simply `hoardy-web organize` them `--to` a new directory.
  `hoardy-web` will automatically deduplicate all the files in the generated result.

  That is to say, for `hoardy-web organize`

  - `--move` is de-duplicating when possible,
  - while `--copy`, `--hardlink`, and `--symlink` are non-duplicating when possible.

  For example, if you duplicate an input directory via `--copy` or `--hardlink`:

  ```bash
  hoardy-web organize --copy     --to ~/hoardy-web/copy1 ~/hoardy-web/original
  hoardy-web organize --hardlink --to ~/hoardy-web/copy2 ~/hoardy-web/original
  ```

  (In real-life use different copies usually end up on different backup drives or some such.)

  Then, repeating the same command would a noop:

  ```bash

  hoardy-web organize --copy     --to ~/hoardy-web/copy1 ~/hoardy-web/original
  hoardy-web organize --hardlink --to ~/hoardy-web/copy2 ~/hoardy-web/original
  ```

  And running the opposite command would also be a noop:

  ```bash

  hoardy-web organize --hardlink --to ~/hoardy-web/copy1 ~/hoardy-web/original
  hoardy-web organize --copy     --to ~/hoardy-web/copy2 ~/hoardy-web/original
  ```

  And copying between copies is also a noop:

  ```bash

  hoardy-web organize --hardlink --to ~/hoardy-web/copy2 ~/hoardy-web/copy1
  hoardy-web organize --copy     --to ~/hoardy-web/copy2 ~/hoardy-web/copy1
  ```

  But doing `hoardy-web organize --move` while supplying directories that have the same data will deduplicate the results:

  ```bash
  hoardy-web organize --move --to ~/hoardy-web/all ~/hoardy-web/copy1 ~/hoardy-web/copy2

  find ~/hoardy-web/copy1 ~/hoardy-web/copy2 -type f


  hoardy-web organize --move --to ~/hoardy-web/original ~/hoardy-web/all


  find ~/hoardy-web/all -type f

  ```

  Similarly, `hoardy-web organize --symlink` resolves its input symlinks and deduplicates its output symlinks:

  ```bash
  hoardy-web organize --symlink --output hupq_msn --to ~/hoardy-web/pointers ~/hoardy-web/original
  hoardy-web organize --symlink --output shupq_msn --to ~/hoardy-web/schemed ~/hoardy-web/original


  hoardy-web organize --symlink --output hupq_msn --to ~/hoardy-web/pointers ~/hoardy-web/original ~/hoardy-web/schemed
  ```

  I.e. the above will produce `~/hoardy-web/pointers` with unique symlinks pointing to each file in `~/hoardy-web/original` only once.



  Assuming you keep your `WRR` dumps in `~/hoardy-web/raw`, the following commands will generate a file system hierarchy under `~/hoardy-web/latest` organized in such a way that, for each URL from `~/hoardy-web/raw`, it will contain a symlink from under `~/hoardy-web/latest` to a file in `~/hoardy-web/raw` pointing to the most recent `WRR` file containing `200 OK` response for that URL:

  ```bash

  hoardy-web import bundle --to ~/hoardy-web/raw ~/Downloads/Hoardy-Web-export-*

  hoardy-web organize --move --to ~/hoardy-web/raw ../simple_server/pwebarc-dump


  hoardy-web organize --symlink --latest --output hupq --to ~/hoardy-web/latest --status-re .200C ~/hoardy-web/raw
  ```

  Personally, I prefer `flat_mhs` format (see the documentation of the `--output` below), as I dislike deep file hierarchies.
  Using it also simplifies filtering in my `ranger` file browser, so I do this:

  ```bash
  hoardy-web organize --symlink --latest --output flat_mhs --to ~/hoardy-web/latest --status-re .200C ~/hoardy-web/raw
  ```



  The above commands rescan the whole contents of `~/hoardy-web/raw` and so can take a while to complete.

  If you have a lot of `WRR` files and you want to keep your symlink tree updated in near-real-time you will need to use a two-stage pipeline by giving the output of `hoardy-web organize --zero-terminated` to `hoardy-web organize --stdin0` to perform complex updates.

  E.g. the following will rename new `WRR` files from `../simple_server/pwebarc-dump` to `~/hoardy-web/raw` renaming them with `--output default` (the `for` loop is there to preserve buckets/profiles):

  ```bash
  for arg in ../simple_server/pwebarc-dump/* ; do
    hoardy-web organize --zero-terminated --to ~/hoardy-web/raw/"$(basename "$arg")" "$arg"
  done > changes
  ```

  Then, you can reuse the paths saved in `changes` file to update the symlink tree, like in the above:

  ```
  hoardy-web organize --symlink --latest --output flat_mhs --to ~/hoardy-web/latest --status-re .200C --stdin0 < changes
  ```

  Then, optionally, you can reuse `changes` file again to symlink all new files from `~/hoardy-web/raw` to `~/hoardy-web/all`, showing all URL versions, by using `--output hupq_msn` format:

  ```bash
  hoardy-web organize --symlink --output hupq_msn --to ~/hoardy-web/all --stdin0 < changes
  ```



  To render your archived data into a local offline static website mirror containing interlinked `HTML` files and their requisite resources similar to (but better than) what `wget -mpk` (`wget --mirror --page-requisites --convert-links`) does, you need to run something like this:

  ```bash

  hoardy-web mirror --to ~/hoardy-web/mirror1 ~/hoardy-web/raw


  hoardy-web mirror --to ~/hoardy-web/mirror1 ~/Downloads/Hoardy-Web-export-*


  hoardy-web mirror --to ~/hoardy-web/mirror1 mitmproxy.*.dump


  hoardy-web mirror --to ~/hoardy-web/mirror1 \
    ~/hoardy-web/raw \
    ~/Downloads/Hoardy-Web-export-* \
    mitmproxy.*.dump
  ```

  On completion, `~/hoardy-web/mirror1` will contain said newly generated interlinked `HTML` files, their resource requisites, and everything else available from given archive files.
  The set of mirrored files can be limited with using several methods described below.

  By default, the resulting `HTML` files will be stripped of all `JavaScript` and other stuff of various levels of evil.
  The results should be completely self-contained (i.e., work inside a browser running in "Work offline" mode) and safe to view in a dumb unconfigured browser (i.e., the resulting web pages should not request any page requisites --- like images, media, `CSS`, fonts, etc --- from the Internet).

  (In practice, though, `hoardy-web mirror` is not completely free of bugs and `HTML5` spec is constantly evolving, with new things getting added there all the time.
  So, it is entirely possible that the output of the above `hoardy-web mirror` invocation will not be completely self-contained.
  Which is why the `Hoardy-Web` extension has its own per-tab `Work offline` mode which, by default, gets enabled for tabs with `file:` URLs.
  That feature prevents the outputs of `hoardy-web mirror` from accessing the Internet regardless of any bugs or missing features in `hoardy-web`.
  It also helps with debugging.)

  If you are unhappy with the above and, for instance, want to keep `JavaScript` and produce human-readable `HTML`s, you can run the following instead:

  ```bash
  hoardy-web mirror \
    -e 'response.body|eb|scrub response &all_refs,+scripts,+pretty' \
    --to ~/hoardy-web/mirror2 ~/hoardy-web/raw
  ```

  Or, say, you want to produce minimized outputs:

  ```bash
  hoardy-web mirror \
    -e 'response.body|eb|scrub response &all_refs,-verbose,-whitespace,-optional_tags' \
    --to ~/hoardy-web/mirror2 ~/hoardy-web/raw
  ```

  See the documentation for the `--remap-*` options of `mirror` sub-command and the options of the `scrub` function below for more info.

  If you instead want a mirror made of raw files without any content censorship or link conversions, run:

  ```bash

  hoardy-web mirror --raw-sbody --to ~/hoardy-web/mirror-raw ~/hoardy-web/raw
  ```

  The later command will render your mirror pretty quickly, but the other `mirror` commands use the `scrub` function, and that will be pretty slow, mostly because `html5lib` and `tinycss2` that `hoardy-web` uses for paranoid `HTML` and `CSS` parsing and filtering are fairly slow.
  Under `CPython` on my 2013-era laptop `hoardy-web mirror` manages to render, on average, 3 `HTML` and `CSS` files per second.
  Though, this is not very characteristic of the overall mirroring speed, since images and other media just get copied around at expected speeds of 300+ files per second.

  Also, enabling `+indent` (or `+pretty`) in `scrub` will make `HTML` scrubbing slightly slower (since it will have to track more stuff) and `CSS` scrubbing a lot slower (since it will force complete structural parsing, not just tokenization).



  By default, `hoardy-web mirror` runs with an implied `--remap-all` option which remaps *all* links in mirrored `HTML` files to local files, even if source `WRR` files for those would-be mirrored files are missing.
  This allows you to easily update your mirror directory incrementally by re-running `hoardy-web mirror` with the same `--to` argument on new inputs.
  For instance:

  ```bash

  hoardy-web mirror --to ~/hoardy-web/mirror1 ~/hoardy-web/raw/*/2023


  hoardy-web mirror --skip-existing --to ~/hoardy-web/mirror1 ~/hoardy-web/raw/*/2024


  hoardy-web mirror --overwrite-dangerously --to ~/hoardy-web/mirror1 ~/hoardy-web/raw/*/2024
  ```

  After the first of the above commands, links from pages generated from `WRR` files of `~/hoardy-web/raw/*/2023` to URLs contained in files from `~/hoardy-web/raw/*/2024` but not contained in files from `~/hoardy-web/raw/*/2023` will point to non-existent, yet unmirrored, files on disk.
  I.e. those links will be broken.
  Running the second or the third command from the example above will then mirror additional files from `~/hoardy-web/raw/*/2024`, thus fixing some or all of those links.



  If you want to treat links pointing to not yet hoarded URLs exactly like `wget -mpk` does, i.e. you want to keep them pointing to their original URLs instead of remapping them to yet non-existent local files (like the default `--remap-all` does), you need to run `mirror` with `--remap-open` option:

  ```bash
  hoardy-web mirror --remap-open --to ~/hoardy-web/mirror4 ~/hoardy-web/raw
  ```

  In practice, however, you probably won't want the exact behaviour of `wget -mpk`, since opening pages generated that way is likely to make your web browser try to access the Internet to load missing page requisites.
  To solve this problem, `hoardy-web` provides `--remap-semi` option, which does what `--remap-open` does, except it also remaps unavailable action links and page requisites into void links, fixing that problem:

  ```bash
  hoardy-web mirror --remap-semi --to ~/hoardy-web/mirror4 ~/hoardy-web/raw
  ```

  See the documentation for the `--remap-*` options below for more info.

  Obviously, using `--remap-open` or `--remap-semi` will make incremental updates to your mirror impossible.





  The simplest way to mirror a subset of your data is to run one of `hoardy-web organize --symlink --latest` commands described above, and then do something like this:

  ```bash
  hoardy-web mirror --to ~/hoardy-web/mirror5 ~/hoardy-web/latest/archiveofourown.org
  ```

  thus mirroring everything ever archived from <https://archiveofourown.org>.



  As an alternative to (or in combination with) keeping a symlink hierarchy of latest versions, you can load (an index of) an assortment of `WRR` files into `hoardy-web`'s memory but then `mirror` only select URLs (and all requisites needed to properly render those pages) by running something like:

  ```
  hoardy-web mirror \
    --to ~/hoardy-web/mirror6 ~/hoardy-web/raw/*/2023 \
    --root-url-prefix 'https://archiveofourown.org/works/3733123' \
    --root-url-prefix 'https://archiveofourown.org/works/30186441'
  ```

  See the documentation for the `--root-*` options below for more info and more `--root-*` variants.

  `hoardy-web` loads (indexes) `WRR` files pretty fast, so if you are running from an SSD, you can totally feed it years of `WRR` files and then only mirror a couple of URLs, and it will take a couple of seconds to finish anyway, since only a couple of files will get `scrub`bed.

  There is also `--depth` option, which works similarly to `wget`'s `--level` option in that it will follow all jump (`a href`) and action links accessible with no more than `--depth` browser navigations from recursion `--root-*`s and then `mirror` all those URLs (and their requisites) too.

  When using `--root-*` options, `--remap-open` works exactly like `wget`'s `--convert-links` in that it will only remap the URLs that are going to be mirrored and will keep the rest as-is.
  Similarly, `--remap-closed` will consider only the URLs reachable from the `--root-*`s in no more that `--depth` jumps as available.



  By default, files are read, queued, and then mirrored in the order they are specified on the command line, in lexicographic file system walk order when an argument is a directory.
  (See `--paths-*` and `--walk-*` options below if you want to change this.)

  However, the above rule does not apply to page requisites, those are always (with or without `--root-*`, regardless of `--paths-*` and `--walk-*` options) get mirrored just after their parent `HTML` document gets parsed and before that document gets written to disk.
  I.e., `mirror` will produce a new file containing an `HTML` document only after first producing all of its requisites.
  I.e., when mirroring into an empty directory, if you see `mirror` generated an `HTML` document, you can be sure that all of its requisites loaded (indexed) by this `mirror` invocation are rendered too.
  Meaning, you can go ahead and open it in your browser, even if `mirror` did not finish yet.

  Moreover, unlike all other sub-commands `mirror` handles duplication in its input files in a special way: it remembers the files it has already seen and ignores them when they are given the second time.
  (All other commands don't, they will just process the same file the second time, the third time, and so on.
  This is by design, other commands are designed to handle potentially enormous file hierarchies in constant memory.)

  The combination of all of the above means you can prioritize rendering of some documents over others by specifying them earlier on the command line and then, in a later argument, specifying their containing directory to allow `mirror` to also see their requisites and documents they link to.
  For instance,

  ```
  hoardy-web mirror \
    --to ~/hoardy-web/mirror7 \
    ~/hoardy-web/latest/archiveofourown.org/works__3733123*.wrr \
    ~/hoardy-web/latest/archiveofourown.org
  ```

  will mirror all of `~/hoardy-web/latest/archiveofourown.org`, but the web pages contained in files named `~/hoardy-web/latest/archiveofourown.org/works__3733123*.wrr` and their requisites will be mirrored first.

  This also works with `--root-*` options.
  E.g., the following

  ```
  hoardy-web mirror \
    --to ~/hoardy-web/mirror7 \
    ~/hoardy-web/latest/archiveofourown.org/works__3733123*.wrr \
    ~/hoardy-web/latest/archiveofourown.org \
    --root-url-prefix 'https://archiveofourown.org/works/'
  ```

  will mirror all pages those URLs start with `https://archiveofourown.org/works/` and all their requisites, but the pages contained in files named `~/hoardy-web/latest/archiveofourown.org/works__3733123*.wrr` and their requisites will be mirrored first.

  Finally, there is also the `--boring` option, which allows you to load some input `PATH`s without adding them as roots, even when no `--root-*` options are specified.
  E.g., the following

  ```
  hoardy-web mirror \
    --to ~/hoardy-web/mirror8 \
    --boring ~/hoardy-web/latest/i.imgur.com \
    --boring ~/hoardy-web/latest/archiveofourown.org \
    ~/hoardy-web/latest/archiveofourown.org/works__[0-9]*.wrr
  ```

  will load (an index of) everything under `~/hoardy-web/latest/i.imgur.com` and `~/hoardy-web/latest/archiveofourown.org` into memory but will only mirror the contents of `~/hoardy-web/latest/archiveofourown.org/works__[0-9]*.wrr` files and their requisites.

  When at least one `--root-*` option is specified, using `--boring` is equivalent to simply appending its argument to the end of the positional `PATH`s.



  By default, `hoardy-web mirror` runs with the implied `--latest` option, which mirrors the latest available version (visit) to each URL.
  Usually, this is fine, as most modern web-sites use versioned page requisites to improve caching.
  But it can produce broken results sometimes.
  For instance, when two different web pages share an unversioned `CSS` file and one those pages was recently revisited while the other was not, then, with the default `--latest`, only the latter version of the `CSS` file in question will be mirrored, making the older page broken.

  To fix this, you can run `mirror` with `--latest-hybrid` option

  ```
  hoardy-web mirror \
    --to ~/hoardy-web/mirror8 \
    --root-url-prefix 'https://en.wikipedia.org/wiki/'
    --latest-hybrid \
    ~/hoardy-web/raw
  ```

  which will mirror each web page with its date-vise closest available resource requisites.
  This takes quite a bit of memory, though, since `mirror` has to index and keep in memory references to all versions of all reqres to produce such hybrid results.

  Similarly, you can also mirror the `--oldest` available version of each URL:

  ```
  hoardy-web mirror \
    --to ~/hoardy-web/mirror9 \
    --root-url-prefix 'https://archiveofourown.org/works/'
    --oldest \
    ~/hoardy-web/raw
  ```

  or a version closest to a certain date:

  ```
  hoardy-web mirror \
    --to ~/hoardy-web/mirror9 \
    --root-url-prefix 'https://en.wikipedia.org/wiki/'
    --nearest 2020-10-31 \
    ~/hoardy-web/raw
  ```

  both of which also have `--*-hybrid` variants.

  There is also `--all`, which mirrors all available versions of all roots and `--depth`-reachable URLs.
  When using `--all`, you'll probably want to switch to a time-versioned output format, otherwise those default simply-numbered `hupq_n` outputs will be impossible to interpret:

  ```
  hoardy-web mirror \
    --to ~/hoardy-web/mirror9 \
    --root-url-prefix 'https://en.wikipedia.org/wiki/'
    --all \
    --output hupq_tn \
    ~/hoardy-web/raw
  ```



  Note that, by default, `hoardy-web mirror` runs with the implied `--hardlink` option, which makes it render and write each mirrored file to `<--to>/_content/<hash/based/path>.<ext>` and only then hardlink the result to `<--to>/<output/format/based/path>.<ext>` target destination.
  The `<hash/based/path>` is derived from the `sha256` hash of the generated file content.

  This trick saves quite a bit of space in many cases.
  E.g., when pages refer to the same resource requisites by slightly different URLs, same images and fonts get distributed via different CDN hosts, when you mirror `--all` visits to some URLs and many of those are absolutely identical, etc.

  You can change the destination those hash-based paths get written to by specifying `--content-to`.
  This allows you to easily share files between different mirrors:

  ```
  hoardy-web mirror \
    --content-to ~/hoardy-web/shared \
    --to ~/hoardy-web/mirror10 \
    --root-url-prefix 'https://archiveofourown.org/works/'
    ~/hoardy-web/raw

  hoardy-web mirror \
    --content-to ~/hoardy-web/shared \
    --to ~/hoardy-web/mirror11 \
    --root-url-prefix 'https://www.royalroad.com/'
    ~/hoardy-web/raw
  ```

  You can also control the path of the generated files by setting `--content-output`, e.g.:

  ```
  hoardy-web mirror \
    --content-output 'format:s/s/s'
    --content-to ~/storage/sha256 \
    --to ~/hoardy-web/mirror12 \
    ~/hoardy-web/raw
  ```

  `hoardy-web mirror` never overwrites any files under `--content-to`.
  It does, however, check that any existing files it references from there have the contents it expects, and generates errors if they do not.
  That is, you can set `--content-output` to anything and give any directory as `--content-to`, and `hoardy-web` will still ensure that the results are consistent, even when the `--content-to` cache is poisoned, or when different file contents compute to the same hash (produce a hash collision).

  Also note that, by default, `mirror` treats jump-links (`a href`, etc) and links to resource requisites quite differently, remappings jump-links to normal `--to` destination paths, while remapping resource requisites to their hash-based `--content-to` paths instead.
  This renders identical `HTML` and `CSS` files referencing identical resources into identical results, which also saves quite a bit of space.

  Note, however, that all of the above does make `mirror` slightly slower, since it needs to compute a lot of hashes and check contents of many files on disk.
  It also requires hardlink support on the target file system.
  Also, pointing `--content-to` outside of `--to` stops the mirrored results in `--to` from being self-contained.

  Which is why you can disable all of this by specifying `--copy`:

  ```
  hoardy-web mirror \
    --to ~/hoardy-web/mirror10 \
    --copy \
    ~/hoardy-web/raw
  ```

  Also, you can make it use `--symlink`s instead of hardlinks.
  Though, enabling `--symlink` also enables the `--absolute` option by default because browsers treat `file://` URLs pointing to symlinks as redirects.



  `hoardy-web` comes with a builtin web server that can do

  - archival of `WRR` captures produced by the [`Hoardy-Web` extension](../extension/) to disk;
    i.e., it can play a role of an archiving server for `Hoardy-Web`, replacing the [`hoardy-web-sas` simple archiving server](../simple_server/);

  - replay of `WRR` and other supported file formats via Wayback Machine-esque URLs like <http://127.0.0.1:3210/web/2/https://archiveofourown.org/works/3733123>;

  - do both at the same time, allowing newly archived URLs to be replayed immediately (after the `200 OK` response to the archiving `POST`).

  In other words, `hoardy-web serve` is, essentially, a combination of [`hoardy-web-sas` archiving server](../simple_server/) and an on-demand `hoardy-web mirror` which talks over `HTTP` instead of just dumping rendered documents to disk.
  For interactive use, this is not only more convenient than `hoardy-web mirror`, it's also usually much faster since required URL rewrites are much cheaper and no recursive requisite resource rendering is required here.
  That is, unlike `mirror`, `serve` is pretty snappy even on ancient hardware.

  When invoking `hoardy-web serve`, the argument to the `--archive-to` option will be used by the archiving server parts, while the positional `PATH` arguments will used by the replay server parts.
  That is,

  ```bash
  hoardy-web serve \
    --archive-to ~/hoardy-web/raw \
    ~/hoardy-web/raw/*/2024 \
    ../simple_server/pwebarc-dump \
    ~/Downloads/Hoardy-Web-export-* \
    mitmproxy.*.dump
  ```

  - will index and allow replay of all visits stored in files under `~/hoardy-web/raw/*/2024` and `../simple_server/pwebarc-dump`,
    as well as all files named `~/Downloads/Hoardy-Web-export-*` (which are, usually, `Hoardy-Web` exports) and
    files named `mitmproxy.*.dump` (which are probably `mitmproxy` dumps);
  - while dumping new captures given by the extension to `~/hoardy-web/raw`.

  When the argument to `--archive-to` and the first `PATH` are the same, you can specify `--implicit` --- or `-i` --- to simplify it:

  ```bash
  hoardy-web serve --implicit --archive-to ~/hoardy-web/raw

  hoardy-web serve --archive-to ~/hoardy-web/raw ~/hoardy-web/raw

  hoardy-web serve -i --to ~/hoardy-web/raw

  hoardy-web serve -i -t ~/hoardy-web/raw
  ```

  By default, `hoardy-web serve` runs with an implied `--all` option, which makes it keep the index of all given archives in memory, allowing arbitrary visits to be replayed.

  If you dislike this behaviour, you can run it with the `--latest` option instead

  ```bash
  hoardy-web serve --latest -i -t ~/hoardy-web/raw
  ```

  which will make it keep and allow replay only of the latest visit to each URL.
  This greatly improves resource consumption, but it also has the same caveats as `hoardy-web mirror --latest` (see above).
  With `--latest`, newly archived `WRR`s will elide older ones from the index, as expected.

  You can also disable indexing and replay completely by running it with `--no-replay`

  ```bath
  hoardy-web serve --no-replay --to ~/hoardy-web/raw
  ```

  which will make it essentially equivalent to `hoardy-web-sas`, except for `serve` having a customizable `--output` format.

  The listening address and port can be controlled with `--host` and `--port` options, exactly the same as `hoardy-web-sas`:

  ```bash
  hoardy-web serve --host 127.0.10.1 --port 4321 --archive-to ~/hoardy-web/raw
  ```

  Currently enabled features can be queried programmatically from `/hoardy-web/server-info` endpoint

  ```bash
  curl 'http://127.0.0.1:3210/hoardy-web/server-info'
  ```

  which returns a `JSON` like

  ```json
  {"version": 1, "dump_wrr": "/pwebarc/dump", "replay_any": "/web/{timestamp}/{url}", "replay_latest": "/web/2/{url}"}
  ```



  See the [`script` sub-directory](./script/) for examples that show how to use `pandoc` and/or `w3m` to turn `WRR` files into previews and readable plain-text that can viewed or listened to via other tools, or dump them into temporary raw data files that can then be immediately fed to `xdg-open` for one-click viewing.





  Inspect, search, organize, programmatically extract values and generate static website mirrors from, archive, view, and replay `HTTP` archives/dumps in `WRR` ("Web Request+Response", produced by the `Hoardy-Web` Web Extension browser add-on) and `mitmproxy` (`mitmdump`) file formats.

  Glossary: a `reqres` (`Reqres` when a Python type) is an instance of a structure representing `HTTP` request+response pair with some additional metadata.

  - options:
    - `--version`
    : show program's version number and exit
    - `-h, --help`
    : show this help message and exit
    - `--markdown`
    : show help messages formatted in Markdown

  - subcommands:
    - `{pprint,print,inspect,get,run,spawn,stream,find,organize,import,mirror,serve}`
      - `pprint (print, inspect)`
      : pretty-print given inputs
      - `get`
      : print values produced by evaluating given expressions on a given input
      - `run (spawn)`
      : spawn a process with temporary files generated from given expressions evaluated on given inputs
      - `stream`
      : stream lists containing values produced by evaluating given expressions on given inputs, a generalized `hoardy-web get`
      - `find`
      : print paths of inputs matching specified criteria
      - `organize`
      : programmatically copy/rename/move/hardlink/symlink given input files based on their metadata and/or contents
      - `import`
      : convert other `HTTP` archive formats into `WRR`
      - `mirror`
      : convert given inputs into a local offline static website mirror stored in interlinked files, a-la `wget -mpk`
      - `serve`
      : run an archiving server and/or serve given input files for replay over HTTP

  - filtering options:
    - `--ignore-case`
    : when filtering with `--*grep*`, match case-insensitively
    - `--case-sensitive`
    : when filtering with `--*grep*`, match case-sensitively
    - `--smart-case`
    : when filtering with `--*grep*`, match case-insensitively if there are no uppercase letters in the corresponding `*PATTERN*` option argument and case-sensitively otherwise; default

  - input filters; if none are specified, then all reqres from input `PATH`s will be taken; can be specified multiple times in arbitrary combinations; the resulting logical expression that will be checked is `all_of(before) and all_of(not_before) and all_of(after) and all_of(not_after) and any_of(protocol) and not any_of(not_protcol) and any_of(request_method) and not any_of(not_request_method) ... and any_of(grep) and not any_of(not_grep) and all_of(and_grep) and not all_of(not_and_grep) and all_of(ands) and any_of(ors)`:
    - `--before DATE`
    : accept reqres for processing when its `stime` is smaller than this; the `DATE` can be specified either as a number of seconds since UNIX epoch using `@<number>` format where `<number>` can be a floating point, or using one of the following formats:`YYYY-mm-DD HH:MM:SS[.NN*] (+|-)HHMM`, `YYYY-mm-DD HH:MM:SS[.NN*]`, `YYYY-mm-DD HH:MM:SS`, `YYYY-mm-DD HH:MM`, `YYYY-mm-DD`, `YYYY-mm`, `YYYY`; if no `(+|-)HHMM` part is specified, the `DATE` is assumed to be in local time; if other parts are unspecified they are inherited from `<year>-01-01 00:00:00.0`
    - `--not-before DATE`
    : accept reqres for processing when its `stime` is larger or equal than this; the `DATE` format is the same as above
    - `--after DATE`
    : accept reqres for processing when its `stime` is larger than this; the `DATE` format is the same as above
    - `--not-after DATE`
    : accept reqres for processing when its `stime` is smaller or equal than this; the `DATE` format is the same as above
    - `--protocol PROTOCOL`
    : accept reqres for processing when one of the given `PROTOCOL` option arguments is equal to its `protocol` (of `hoardy-web get --expr`, which see); in short, this option defines a whitelisted element rule
    - `--protocol-prefix PROTOCOL_PREFIX`
    : accept reqres for processing when one of the given `PROTOCOL_PREFIX` option arguments is a prefix of its `protocol` (of `hoardy-web get --expr`, which see); in short, this option defines a whitelisted element rule
    - `--protocol-re PROTOCOL_RE`
    : accept reqres for processing when one of the given `PROTOCOL_RE` regular expressions matches its `protocol` (of `hoardy-web get --expr`, which see); this option matches the given regular expression against the whole input value; to match against any part of the input value, use `.*<re>.*` or `^.*<re>.*$`; in short, this option defines a whitelisted element rule
    - `--not-protocol NOT_PROTOCOL`
    : accept reqres for processing when none of the given `NOT_PROTOCOL` option arguments are equal to its `protocol` (of `hoardy-web get --expr`, which see); in short, this option defines a blacklisted element rule
    - `--not-protocol-prefix NOT_PROTOCOL_PREFIX`
    : accept reqres for processing when none of the given `NOT_PROTOCOL_PREFIX` option arguments are a prefix of its `protocol` (of `hoardy-web get --expr`, which see); in short, this option defines a blacklisted element rule
    - `--not-protocol-re NOT_PROTOCOL_RE`
    : accept reqres for processing when none of the given `NOT_PROTOCOL_RE` regular expressions match its `protocol` (of `hoardy-web get --expr`, which see); this option matches the given regular expression against the whole input value; to match against any part of the input value, use `.*<re>.*` or `^.*<re>.*$`; in short, this option defines a blacklisted element rule
    - `--request-method REQUEST_METHOD, --method REQUEST_METHOD`
    : accept reqres for processing when one of the given `REQUEST_METHOD` option arguments is equal to its `request.method` (of `hoardy-web get --expr`, which see); in short, this option defines a whitelisted element rule
    - `--request-method-prefix REQUEST_METHOD_PREFIX, --method-prefix REQUEST_METHOD_PREFIX`
    : accept reqres for processing when one of the given `REQUEST_METHOD_PREFIX` option arguments is a prefix of its `request.method` (of `hoardy-web get --expr`, which see); in short, this option defines a whitelisted element rule
    - `--request-method-re REQUEST_METHOD_RE, --method-re REQUEST_METHOD_RE`
    : accept reqres for processing when one of the given `REQUEST_METHOD_RE` regular expressions matches its `request.method` (of `hoardy-web get --expr`, which see); this option matches the given regular expression against the whole input value; to match against any part of the input value, use `.*<re>.*` or `^.*<re>.*$`; in short, this option defines a whitelisted element rule
    - `--not-request-method NOT_REQUEST_METHOD, --not-method NOT_REQUEST_METHOD`
    : accept reqres for processing when none of the given `NOT_REQUEST_METHOD` option arguments are equal to its `request.method` (of `hoardy-web get --expr`, which see); in short, this option defines a blacklisted element rule
    - `--not-request-method-prefix NOT_REQUEST_METHOD_PREFIX`
    : accept reqres for processing when none of the given `NOT_REQUEST_METHOD_PREFIX` option arguments are a prefix of its `request.method` (of `hoardy-web get --expr`, which see); in short, this option defines a blacklisted element rule
    - `--not-request-method-re NOT_REQUEST_METHOD_RE`
    : accept reqres for processing when none of the given `NOT_REQUEST_METHOD_RE` regular expressions match its `request.method` (of `hoardy-web get --expr`, which see); this option matches the given regular expression against the whole input value; to match against any part of the input value, use `.*<re>.*` or `^.*<re>.*$`; in short, this option defines a blacklisted element rule
    - `--status STATUS`
    : accept reqres for processing when one of the given `STATUS` option arguments is equal to its `status` (of `hoardy-web get --expr`, which see); in short, this option defines a whitelisted element rule
    - `--status-prefix STATUS_PREFIX`
    : accept reqres for processing when one of the given `STATUS_PREFIX` option arguments is a prefix of its `status` (of `hoardy-web get --expr`, which see); in short, this option defines a whitelisted element rule
    - `--status-re STATUS_RE`
    : accept reqres for processing when one of the given `STATUS_RE` regular expressions matches its `status` (of `hoardy-web get --expr`, which see); this option matches the given regular expression against the whole input value; to match against any part of the input value, use `.*<re>.*` or `^.*<re>.*$`; in short, this option defines a whitelisted element rule
    - `--not-status NOT_STATUS`
    : accept reqres for processing when none of the given `NOT_STATUS` option arguments are equal to its `status` (of `hoardy-web get --expr`, which see); in short, this option defines a blacklisted element rule
    - `--not-status-prefix NOT_STATUS_PREFIX`
    : accept reqres for processing when none of the given `NOT_STATUS_PREFIX` option arguments are a prefix of its `status` (of `hoardy-web get --expr`, which see); in short, this option defines a blacklisted element rule
    - `--not-status-re NOT_STATUS_RE`
    : accept reqres for processing when none of the given `NOT_STATUS_RE` regular expressions match its `status` (of `hoardy-web get --expr`, which see); this option matches the given regular expression against the whole input value; to match against any part of the input value, use `.*<re>.*` or `^.*<re>.*$`; in short, this option defines a blacklisted element rule
    - `--url URL`
    : accept reqres for processing when one of the given `URL` option arguments is equal to its `net_url` (of `hoardy-web get --expr`, which see); Punycode UTS46 IDNAs, plain UNICODE IDNAs, percent-encoded URL components, and UNICODE URL components in arbitrary mixes and combinations are allowed; e.g. `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` will be silently normalized into its Punycode UTS46 and percent-encoded version of `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, which will then be matched against; in short, this option defines a whitelisted element rule
    - `--url-prefix URL_PREFIX`
    : accept reqres for processing when one of the given `URL_PREFIX` option arguments is a prefix of its `net_url` (of `hoardy-web get --expr`, which see); similarly to the previous option, arbitrary mixes of URL encodinds are allowed; in short, this option defines a whitelisted element rule
    - `--url-re URL_RE`
    : accept reqres for processing when one of the given `URL_RE` regular expressions matches its `net_url` or `pretty_net_url` (of `hoardy-web get --expr`, which see); only Punycode UTS46 IDNAs with percent-encoded URL components or plain UNICODE IDNAs with UNICODE URL components are allowed; regular expressions that use mixes of differently encoded parts will fail to match properly; this option matches the given regular expression against the whole input value; to match against any part of the input value, use `.*<re>.*` or `^.*<re>.*$`; in short, this option defines a whitelisted element rule
    - `--not-url NOT_URL`
    : accept reqres for processing when none of the given `NOT_URL` option arguments are equal to its `net_url` (of `hoardy-web get --expr`, which see); option argument format and caveats are idential to the `not-`less option above; in short, this option defines a blacklisted element rule
    - `--not-url-prefix NOT_URL_PREFIX`
    : accept reqres for processing when none of the given `NOT_URL_PREFIX` option arguments are a prefix of its `net_url` (of `hoardy-web get --expr`, which see); option argument format and caveats are idential to the `not-`less option above; in short, this option defines a blacklisted element rule
    - `--not-url-re NOT_URL_RE`
    : accept reqres for processing when none of the given `NOT_URL_RE` regular expressions match its `net_url` or `pretty_net_url` (of `hoardy-web get --expr`, which see); option argument format and caveats are idential to the `not-`less option above; in short, this option defines a blacklisted element rule
    - `--request-headers-or-grep OR_PATTERN, --request-headers-grep OR_PATTERN`
    : accept reqres for processing when at least one of the given `OR_PATTERN` option arguments is a substring of at least one of the elements of the list containing all `request.headers` (of `hoardy-web get --expr`, which see); each `HTTP` header of `*.headers` is matched as a single `<header_name>: <header_value>` value; at the moment, binary values are matched against given option arguments by encoding the latter into `UTF-8` first, which means that `*.headers` and `*.body` values that use encodings other than `UTF-8` are not guaranteed to match properly; in short, this option defines a whitelisted element rule
    - `--request-headers-or-grep-re OR_PATTERN_RE, --request-headers-grep-re OR_PATTERN_RE`
    : accept reqres for processing when at least one of the given `OR_PATTERN_RE` regular expressions matches a substring of at least one of the elements of the above list; matching caveats are the same as above; in short, this option defines a whitelisted element rule
    - `--not-request-headers-or-grep NOT_OR_PATTERN, --not-request-headers-grep NOT_OR_PATTERN`
    : accept reqres for processing when none of the given `NOT_OR_PATTERN` option arguments are substrings of any of the elements of the above list; matching caveats are the same as above; in short, this option defines a blacklisted element rule
    - `--not-request-headers-or-grep-re NOT_OR_PATTERN_RE, --not-request-headers-grep-re NOT_OR_PATTERN_RE`
    : accept reqres for processing when none of the given `NOT_OR_PATTERN_RE` regular expressions match any substrings of any of the elements of the above list; matching caveats are the same as above; in short, this option defines a blacklisted element rule
    - `--request-headers-and-grep AND_PATTERN`
    : accept reqres for processing when each of the given `AND_PATTERN` option arguments is a substring of some element of the above list; matching caveats are the same as above
    - `--request-headers-and-grep-re AND_PATTERN_RE`
    : accept reqres for processing when each of the given `AND_PATTERN_RE` regular expressions matches a substring of some element of the above list; matching caveats are the same as above
    - `--not-request-headers-and-grep NOT_AND_PATTERN`
    : accept reqres for processing when one or more of the given `NOT_AND_PATTERN` option arguments is not a substring of the elements of the above list; matching caveats are the same as above
    - `--not-request-headers-and-grep-re NOT_AND_PATTERN_RE`
    : accept reqres for processing when one or more of the given `NOT_AND_PATTERN_RE` regular expressions fails to match any substrings of the elements of the above list; matching caveats are the same as above
    - `--request-body-or-grep OR_PATTERN, --request-body-grep OR_PATTERN`
    : accept reqres for processing when at least one of the given `OR_PATTERN` option arguments is a substring of `request.body` (of `hoardy-web get --expr`, which see); at the moment, binary values are matched against given option arguments by encoding the latter into `UTF-8` first, which means that `*.headers` and `*.body` values that use encodings other than `UTF-8` are not guaranteed to match properly; in short, this option defines a whitelisted element rule
    - `--request-body-or-grep-re OR_PATTERN_RE, --request-body-grep-re OR_PATTERN_RE`
    : accept reqres for processing when at least one of the given `OR_PATTERN_RE` regular expressions matches a substring of `request.body`; matching caveats are the same as above; in short, this option defines a whitelisted element rule
    - `--not-request-body-or-grep NOT_OR_PATTERN, --not-request-body-grep NOT_OR_PATTERN`
    : accept reqres for processing when none of the given `NOT_OR_PATTERN` option arguments are substrings of `request.body`; matching caveats are the same as above; in short, this option defines a blacklisted element rule
    - `--not-request-body-or-grep-re NOT_OR_PATTERN_RE, --not-request-body-grep-re NOT_OR_PATTERN_RE`
    : accept reqres for processing when none of the given `NOT_OR_PATTERN_RE` regular expressions match any substrings of `request.body`; matching caveats are the same as above; in short, this option defines a blacklisted element rule
    - `--request-body-and-grep AND_PATTERN`
    : accept reqres for processing when each of the given `AND_PATTERN` option arguments is a substring of `request.body`; matching caveats are the same as above
    - `--request-body-and-grep-re AND_PATTERN_RE`
    : accept reqres for processing when each of the given `AND_PATTERN_RE` regular expressions matches a substring of `request.body`; matching caveats are the same as above
    - `--not-request-body-and-grep NOT_AND_PATTERN`
    : accept reqres for processing when one or more of the given `NOT_AND_PATTERN` option arguments is not a substring of `request.body`; matching caveats are the same as above
    - `--not-request-body-and-grep-re NOT_AND_PATTERN_RE`
    : accept reqres for processing when one or more of the given `NOT_AND_PATTERN_RE` regular expressions fails to match any substrings of `request.body`; matching caveats are the same as above
    - `--request-mime REQUEST_MIME`
    : accept reqres for processing when one of the given `REQUEST_MIME` option arguments is equal to its `request_mime` (of `hoardy-web get --expr`, which see); both canonical and non-canonical MIME types are allowed; e.g., giving `application/x-grip` or `application/gzip` will produce the same predicate; in short, this option defines a whitelisted element rule
    - `--request-mime-prefix REQUEST_MIME_PREFIX`
    : accept reqres for processing when one of the given `REQUEST_MIME_PREFIX` option arguments is a prefix of its `request_mime` (of `hoardy-web get --expr`, which see); given prefixes will only ever be matched against canonicalized MIME types; in short, this option defines a whitelisted element rule
    - `--request-mime-re REQUEST_MIME_RE`
    : accept reqres for processing when one of the given `REQUEST_MIME_RE` regular expressions matches its `request_mime` (of `hoardy-web get --expr`, which see); given regular expressions will only ever be matched against canonicalized MIME types; this option matches the given regular expression against the whole input value; to match against any part of the input value, use `.*<re>.*` or `^.*<re>.*$`; in short, this option defines a whitelisted element rule
    - `--not-request-mime NOT_REQUEST_MIME`
    : accept reqres for processing when none of the given `NOT_REQUEST_MIME` option arguments are equal to its `request_mime` (of `hoardy-web get --expr`, which see); option argument format and caveats are idential to the `not-`less option above; in short, this option defines a blacklisted element rule
    - `--not-request-mime-prefix NOT_REQUEST_MIME_PREFIX`
    : accept reqres for processing when none of the given `NOT_REQUEST_MIME_PREFIX` option arguments are a prefix of its `request_mime` (of `hoardy-web get --expr`, which see); option argument format and caveats are idential to the `not-`less option above; in short, this option defines a blacklisted element rule
    - `--not-request-mime-re NOT_REQUEST_MIME_RE`
    : accept reqres for processing when none of the given `NOT_REQUEST_MIME_RE` regular expressions match its `request_mime` (of `hoardy-web get --expr`, which see); option argument format and caveats are idential to the `not-`less option above; in short, this option defines a blacklisted element rule
    - `--response-headers-or-grep OR_PATTERN, --response-headers-grep OR_PATTERN`
    : accept reqres for processing when at least one of the given `OR_PATTERN` option arguments is a substring of at least one of the elements of the list containing all `response.headers` (of `hoardy-web get --expr`, which see); each `HTTP` header of `*.headers` is matched as a single `<header_name>: <header_value>` value; at the moment, binary values are matched against given option arguments by encoding the latter into `UTF-8` first, which means that `*.headers` and `*.body` values that use encodings other than `UTF-8` are not guaranteed to match properly; in short, this option defines a whitelisted element rule
    - `--response-headers-or-grep-re OR_PATTERN_RE, --response-headers-grep-re OR_PATTERN_RE`
    : accept reqres for processing when at least one of the given `OR_PATTERN_RE` regular expressions matches a substring of at least one of the elements of the above list; matching caveats are the same as above; in short, this option defines a whitelisted element rule
    - `--not-response-headers-or-grep NOT_OR_PATTERN, --not-response-headers-grep NOT_OR_PATTERN`
    : accept reqres for processing when none of the given `NOT_OR_PATTERN` option arguments are substrings of any of the elements of the above list; matching caveats are the same as above; in short, this option defines a blacklisted element rule
    - `--not-response-headers-or-grep-re NOT_OR_PATTERN_RE, --not-response-headers-grep-re NOT_OR_PATTERN_RE`
    : accept reqres for processing when none of the given `NOT_OR_PATTERN_RE` regular expressions match any substrings of any of the elements of the above list; matching caveats are the same as above; in short, this option defines a blacklisted element rule
    - `--response-headers-and-grep AND_PATTERN`
    : accept reqres for processing when each of the given `AND_PATTERN` option arguments is a substring of some element of the above list; matching caveats are the same as above
    - `--response-headers-and-grep-re AND_PATTERN_RE`
    : accept reqres for processing when each of the given `AND_PATTERN_RE` regular expressions matches a substring of some element of the above list; matching caveats are the same as above
    - `--not-response-headers-and-grep NOT_AND_PATTERN`
    : accept reqres for processing when one or more of the given `NOT_AND_PATTERN` option arguments is not a substring of the elements of the above list; matching caveats are the same as above
    - `--not-response-headers-and-grep-re NOT_AND_PATTERN_RE`
    : accept reqres for processing when one or more of the given `NOT_AND_PATTERN_RE` regular expressions fails to match any substrings of the elements of the above list; matching caveats are the same as above
    - `--response-body-or-grep OR_PATTERN, --response-body-grep OR_PATTERN`
    : accept reqres for processing when at least one of the given `OR_PATTERN` option arguments is a substring of `response.body` (of `hoardy-web get --expr`, which see); at the moment, binary values are matched against given option arguments by encoding the latter into `UTF-8` first, which means that `*.headers` and `*.body` values that use encodings other than `UTF-8` are not guaranteed to match properly; in short, this option defines a whitelisted element rule
    - `--response-body-or-grep-re OR_PATTERN_RE, --response-body-grep-re OR_PATTERN_RE`
    : accept reqres for processing when at least one of the given `OR_PATTERN_RE` regular expressions matches a substring of `response.body`; matching caveats are the same as above; in short, this option defines a whitelisted element rule
    - `--not-response-body-or-grep NOT_OR_PATTERN, --not-response-body-grep NOT_OR_PATTERN`
    : accept reqres for processing when none of the given `NOT_OR_PATTERN` option arguments are substrings of `response.body`; matching caveats are the same as above; in short, this option defines a blacklisted element rule
    - `--not-response-body-or-grep-re NOT_OR_PATTERN_RE, --not-response-body-grep-re NOT_OR_PATTERN_RE`
    : accept reqres for processing when none of the given `NOT_OR_PATTERN_RE` regular expressions match any substrings of `response.body`; matching caveats are the same as above; in short, this option defines a blacklisted element rule
    - `--response-body-and-grep AND_PATTERN`
    : accept reqres for processing when each of the given `AND_PATTERN` option arguments is a substring of `response.body`; matching caveats are the same as above
    - `--response-body-and-grep-re AND_PATTERN_RE`
    : accept reqres for processing when each of the given `AND_PATTERN_RE` regular expressions matches a substring of `response.body`; matching caveats are the same as above
    - `--not-response-body-and-grep NOT_AND_PATTERN`
    : accept reqres for processing when one or more of the given `NOT_AND_PATTERN` option arguments is not a substring of `response.body`; matching caveats are the same as above
    - `--not-response-body-and-grep-re NOT_AND_PATTERN_RE`
    : accept reqres for processing when one or more of the given `NOT_AND_PATTERN_RE` regular expressions fails to match any substrings of `response.body`; matching caveats are the same as above
    - `--response-mime RESPONSE_MIME`
    : accept reqres for processing when one of the given `RESPONSE_MIME` option arguments is equal to its `response_mime` (of `hoardy-web get --expr`, which see); both canonical and non-canonical MIME types are allowed; e.g., giving `application/x-grip` or `application/gzip` will produce the same predicate; in short, this option defines a whitelisted element rule
    - `--response-mime-prefix RESPONSE_MIME_PREFIX`
    : accept reqres for processing when one of the given `RESPONSE_MIME_PREFIX` option arguments is a prefix of its `response_mime` (of `hoardy-web get --expr`, which see); given prefixes will only ever be matched against canonicalized MIME types; in short, this option defines a whitelisted element rule
    - `--response-mime-re RESPONSE_MIME_RE`
    : accept reqres for processing when one of the given `RESPONSE_MIME_RE` regular expressions matches its `response_mime` (of `hoardy-web get --expr`, which see); given regular expressions will only ever be matched against canonicalized MIME types; this option matches the given regular expression against the whole input value; to match against any part of the input value, use `.*<re>.*` or `^.*<re>.*$`; in short, this option defines a whitelisted element rule
    - `--not-response-mime NOT_RESPONSE_MIME`
    : accept reqres for processing when none of the given `NOT_RESPONSE_MIME` option arguments are equal to its `response_mime` (of `hoardy-web get --expr`, which see); option argument format and caveats are idential to the `not-`less option above; in short, this option defines a blacklisted element rule
    - `--not-response-mime-prefix NOT_RESPONSE_MIME_PREFIX`
    : accept reqres for processing when none of the given `NOT_RESPONSE_MIME_PREFIX` option arguments are a prefix of its `response_mime` (of `hoardy-web get --expr`, which see); option argument format and caveats are idential to the `not-`less option above; in short, this option defines a blacklisted element rule
    - `--not-response-mime-re NOT_RESPONSE_MIME_RE`
    : accept reqres for processing when none of the given `NOT_RESPONSE_MIME_RE` regular expressions match its `response_mime` (of `hoardy-web get --expr`, which see); option argument format and caveats are idential to the `not-`less option above; in short, this option defines a blacklisted element rule
    - `--or-grep OR_PATTERN, --grep OR_PATTERN`
    : accept reqres for processing when at least one of the given `OR_PATTERN` option arguments is a substring of at least one of the elements of the list containing `raw_url`, `url`, `pretty_url`, all `request.headers`, `request.body`, all `response.headers`, and `response.body` (of `hoardy-web get --expr`, which see); each `HTTP` header of `*.headers` is matched as a single `<header_name>: <header_value>` value; at the moment, binary values are matched against given option arguments by encoding the latter into `UTF-8` first, which means that `*.headers` and `*.body` values that use encodings other than `UTF-8` are not guaranteed to match properly; in short, this option defines a whitelisted element rule
    - `--or-grep-re OR_PATTERN_RE, --grep-re OR_PATTERN_RE`
    : accept reqres for processing when at least one of the given `OR_PATTERN_RE` regular expressions matches a substring of at least one of the elements of the above list; matching caveats are the same as above; in short, this option defines a whitelisted element rule
    - `--not-or-grep NOT_OR_PATTERN, --not-grep NOT_OR_PATTERN`
    : accept reqres for processing when none of the given `NOT_OR_PATTERN` option arguments are substrings of any of the elements of the above list; matching caveats are the same as above; in short, this option defines a blacklisted element rule
    - `--not-or-grep-re NOT_OR_PATTERN_RE, --not-grep-re NOT_OR_PATTERN_RE`
    : accept reqres for processing when none of the given `NOT_OR_PATTERN_RE` regular expressions match any substrings of any of the elements of the above list; matching caveats are the same as above; in short, this option defines a blacklisted element rule
    - `--and-grep AND_PATTERN`
    : accept reqres for processing when each of the given `AND_PATTERN` option arguments is a substring of some element of the above list; matching caveats are the same as above
    - `--and-grep-re AND_PATTERN_RE`
    : accept reqres for processing when each of the given `AND_PATTERN_RE` regular expressions matches a substring of some element of the above list; matching caveats are the same as above
    - `--not-and-grep NOT_AND_PATTERN`
    : accept reqres for processing when one or more of the given `NOT_AND_PATTERN` option arguments is not a substring of the elements of the above list; matching caveats are the same as above
    - `--not-and-grep-re NOT_AND_PATTERN_RE`
    : accept reqres for processing when one or more of the given `NOT_AND_PATTERN_RE` regular expressions fails to match any substrings of the elements of the above list; matching caveats are the same as above
    - `--and EXPR`
    : accept reqres for processing when all of the given expressions of the same format as `hoardy-web get --expr` (which see) evaluate to `true`
    - `--or EXPR`
    : accept reqres for processing when some of the given expressions of the same format as `hoardy-web get --expr` (which see) evaluate to `true`



  Pretty-print given inputs to stdout.

  - options:
    - `-q, --quiet`
    : don't print end-of-filtering warnings to stderr
    - `-u, --unabridged`
    : print all data in full
    - `--abridged`
    : shorten long strings for brevity, useful when you want to visually scan through batch data dumps; default

  - error handling:
    - `--errors {fail,skip,ignore}`
    : when an error occurs:
      - `fail`: report failure and stop the execution; default
      - `skip`: report failure but skip the reqres that produced it from the output and continue
      - `ignore`: `skip`, but don't report the failure

  - path ordering:
    - `--paths-given-order`
    : `argv` and `--stdin0` `PATH`s are processed in the order they are given; default
    - `--paths-sorted`
    : `argv` and `--stdin0` `PATH`s are processed in lexicographic order
    - `--paths-reversed`
    : `argv` and `--stdin0` `PATH`s are processed in reverse lexicographic order
    - `--walk-fs-order`
    : recursive file system walk is done in the order `readdir(2)` gives results
    - `--walk-sorted`
    : recursive file system walk is done in lexicographic order; default
    - `--walk-reversed`
    : recursive file system walk is done in reverse lexicographic order

  - input loading:
    - `--load-any`
    : for each given input `PATH`, decide which loader to use based on its file extension; default
    - `--load-wrr`
    : load all inputs using the single-`WRR` per-file loader
    - `--load-wrrb`
    : load all inputs using the `WRR` bundle loader, this will load separate `WRR` files as single-`WRR` bundles too
    - `--load-mitmproxy`
    : load inputs using the `mitmproxy` dump loader
    - `--stdin0`
    : read zero-terminated `PATH`s from stdin, these will be processed after `PATH`s specified as command-line arguments
    - `PATH`
    : inputs, can be a mix of files and directories (which will be traversed recursively)

  - `MIME` type sniffing; this controls the use of the [`mimesniff` algorithm](https://mimesniff.spec.whatwg.org/); for this sub-command this simply populates the `potentially` lists in the output in various ways:
    - `--sniff-default`
    : run `mimesniff` when the spec says it should be run; i.e. trust `Content-Type` `HTTP` headers most of the time; default
    - `--sniff-force`
    : run `mimesniff` regardless of what `Content-Type`  and `X-Content-Type-Options` `HTTP` headers say; i.e. for each reqres, run `mimesniff` algorithm on the `Content-Type` `HTTP` header and the actual contents of `(request|response).body` (depending on the first argument of `scrub`) to determine what the body actually contains, then interpret the data as intersection of what `Content-Type` and `mimesniff` claim it to be; e.g. if `Content-Type` says `text/plain` but `mimesniff` says `text/plain or text/javascript`, interpret it as `text/plain`
    - `--sniff-paranoid`
    : do what `--sniff-force` does, but interpret the results in the most paranoid way possible; e.g. if `Content-Type` says `text/plain` but `mimesniff` says `text/plain or text/javascript`, interpret it as `text/plain or text/javascript`; which, for instance, will then make `scrub` with `-scripts` censor it out, since it can be interpreted as a script



  Print results produced by evaluating given `EXPR`essions on a given input to stdout.

  Algorithm:

  - Load input `PATH`;
  - evaluate all `EXPR` expressions on the resulting reqres;
  - print all the results to stdout, terminating each value as specified.

  The end.

  - positional arguments:
    - `PATH`
    : input `WRR` file path

  - `MIME` type sniffing; this controls the use of the [`mimesniff` algorithm](https://mimesniff.spec.whatwg.org/); for this sub-command higher values make the `scrub` function (which see) censor out more things when `-unknown`, `-styles`, or `-scripts` options are set; in particular, at the moment, with `--sniff-paranoid` and `-scripts` most plain text files will be censored out as potential `JavaScript`:
    - `--sniff-default`
    : run `mimesniff` when the spec says it should be run; i.e. trust `Content-Type` `HTTP` headers most of the time; default
    - `--sniff-force`
    : run `mimesniff` regardless of what `Content-Type`  and `X-Content-Type-Options` `HTTP` headers say; i.e. for each reqres, run `mimesniff` algorithm on the `Content-Type` `HTTP` header and the actual contents of `(request|response).body` (depending on the first argument of `scrub`) to determine what the body actually contains, then interpret the data as intersection of what `Content-Type` and `mimesniff` claim it to be; e.g. if `Content-Type` says `text/plain` but `mimesniff` says `text/plain or text/javascript`, interpret it as `text/plain`
    - `--sniff-paranoid`
    : do what `--sniff-force` does, but interpret the results in the most paranoid way possible; e.g. if `Content-Type` says `text/plain` but `mimesniff` says `text/plain or text/javascript`, interpret it as `text/plain or text/javascript`; which, for instance, will then make `scrub` with `-scripts` censor it out, since it can be interpreted as a script

  - expression evaluation:
    - `--expr-fd INT`
    : file descriptor to which the results of evaluations of the following `--expr`s computations should be written; can be specified multiple times, thus separating different `--expr`s into different output streams; default: `1`, i.e. `stdout`
    - `-e EXPR, --expr EXPR`
    : an expression to compute; can be specified multiple times in which case computed outputs will be printed sequentially (see also "printing" options below); the default depends on options below; each `EXPR` describes a state-transformer (pipeline) which starts from value `None` and evaluates a script built from the following:
      - constants and functions:
        - `es`: replace `None` value with an empty string `""`
        - `eb`: replace `None` value with an empty byte string `b""`
        - `false`: replace `None` value with `False`
        - `true`: replace `None` value with `True`
        - `missing`: `True` if the value is `None`
        - `0`: replace `None` value with `0`
        - `1`: replace `None` value with `1`
        - `not`: apply logical `not` to value
        - `len`: apply `len` to value
        - `str`: cast value to `str` or fail
        - `bytes`: cast value to `bytes` or fail
        - `bool`: cast value to `bool` or fail
        - `int`: cast value to `int` or fail
        - `float`: cast value to `float` or fail
        - `echo`: replace the value with the given string
        - `quote`: URL-percent-encoding quote value
        - `quote_plus`: URL-percent-encoding quote value and replace spaces with `+` symbols
        - `unquote`: URL-percent-encoding unquote value
        - `unquote_plus`: URL-percent-encoding unquote value and replace `+` symbols with spaces
        - `to_ascii`: encode `str` value into `bytes` with "ascii" codec, do nothing if the value is already `bytes`
        - `to_utf8`: encode `str` value into `bytes` with "utf-8" codec, do nothing if the value is already `bytes`
        - `to_hex`: replace `bytes` value with its hexadecimal `str` representation
        - `from_hex`: replace hexadecimal `str` value with its decoded `bytes` value
        - `sha256`: replace `bytes` value with its `sha256` hash digest
        - `~=`: check if the current value matches the regular exprission `arg`
        - `==`: apply `== arg`, `arg` is cast to the same type as the current value
        - `!=`: apply `!= arg`, similarly
        - `<`: apply `< arg`, similarly
        - `<=`: apply `<= arg`, similarly
        - `>`: apply `> arg`, similarly
        - `>=`: apply `>= arg`, similarly
        - `add_prefix`: add prefix to the current value
        - `add_suffix`: add suffix to the current value
        - `take_prefix`: take first `arg` characters or list elements from the current value
        - `take_suffix`: take last `arg` characters or list elements  from the current value
        - `abbrev`: leave the current value as-is if if its length is less or equal than `arg` characters, otherwise take first `arg/2` followed by last `arg/2` characters
        - `abbrev_each`: `abbrev arg` each element in a value `list`
        - `replace`: replace all occurences of the first argument in the current value with the second argument, casts arguments to the same type as the current value
        - `parse_path`: parse a URL path component `str` into `path_parts` `list`
        - `unparse_path`: encode `path_parts` `list` into a URL path component `str`
        - `parse_query`: parse a URL query component `str` into `query_parts` `list`
        - `unparse_query`: encode `query_parts` `list` into a URL query component `str`
        - `pp_to_path`: encode `*path_parts` `list` into a POSIX path, quoting as little as needed
        - `qsl_to_path`: encode `query_parts` `list` into a POSIX path, quoting as little as needed
        - `scrub`: scrub the value by optionally rewriting links and/or removing dynamic content from it; what gets done depends on the `MIME` type of the value itself and the scrubbing options described below; this function takes two arguments:
              - the first must be either of `request|response`, it controls which `HTTP` headers `scrub` should inspect to help it detect the `MIME` type;
              - the second is either `defaults` or ","-separated string of tokens which control the scrubbing behaviour:
                - `(+|-|*|/|&)jumps` controls how jump-links (`a href`, `area href`, and similar `HTML` tag attributes) should be remapped or censored out:
                  - `+` rewrites their values into full URLs, e.g. `<a href="/path?query">` -> `a href="https://example.org/path?query">`;
                  - `-` "voids" all of them, i.e. rewrites them to `javascript:void(0)` and empty `data:` URLs;
                  - `*` rewrites links in an "open"-ended way, i.e. points them to locally mirrored versions of their URLs when available and leaves them pointing to their original URL otherwise; this is only supported when `scrub` is used with `mirror` sub-command; under other sub-commands this is equivalent to `+`;
                  - `/` rewrites links in a "close"-ended way, i.e. points them to locally mirrored versions of their URLs when available and voids them otherwise; this is only supported when `scrub` is used with `mirror` sub-command; under other sub-commands this is equivalent to `-`;
                  - `&` rewrites links in a "close"-ended way like `/` does, except this option uses fallbacks to remap unavailable URLs whenever possible; this is only supported when `scrub` is used with `mirror` sub-command; under other sub-commands this is equivalent to `-`; see the documentation of the `--remap-all` option for more info;
                - `(+|-|*|/|&)actions` controls how action-links (`a ping`, `form action`, and similar `HTML` tag attributes) should be remapped or censored out; same rewrite options as above;
                - `(+|-|*|/|&)reqs` controls how references to page requisites (`img src`, `iframe src`, and similar `HTML` tag attributes, as well as `link src` attributes which have `rel` attribute of their `HTML` tag set to `stylesheet` or `icon`, `CSS` `url` references, etc) should be remapped or censored out; same rewrite options as above;
                - `(+|-|*|/|&)all_refs` is equivalent to setting all of `jumps`, `actions`, and `reqs` simultaneously;
                - `(+|-)styles` controls whether `CSS` stylesheets (both separate files and `HTML` tags and attributes) should be kept in or censored out;
                - `(+|-)scripts` controls whether `JavaScript` (both separate files and `HTML` tags and attributes) should be kept in or censored out;
                - `(+|-)iepragmas` controls whether Internet Explorer's `HTML` pragmas should be kept in or censored out;
                - `(+|-)iframes` controls whether `<iframe>` `HTML` tags should be kept in or censored out;
                - `(+|-)prefetches` controls whether `HTML` content prefetch `link` tags should be kept in or censored out;
                - `(+|-)tracking` controls whether other tracking `HTML` tags and attributes (like `a ping`) should be kept in or censored out;
                - `(+|-)navigations` controls whether automatic navigations (`Refresh` `HTTP` headers and `<meta http-equiv>` `HTML` tags) should be kept in or censored out;
                - `(+|-)all_dyns` is equivalent to setting all of `styles`, `scripts`, `iepragmas`, `iframes`, `prefetches`, `tracking`, and `navigations` simultaneously;
                - `(+|-)interpret_noscript` controls whether the contents of `noscript` tags should be inlined when `-scripts` is set;
                - `(+|-)unknown` controls if the data with unknown content types should passed to the output unchanged or censored out (respectively);
                - `(+|-)verbose` controls whether tag censoring controlled by the above options is to be reported in the output (as comments) or stuff should be wiped from existence without evidence instead;
                - `(+|-)whitespace` controls whether `HTML` and `CSS` renderers should keep the original whitespace as-is or collapse it away;
                - `(+|-)optional_tags` controls whether `HTML` renderer should put optional `HTML` tags into the output or skip them;
                - `(+|-)indent` controls whether `HTML` and `CSS` renderers should indent their outputs (where whitespace placement in the original markup allows for it) or not;
                - `+pretty` is an alias for `-whitespace,+indent` which produces the prettiest possible human-readable output that keeps the original whitespace semantics;
                - `-pretty` is an alias for `+whitespace,-indent` which produces the approximation of the original markup with censoring applied;
                - `+debug` is a variant of `+pretty` that also uses a much more aggressive version of `indent` that ignores the semantics of original whitespace placement, i.e. it indents `<p>not<em>sep</em>arated</p>` as if there was whitespace before and after `p`, `em`, `/em`, and `/p` tags; this is useful for debugging;
                - `-debug` is a noop;
              - the `defaults` are:
                - `*jumps,&actions,&reqs`, because these produce a self-contained result that can be fed into another tool --- be it a web browser or `pandoc` --- without that tool trying to access the Internet;
                - `-prefetches,-tracking,-navigations`, because these ensure the result will not try to prefetch or track anything, or re-navigate elsewhere, when loaded in a web browser;
                - `+styles,+iframes`, because these are are `scrub`bed properly;
                - `-scripts`, because `scrub`bing of `JavaScript` (code whitelisting) is not supported yet;
                - `-iepragmas`, because censoring of contents of such pragmas is not supported yet;
                - `+interpret_noscript`, because this usually helps;
                - `+verbose`, because this allows you to inspect the generated output and see what `hoardy-web` did to it, i.e., this minimizes surprises;
                - `+whitespace,-indent`, to keep the output as close to the original as possible;
                - `+optional_tags`, because many tools fail to parse minimized `HTML` properly;
                - `+unknown` which keeps data of unknown content `MIME` types as-is;
              - note however, that most `--remap-*` options set different defaults;

      - reqres fields, these work the same way as constants above, i.e. they replace current value of `None` with field's value, if reqres is missing the field in question, which could happen for `response*` fields, the result is `None`:
        - `version`: WEBREQRES format version; int
        - `agent`: `+`-separated list of applications that produced this reqres; str
        - `protocol`: protocol; e.g. `"HTTP/1.1"`, `"HTTP/2.0"`; str
        - `request.started_at`: request start time in seconds since 1970-01-01 00:00; TimeStamp
        - `request.method`: request `HTTP` method; e.g. `"GET"`, `"POST"`, etc; str
        - `request.url`: request URL, including the `fragment`/hash part; str
        - `request.headers`: request headers; list[tuple[str, bytes]]
        - `request.complete`: is request body complete?; bool
        - `request.body`: request body; bytes
        - `response.started_at`: response start time in seconds since 1970-01-01 00:00; TimeStamp
        - `response.code`: `HTTP` response code; e.g. `200`, `404`, etc; int
        - `response.reason`: `HTTP` response reason; e.g. `"OK"`, `"Not Found"`, etc; usually empty for Chromium and filled for Firefox; str
        - `response.headers`: response headers; list[tuple[str, bytes]]
        - `response.complete`: is response body complete?; bool
        - `response.body`: response body; Firefox gives raw bytes, Chromium gives UTF-8 encoded strings; bytes | str
        - `finished_at`: request completion time in seconds since 1970-01-01 00:00; TimeStamp
        - `websocket`: a list of WebSocket frames
      - derived attributes:
        - `fs_path`: file system path for the WRR file containing this reqres; str | bytes | None
        - `raw_url`: aliast for `request.url`; str
        - `method`: aliast for `request.method`; str
        - `qtime`: aliast for `request.started_at`; mnemonic: "reQuest TIME"; seconds since UNIX epoch; TimeStamp
        - `qtime_ms`: `qtime` in milliseconds rounded down to nearest integer; milliseconds since UNIX epoch; int
        - `qtime_msq`: three least significant digits of `qtime_ms`; int
        - `qyear`: year number of `gmtime(qtime)` (UTC year number of `qtime`); int
        - `qmonth`: month number of `gmtime(qtime)`; int
        - `qday`: day of the month of `gmtime(qtime)`; int
        - `qhour`: hour of `gmtime(qtime)` in 24h format; int
        - `qminute`: minute of `gmtime(qtime)`; int
        - `qsecond`: second of `gmtime(qtime)`; int
        - `stime`: `response.started_at` if there was a response, `finished_at` otherwise; mnemonic: "reSponse TIME"; seconds since UNIX epoch; TimeStamp
        - `stime_ms`: `stime` in milliseconds rounded down to nearest integer; milliseconds since UNIX epoch; int
        - `stime_msq`: three least significant digits of `stime_ms`; int
        - `syear`: similar to `qyear`, but for `stime`; int
        - `smonth`: similar to `qmonth`, but for `stime`; int
        - `sday`: similar to `qday`, but for `stime`; int
        - `shour`: similar to `qhour`, but for `stime`; int
        - `sminute`: similar to `qminute`, but for `stime`; int
        - `ssecond`: similar to `qsecond`, but for `stime`; int
        - `ftime`: aliast for `finished_at`; seconds since UNIX epoch; TimeStamp
        - `ftime_ms`: `ftime` in milliseconds rounded down to nearest integer; milliseconds since UNIX epoch; int
        - `ftime_msq`: three least significant digits of `ftime_ms`; int
        - `fyear`: similar to `qyear`, but for `ftime`; int
        - `fmonth`: similar to `qmonth`, but for `ftime`; int
        - `fday`: similar to `qday`, but for `ftime`; int
        - `fhour`: similar to `qhour`, but for `ftime`; int
        - `fminute`: similar to `qminute`, but for `ftime`; int
        - `fsecond`: similar to `qsecond`, but for `ftime`; int
        - `net_url`: a variant of `raw_url` that uses Punycode UTS46 IDNA encoded `net_hostname`, has all unsafe characters of `raw_path` and `raw_query` quoted, and comes without the `fragment`/hash part; this is the URL that actually gets sent to an `HTTP` server when you request `raw_url`; str
        - `url`: `net_url` with `fragment`/hash part appended; str
        - `pretty_net_url`: a variant of `raw_url` that uses UNICODE IDNA `hostname` without Punycode, minimally quoted `mq_path` and `mq_query`, and comes without the `fragment`/hash part; this is a human-readable version of `net_url`; str
        - `pretty_url`: `pretty_net_url` with `fragment`/hash part appended; str
        - `pretty_net_nurl`: a variant of `pretty_net_url` that uses `mq_npath` instead of `mq_path` and `mq_nquery` instead of `mq_query`; i.e. this is `pretty_net_url` with normalized path and query; str
        - `pretty_nurl`: `pretty_net_nurl` with `fragment`/hash part appended; str
        - `scheme`: scheme part of `raw_url`; e.g. `http`, `https`, etc; str
        - `raw_hostname`: hostname part of `raw_url` as it is recorded in the reqres; str
        - `net_hostname`: hostname part of `raw_url`, encoded as Punycode UTS46 IDNA; this is what actually gets sent to the server; ASCII str
        - `hostname`: `net_hostname` decoded back into UNICODE; this is the canonical hostname representation for which IDNA-encoding and decoding are bijective; UNICODE str
        - `rhostname`: `hostname` with the order of its parts reversed; e.g. `"www.example.org"` -> `"com.example.www"`; str
        - `port`: port part of `raw_url`; str
        - `netloc`: netloc part of `raw_url`; i.e., in the most general case, `<username>:<password>@<hostname>:<port>`; str
        - `raw_path`: raw path part of `raw_url` as it is recorded is the reqres; e.g. `"https://www.example.org"` -> `""`, `"https://www.example.org/"` -> `"/"`, `"https://www.example.org/index.html"` -> `"/index.html"`; str
        - `path_parts`: component-wise unquoted "/"-split `raw_path`; list[str]
        - `path`: `path_parts` turned back into a quoted string, i.e. `raw_path` normalized like browsers do it; str
        - `npath_parts`: `path_parts` with empty components removed and dots and double dots interpreted away; e.g. `"https://www.example.org"` -> `[]`, `"https://www.example.org/"` -> `[]`, `"https://www.example.org/index.html"` -> `["index.html"]` , `"https://www.example.org/skipped/.//../used/"` -> `["used"]`; list[str]
        - `mq_path`: `path_parts` turned back into a minimally-quoted string; str
        - `mq_npath`: `npath_parts` turned back into a minimally-quoted string; str
        - `raw_query`: query part of `raw_url`, i.e. everything after the `?` character and before the `#` character; str
        - `query_parts`: parsed and component-wise unquoted `raw_query`; list[tuple[str, str | None]]
        - `query`: `query_parts` turned back into a quoted string, i.e. `raw_query` normalized like browsers do it; str
        - `query_nparts`: `query_parts` with empty query parameters removed; list[tuple[str, str]]
        - `mq_query`: `query_parts` turned back into a minimally-quoted string appropriate for use in filenames; str
        - `mq_nquery`: `query_ne_parts` turned back into a minimally-quoted string appropriate for use in filenames; str
        - `oqm`: optional query mark: `?` character if `query` is non-empty, an empty string otherwise; str
        - `fragment`: fragment (hash) part of the url; str
        - `ofm`: optional fragment mark: `#` character if `fragment` is non-empty, an empty string otherwise; str
        - `status`: `"I"` or  `"C"` for `request.complete` (`I` for `false` , `C` for `true`) followed by either `"N"` when `response is None`, or `str(response.code)` followed by `"I"` or  `"C"` for `response.complete`; e.g. `C200C` (all "OK"), `CN` (request was sent, but it got no response), `I200C` (partial request with complete "OK" response), `C200I` (complete request with incomplete response, e.g. if download was interrupted), `C404C` (complete request with complete "Not Found" response), etc; str
        - `request_mime`: `request.body` `MIME` type, note the underscore, this is not a field of `request`, this is a derived value that depends on `request` `Content-Type` header and `--sniff*` settings; str or None
        - `response_mime`: `response.body` `MIME` type, note the underscore, this is not a field of `response`, this is a derived value that depends on `response` `Content-Type` header and `--sniff*` settings; str or None
        - `filepath_parts`: `npath_parts` transformed into components usable as an exportable file name; i.e. `npath_parts` with an optional additional `"index"` appended, depending on `raw_url` and `response_mime`; extension will be stored separately in `filepath_ext`; e.g. for `HTML` documents `"https://www.example.org/"` -> `["index"]`, `"https://www.example.org/test.html"` -> `["test"]`, `"https://www.example.org/test"` -> `["test", "index"]`, `"https://www.example.org/test.json"` -> `["test.json", "index"]`, but if it has a `JSON` `MIME` type then `"https://www.example.org/test.json"` -> `["test"]` (and `filepath_ext` will be set to `".json"`); this is similar to what `wget -mpk` does, but a bit smarter; list[str]
        - `filepath_ext`: extension of the last component of `filepath_parts` for recognized `MIME` types, `".data"` otherwise; str
      - a compound expression built by piping (`|`) the above, for example:
        - `response.body|eb` (the default for `get` and `run`) will print raw `response.body` or an empty byte string, if there was no response;
        - `response.body|eb|scrub response defaults` will take the above value, `scrub` it using default content scrubbing settings which will censor out all actions and references to page requisites;
        - `response.complete` will print the value of `response.complete` or `None`, if there was no response;
        - `response.complete|false` will print `response.complete` or `False`;
        - `net_url|to_ascii|sha256|to_hex` will print a hexadecimal representation of the `sha256` hash of the URL that was actually sent over the network;
        - `net_url|to_ascii|sha256|take_prefix 2|to_hex` will print the first 2 bytes (4 characters) of the above;
        - `path_parts|take_prefix 3|pp_to_path` will print first 3 path components of the URL, minimally quoted to be used as a path;
        - `query_ne_parts|take_prefix 3|qsl_to_path|abbrev 128` will print first 3 non-empty query parameters of the URL, abbreviated to 128 characters or less, minimally quoted to be used as a path;

      Example URL mappings:
        - `raw_url`:
          - `https://example.org` -> `https://example.org`
          - `https://example.org/` -> `https://example.org/`
          - `https://example.org/index.html` -> `https://example.org/index.html`
          - `https://example.org/media` -> `https://example.org/media`
          - `https://example.org/media/` -> `https://example.org/media/`
          - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https://example.org/view?one=1&two=2&three=&three=3#fragment`
          - `https://königsgäßchen.example.org/index.html` -> `https://königsgäßchen.example.org/index.html`
          - `https://ジャジェメント.ですの.example.org/испытание/is/` -> `https://ジャジェメント.ですの.example.org/испытание/is/`
          - `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`
          - `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`
        - `net_url`:
          - `https://example.org`, `https://example.org/` -> `https://example.org/`
          - `https://example.org/index.html` -> `https://example.org/index.html`
          - `https://example.org/media` -> `https://example.org/media`
          - `https://example.org/media/` -> `https://example.org/media/`
          - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https://example.org/view?one=1&two=2&three=&three=3`
          - `https://königsgäßchen.example.org/index.html` -> `https://xn--knigsgchen-b4a3dun.example.org/index.html`
          - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`
        - `pretty_url`:
          - `https://example.org`, `https://example.org/` -> `https://example.org/`
          - `https://example.org/index.html` -> `https://example.org/index.html`
          - `https://example.org/media` -> `https://example.org/media`
          - `https://example.org/media/` -> `https://example.org/media/`
          - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https://example.org/view?one=1&two=2&three=&three=3#fragment`
          - `https://königsgäßchen.example.org/index.html` -> `https://königsgäßchen.example.org/index.html`
          - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https://ジャジェメント.ですの.example.org/испытание/is/`
        - `pretty_nurl`:
          - `https://example.org`, `https://example.org/` -> `https://example.org/`
          - `https://example.org/index.html` -> `https://example.org/index.html`
          - `https://example.org/media` -> `https://example.org/media`
          - `https://example.org/media/` -> `https://example.org/media/`
          - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https://example.org/view?one=1&two=2&three=3#fragment`
          - `https://königsgäßchen.example.org/index.html` -> `https://königsgäßchen.example.org/index.html`
          - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https://ジャジェメント.ですの.example.org/испытание/is/`

  - printing of `--expr` values:
    - `--not-separated`
    : print `--expr` values without separating them with anything, just concatenate them
    - `-l, --lf-separated`
    : print `--expr` values separated with `\n` (LF) newline characters; default
    - `-z, --zero-separated`
    : print `--expr` values separated with `\0` (NUL) bytes

  - default value of `--expr`:
    - `--raw-qbody`
    : set the default value of `--expr` to `request.body|eb`; i.e. produce the raw request body
    - `--raw-sbody, --no-remap`
    : set the default value of `--expr` to `response.body|eb`; i.e. produce the raw response body; default
    - `--remap-id`
    : set the default value of `--expr` to `response.body|eb|scrub response +all_refs`; i.e. `scrub` response body as follows: remap all URLs with an identity function (which, as a whole, is NOT an identity function, it will transform all relative URLs into absolute ones), censor out all dynamic content (e.g. `JavaScript`); results will NOT be self-contained
    - `--remap-void`
    : set the default value of `--expr` to `response.body|eb|scrub response -all_refs`; i.e. `scrub` response body as follows: remap all URLs into `javascript:void(0)` and empty `data:` URLs, censor out all dynamic content; results will be self-contained



  Spawn `COMMAND` with given static `ARG`uments and `NUM` additional arguments generated by evaluating given `EXPR`essions on given `PATH`s into temporary files.

  Algorithm:

  - Load `NUM` given `PATH`s (`--num-args` decides the point at which `argv` get split into `ARG`s and `PATH`s);
  - for each of `NUM` resulting reqres:
    - evaluate `EXPR` expressions;
    - write the results into a newly generated temporary file, terminating each value as specified;
  - spawn given `COMMAND` with given `ARG` arguments and `NUM` additional arguments that are paths of the files generated in the previous step,
  - wait for it to finish,
  - delete the temporary files,
  - exit with the return code of the spawned process.

  The end.

  Essentially, this is `{__prog__} get` into a temporary file for each given `PATH`, followed by spawning of `COMMAND`, followed by cleanup when it finishes.

  - positional arguments:
    - `COMMAND`
    : command to spawn
    - `ARG`
    : static arguments to give to the `COMMAND`
    - `PATH`
    : input `WRR` file paths to be mapped into new temporary files

  - options:
    - `-n NUM, --num-args NUM`
    : number of `PATH`s; default: `1`

  - `MIME` type sniffing; this controls the use of the [`mimesniff` algorithm](https://mimesniff.spec.whatwg.org/); for this sub-command higher values make the `scrub` function (which see) censor out more things when `-unknown`, `-styles`, or `-scripts` options are set; in particular, at the moment, with `--sniff-paranoid` and `-scripts` most plain text files will be censored out as potential `JavaScript`:
    - `--sniff-default`
    : run `mimesniff` when the spec says it should be run; i.e. trust `Content-Type` `HTTP` headers most of the time; default
    - `--sniff-force`
    : run `mimesniff` regardless of what `Content-Type`  and `X-Content-Type-Options` `HTTP` headers say; i.e. for each reqres, run `mimesniff` algorithm on the `Content-Type` `HTTP` header and the actual contents of `(request|response).body` (depending on the first argument of `scrub`) to determine what the body actually contains, then interpret the data as intersection of what `Content-Type` and `mimesniff` claim it to be; e.g. if `Content-Type` says `text/plain` but `mimesniff` says `text/plain or text/javascript`, interpret it as `text/plain`
    - `--sniff-paranoid`
    : do what `--sniff-force` does, but interpret the results in the most paranoid way possible; e.g. if `Content-Type` says `text/plain` but `mimesniff` says `text/plain or text/javascript`, interpret it as `text/plain or text/javascript`; which, for instance, will then make `scrub` with `-scripts` censor it out, since it can be interpreted as a script

  - expression evaluation:
    - `-e EXPR, --expr EXPR`
    : an expression to compute, same expression format and semantics as `hoardy-web get --expr` (which see); can be specified multiple times; the default depends on `--remap-*` options below

  - printing of `--expr` values:
    - `--not-separated`
    : print `--expr` values without separating them with anything, just concatenate them
    - `-l, --lf-separated`
    : print `--expr` values separated with `\n` (LF) newline characters; default
    - `-z, --zero-separated`
    : print `--expr` values separated with `\0` (NUL) bytes

  - default value of `--expr`:
    - `--raw-qbody`
    : set the default value of `--expr` to `request.body|eb`; i.e. produce the raw request body
    - `--raw-sbody, --no-remap`
    : set the default value of `--expr` to `response.body|eb`; i.e. produce the raw response body; default
    - `--remap-id`
    : set the default value of `--expr` to `response.body|eb|scrub response +all_refs`; i.e. `scrub` response body as follows: remap all URLs with an identity function (which, as a whole, is NOT an identity function, it will transform all relative URLs into absolute ones), censor out all dynamic content (e.g. `JavaScript`); results will NOT be self-contained
    - `--remap-void`
    : set the default value of `--expr` to `response.body|eb|scrub response -all_refs`; i.e. `scrub` response body as follows: remap all URLs into `javascript:void(0)` and empty `data:` URLs, censor out all dynamic content; results will be self-contained



  Stream lists of results produced by evaluating given `EXPR`essions on given inputs to stdout.

  Algorithm:

  - For each input `PATH`:
    - load it;
    - evaluate all `EXPR` expressions on the resulting reqres;
    - encode the resulting list into a requested `FORMAT`;
    - print it to stdout.

  The end.

  Esentially, this is a generalized `{__prog__} get`.

  - options:
    - `-q, --quiet`
    : don't print end-of-filtering warnings to stderr
    - `-u, --unabridged`
    : print all data in full
    - `--abridged`
    : shorten long strings for brevity, useful when you want to visually scan through batch data dumps; default
    - `--format FORMAT`
    : generate output in:
      - py: Pythonic Object Representation aka `repr`; default
      - cbor: Concise Binary Object Representation aka `CBOR` (RFC8949)
      - json: JavaScript Object Notation aka `JSON`; **binary data can't be represented, UNICODE replacement characters will be used**
      - raw: concatenate raw values; termination is controlled by `*-terminated` options

  - error handling:
    - `--errors {fail,skip,ignore}`
    : when an error occurs:
      - `fail`: report failure and stop the execution; default
      - `skip`: report failure but skip the reqres that produced it from the output and continue
      - `ignore`: `skip`, but don't report the failure

  - path ordering:
    - `--paths-given-order`
    : `argv` and `--stdin0` `PATH`s are processed in the order they are given; default
    - `--paths-sorted`
    : `argv` and `--stdin0` `PATH`s are processed in lexicographic order
    - `--paths-reversed`
    : `argv` and `--stdin0` `PATH`s are processed in reverse lexicographic order
    - `--walk-fs-order`
    : recursive file system walk is done in the order `readdir(2)` gives results
    - `--walk-sorted`
    : recursive file system walk is done in lexicographic order; default
    - `--walk-reversed`
    : recursive file system walk is done in reverse lexicographic order

  - input loading:
    - `--load-any`
    : for each given input `PATH`, decide which loader to use based on its file extension; default
    - `--load-wrr`
    : load all inputs using the single-`WRR` per-file loader
    - `--load-wrrb`
    : load all inputs using the `WRR` bundle loader, this will load separate `WRR` files as single-`WRR` bundles too
    - `--load-mitmproxy`
    : load inputs using the `mitmproxy` dump loader
    - `--stdin0`
    : read zero-terminated `PATH`s from stdin, these will be processed after `PATH`s specified as command-line arguments
    - `PATH`
    : inputs, can be a mix of files and directories (which will be traversed recursively)

  - `MIME` type sniffing; this controls the use of the [`mimesniff` algorithm](https://mimesniff.spec.whatwg.org/); for this sub-command higher values make the `scrub` function (which see) censor out more things when `-unknown`, `-styles`, or `-scripts` options are set; in particular, at the moment, with `--sniff-paranoid` and `-scripts` most plain text files will be censored out as potential `JavaScript`:
    - `--sniff-default`
    : run `mimesniff` when the spec says it should be run; i.e. trust `Content-Type` `HTTP` headers most of the time; default
    - `--sniff-force`
    : run `mimesniff` regardless of what `Content-Type`  and `X-Content-Type-Options` `HTTP` headers say; i.e. for each reqres, run `mimesniff` algorithm on the `Content-Type` `HTTP` header and the actual contents of `(request|response).body` (depending on the first argument of `scrub`) to determine what the body actually contains, then interpret the data as intersection of what `Content-Type` and `mimesniff` claim it to be; e.g. if `Content-Type` says `text/plain` but `mimesniff` says `text/plain or text/javascript`, interpret it as `text/plain`
    - `--sniff-paranoid`
    : do what `--sniff-force` does, but interpret the results in the most paranoid way possible; e.g. if `Content-Type` says `text/plain` but `mimesniff` says `text/plain or text/javascript`, interpret it as `text/plain or text/javascript`; which, for instance, will then make `scrub` with `-scripts` censor it out, since it can be interpreted as a script

  - expression evaluation:
    - `-e EXPR, --expr EXPR`
    : an expression to compute, same expression format and semantics as `hoardy-web get --expr` (which see); can be specified multiple times; the default depends on `--remap-*` options below

  - `--format=raw` `--expr` printing:
    - `--not-terminated`
    : print `--format=raw` `--expr` output values without terminating them with anything, just concatenate them
    - `-l, --lf-terminated`
    : print `--format=raw` `--expr` output values terminated with `\n` (LF) newline characters; default
    - `-z, --zero-terminated`
    : print `--format=raw` `--expr` output values terminated with `\0` (NUL) bytes

  - default value of `--expr`:
    - `--structure`
    : set the default value of `--expr` to `.`; i.e. dump the whole structure; default
    - `--raw-qbody`
    : set the default value of `--expr` to `request.body|eb`; i.e. produce the raw request body
    - `--raw-sbody, --no-remap`
    : set the default value of `--expr` to `response.body|eb`; i.e. produce the raw response body
    - `--remap-id`
    : set the default value of `--expr` to `response.body|eb|scrub response +all_refs`; i.e. `scrub` response body as follows: remap all URLs with an identity function (which, as a whole, is NOT an identity function, it will transform all relative URLs into absolute ones), censor out all dynamic content (e.g. `JavaScript`); results will NOT be self-contained
    - `--remap-void`
    : set the default value of `--expr` to `response.body|eb|scrub response -all_refs`; i.e. `scrub` response body as follows: remap all URLs into `javascript:void(0)` and empty `data:` URLs, censor out all dynamic content; results will be self-contained



  Print paths of inputs matching specified criteria.

  Algorithm:

  - For each input `PATH`:
    - load it;
    - check this reqres satisfies given filters and skip it if it does not,
    - print its path to stdout.

  The end.

  - options:
    - `-q, --quiet`
    : don't print end-of-filtering warnings to stderr

  - error handling:
    - `--errors {fail,skip,ignore}`
    : when an error occurs:
      - `fail`: report failure and stop the execution; default
      - `skip`: report failure but skip the reqres that produced it from the output and continue
      - `ignore`: `skip`, but don't report the failure

  - path ordering:
    - `--paths-given-order`
    : `argv` and `--stdin0` `PATH`s are processed in the order they are given; default
    - `--paths-sorted`
    : `argv` and `--stdin0` `PATH`s are processed in lexicographic order
    - `--paths-reversed`
    : `argv` and `--stdin0` `PATH`s are processed in reverse lexicographic order
    - `--walk-fs-order`
    : recursive file system walk is done in the order `readdir(2)` gives results
    - `--walk-sorted`
    : recursive file system walk is done in lexicographic order; default
    - `--walk-reversed`
    : recursive file system walk is done in reverse lexicographic order

  - input loading:
    - `--load-any`
    : for each given input `PATH`, decide which loader to use based on its file extension; default
    - `--load-wrr`
    : load all inputs using the single-`WRR` per-file loader
    - `--load-wrrb`
    : load all inputs using the `WRR` bundle loader, this will load separate `WRR` files as single-`WRR` bundles too
    - `--load-mitmproxy`
    : load inputs using the `mitmproxy` dump loader
    - `--stdin0`
    : read zero-terminated `PATH`s from stdin, these will be processed after `PATH`s specified as command-line arguments
    - `PATH`
    : inputs, can be a mix of files and directories (which will be traversed recursively)

  - `MIME` type sniffing; this controls the use of the [`mimesniff` algorithm](https://mimesniff.spec.whatwg.org/); for this sub-command higher values make the `scrub` function (which see) censor out more things when `-unknown`, `-styles`, or `-scripts` options are set; in particular, at the moment, with `--sniff-paranoid` and `-scripts` most plain text files will be censored out as potential `JavaScript`:
    - `--sniff-default`
    : run `mimesniff` when the spec says it should be run; i.e. trust `Content-Type` `HTTP` headers most of the time; default
    - `--sniff-force`
    : run `mimesniff` regardless of what `Content-Type`  and `X-Content-Type-Options` `HTTP` headers say; i.e. for each reqres, run `mimesniff` algorithm on the `Content-Type` `HTTP` header and the actual contents of `(request|response).body` (depending on the first argument of `scrub`) to determine what the body actually contains, then interpret the data as intersection of what `Content-Type` and `mimesniff` claim it to be; e.g. if `Content-Type` says `text/plain` but `mimesniff` says `text/plain or text/javascript`, interpret it as `text/plain`
    - `--sniff-paranoid`
    : do what `--sniff-force` does, but interpret the results in the most paranoid way possible; e.g. if `Content-Type` says `text/plain` but `mimesniff` says `text/plain or text/javascript`, interpret it as `text/plain or text/javascript`; which, for instance, will then make `scrub` with `-scripts` censor it out, since it can be interpreted as a script

  - found files printing:
    - `-l, --lf-terminated`
    : print absolute paths of matching `WRR` files terminated with `\n` (LF) newline characters; default
    - `-z, --zero-terminated`
    : print absolute paths of matching `WRR` files terminated with `\0` (NUL) bytes



  Programmatically copy/rename/move/hardlink/symlink given input files based on their metadata and/or contents.

  Algorithm:

  - For each input `PATH`:
    - load it;
    - check this reqres satisfies given filters and skip it if it does not,
    - copy/rename/move/hardlink/symlink each file to `OUTPUT_DESTINATION` with the new path derived from each reqres' metadata.

  The end.

  Operations that could lead to accidental data loss are not permitted.
  E.g. `hoardy-web organize --move` will not overwrite any files, which is why the default `--output` contains `d`.

  - options:
    - `--dry-run`
    : perform a trial run without actually performing any changes
    - `-q, --quiet`
    : don't log computed updates and don't print end-of-filtering warnings to stderr

  - caching, deferring, and batching:
    - `--seen-number INT`
    : track at most this many distinct generated `--output` values; default: `16384`;
      making this larger improves disk performance at the cost of increased memory consumption;
      setting it to zero will force force `hoardy-web` to constantly re-check existence of `--output` files and force `hoardy-web` to execute  all IO actions immediately, disregarding `--defer-number` setting
    - `--cache-number INT`
    : cache `stat(2)` information about this many files in memory; default: `8192`;
      making this larger improves performance at the cost of increased memory consumption;
      setting this to a too small number will likely force `hoardy-web` into repeatedly performing lots of `stat(2)` system calls on the same files;
      setting this to a value smaller than `--defer-number` will not improve memory consumption very much since deferred IO actions also cache information about their own files
    - `--defer-number INT`
    : defer at most this many IO actions; default: `1024`;
      making this larger improves performance at the cost of increased memory consumption;
      setting it to zero will force all IO actions to be applied immediately
    - `--batch-number INT`
    : queue at most this many deferred IO actions to be applied together in a batch; this queue will only be used if all other resource constraints are met; default: `128`
    - `--max-memory INT`
    : the caches, the deferred actions queue, and the batch queue, all taken together, must not take more than this much memory in MiB; default: `1024`;
      making this larger improves performance;
      the actual maximum whole-program memory consumption is `O(<size of the largest reqres> + <--seen-number> + <sum of lengths of the last --seen-number generated --output paths> + <--cache-number> + <--defer-number> + <--batch-number> + <--max-memory>)`
    - `--lazy`
    : sets all of the above options to positive infinity;
      most useful when doing `hoardy-web organize --symlink --latest --output flat` or similar, where the number of distinct generated `--output` values and the amount of other data `hoardy-web` needs to keep in memory is small, in which case it will force `hoardy-web` to compute the desired file system state first and then perform all disk writes in a single batch

  - error handling:
    - `--errors {fail,skip,ignore}`
    : when an error occurs:
      - `fail`: report failure and stop the execution; default
      - `skip`: report failure but skip the reqres that produced it from the output and continue
      - `ignore`: `skip`, but don't report the failure

  - path ordering:
    - `--paths-given-order`
    : `argv` and `--stdin0` `PATH`s are processed in the order they are given; default when `--no-overwrite`
    - `--paths-sorted`
    : `argv` and `--stdin0` `PATH`s are processed in lexicographic order
    - `--paths-reversed`
    : `argv` and `--stdin0` `PATH`s are processed in reverse lexicographic order; default when `--latest`
    - `--walk-fs-order`
    : recursive file system walk is done in the order `readdir(2)` gives results
    - `--walk-sorted`
    : recursive file system walk is done in lexicographic order; default when `--no-overwrite`
    - `--walk-reversed`
    : recursive file system walk is done in reverse lexicographic order; default when `--latest`

  - input loading:
    - `--load-any`
    : for each given input `PATH`, decide which loader to use based on its file extension; default
    - `--load-wrr`
    : load all inputs using the single-`WRR` per-file loader
    - `--load-wrrb`
    : load all inputs using the `WRR` bundle loader, this will load separate `WRR` files as single-`WRR` bundles too
    - `--load-mitmproxy`
    : load inputs using the `mitmproxy` dump loader
    - `--stdin0`
    : read zero-terminated `PATH`s from stdin, these will be processed after `PATH`s specified as command-line arguments
    - `PATH`
    : inputs, can be a mix of files and directories (which will be traversed recursively)

  - `MIME` type sniffing; this controls the use of the [`mimesniff` algorithm](https://mimesniff.spec.whatwg.org/); for this sub-command this influeences generated file names because `filepath_parts` and `filepath_ext` of `hoardy-web get --expr` (which see) depend on both the original file extension present in the URL and the detected `MIME` type of its content:
    - `--sniff-default`
    : run `mimesniff` when the spec says it should be run; i.e. trust `Content-Type` `HTTP` headers most of the time; default
    - `--sniff-force`
    : run `mimesniff` regardless of what `Content-Type`  and `X-Content-Type-Options` `HTTP` headers say; i.e. for each reqres, run `mimesniff` algorithm on the `Content-Type` `HTTP` header and the actual contents of `(request|response).body` (depending on the first argument of `scrub`) to determine what the body actually contains, then interpret the data as intersection of what `Content-Type` and `mimesniff` claim it to be; e.g. if `Content-Type` says `text/plain` but `mimesniff` says `text/plain or text/javascript`, interpret it as `text/plain`
    - `--sniff-paranoid`
    : do what `--sniff-force` does, but interpret the results in the most paranoid way possible; e.g. if `Content-Type` says `text/plain` but `mimesniff` says `text/plain or text/javascript`, interpret it as `text/plain or text/javascript`; which, for instance, will then make `scrub` with `-scripts` censor it out, since it can be interpreted as a script

  - action:
    - `--move`
    : move source files under `OUTPUT_DESTINATION`; default
    - `--copy`
    : copy source files to files under `OUTPUT_DESTINATION`
    - `--hardlink`
    : create hardlinks from source files to paths under `OUTPUT_DESTINATION`
    - `--symlink`
    : create symlinks from source files to paths under `OUTPUT_DESTINATION`

  - file outputs:
    - `-t OUTPUT_DESTINATION, --to OUTPUT_DESTINATION, --organize-to OUTPUT_DESTINATION`
    : destination directory; when unset each source `PATH` must be a directory which will be treated as its own `OUTPUT_DESTINATION`
    - `-o OUTPUT_FORMAT, --output OUTPUT_FORMAT`
    : format describing generated output paths, an alias name or "format:" followed by a custom pythonic  string:
      - available aliases and corresponding :
        - `default`     : `d/02d/02d/02d02d02d03d_s_s_s_s_localhosts_d`; the default
              - `https://example.org`, `https://example.org/` -> `1970/01/01/001640000_0_GET_8198_C200C_example.org_0`
              - `https://example.org/index.html` -> `1970/01/01/001640000_0_GET_f0dc_C200C_example.org_0`
              - `https://example.org/media` -> `1970/01/01/001640000_0_GET_086d_C200C_example.org_0`
              - `https://example.org/media/` -> `1970/01/01/001640000_0_GET_3fbb_C200C_example.org_0`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `1970/01/01/001640000_0_GET_5658_C200C_example.org_0`
              - `https://königsgäßchen.example.org/index.html` -> `1970/01/01/001640000_0_GET_4f11_C200C_königsgäßchen.example.org_0`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `1970/01/01/001640000_0_GET_c4ae_C200C_ジャジェメント.ですの.example.org_0`
        - `short`       : `d/02d/02d/d_s_d`
              - `https://example.org`, `https://example.org/`, `https://example.org/index.html`, `https://example.org/media`, `https://example.org/media/`, `https://example.org/view?one=1&two=2&three=&three=3#fragment`, `https://königsgäßchen.example.org/index.html`, `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `1970/01/01/1000000_0_0`
        - `surl`        : `s/s/sss`
              - `https://example.org`, `https://example.org/` -> `https/example.org/`
              - `https://example.org/index.html` -> `https/example.org/index.html`
              - `https://example.org/media`, `https://example.org/media/` -> `https/example.org/media`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/example.org/view?one=1&two=2&three=&three=3`
              - `https://königsgäßchen.example.org/index.html` -> `https/königsgäßchen.example.org/index.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/ジャジェメント.ですの.example.org/испытание/is`
        - `surl_msn`    : `s/s/sss__s_s_d`
              - `https://example.org`, `https://example.org/` -> `https/example.org/__GET_C200C_0`
              - `https://example.org/index.html` -> `https/example.org/index.html__GET_C200C_0`
              - `https://example.org/media`, `https://example.org/media/` -> `https/example.org/media__GET_C200C_0`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/example.org/view?one=1&two=2&three=&three=3__GET_C200C_0`
              - `https://königsgäßchen.example.org/index.html` -> `https/königsgäßchen.example.org/index.html__GET_C200C_0`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/ジャジェメント.ですの.example.org/испытание/is__GET_C200C_0`
        - `surl_mstn`   : `s/s/sss__s_s_d-02d-02d_02d02d02d03d_d`
              - `https://example.org`, `https://example.org/` -> `https/example.org/__GET_C200C_1970-01-01_001640000_0`
              - `https://example.org/index.html` -> `https/example.org/index.html__GET_C200C_1970-01-01_001640000_0`
              - `https://example.org/media`, `https://example.org/media/` -> `https/example.org/media__GET_C200C_1970-01-01_001640000_0`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/example.org/view?one=1&two=2&three=&three=3__GET_C200C_1970-01-01_001640000_0`
              - `https://königsgäßchen.example.org/index.html` -> `https/königsgäßchen.example.org/index.html__GET_C200C_1970-01-01_001640000_0`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/ジャジェメント.ですの.example.org/испытание/is__GET_C200C_1970-01-01_001640000_0`
        - `shupq`       : `s/localhosts/ssss`
              - `https://example.org`, `https://example.org/` -> `https/example.org/index.htm`
              - `https://example.org/index.html` -> `https/example.org/index.html`
              - `https://example.org/media`, `https://example.org/media/` -> `https/example.org/media/index.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/example.org/view/index?one=1&two=2&three=&three=3.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/königsgäßchen.example.org/index.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/ジャジェメント.ですの.example.org/испытание/is/index.htm`
        - `shupq_n`     : `s/localhosts/sss.ds`
              - `https://example.org`, `https://example.org/` -> `https/example.org/index.0.htm`
              - `https://example.org/index.html` -> `https/example.org/index.0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `https/example.org/media/index.0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/example.org/view/index?one=1&two=2&three=&three=3.0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/königsgäßchen.example.org/index.0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/ジャジェメント.ですの.example.org/испытание/is/index.0.htm`
        - `shupq_tn`    : `s/localhosts/sss.d-02d-02d_02d02d02d03d_ds`
              - `https://example.org`, `https://example.org/` -> `https/example.org/index.1970-01-01_001640000_0.htm`
              - `https://example.org/index.html` -> `https/example.org/index.1970-01-01_001640000_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `https/example.org/media/index.1970-01-01_001640000_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/example.org/view/index?one=1&two=2&three=&three=3.1970-01-01_001640000_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/königsgäßchen.example.org/index.1970-01-01_001640000_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/ジャジェメント.ですの.example.org/испытание/is/index.1970-01-01_001640000_0.htm`
        - `shupq_msn`   : `s/localhosts/sss.s_s_ds`
              - `https://example.org`, `https://example.org/` -> `https/example.org/index.GET_C200C_0.htm`
              - `https://example.org/index.html` -> `https/example.org/index.GET_C200C_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `https/example.org/media/index.GET_C200C_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/example.org/view/index?one=1&two=2&three=&three=3.GET_C200C_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/königsgäßchen.example.org/index.GET_C200C_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/ジャジェメント.ですの.example.org/испытание/is/index.GET_C200C_0.htm`
        - `shupq_mstn`  : `s/localhosts/sss.s_s_d-02d-02d_02d02d02d03d_ds`
              - `https://example.org`, `https://example.org/` -> `https/example.org/index.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/index.html` -> `https/example.org/index.GET_C200C_1970-01-01_001640000_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `https/example.org/media/index.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/example.org/view/index?one=1&two=2&three=&three=3.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/königsgäßchen.example.org/index.GET_C200C_1970-01-01_001640000_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/ジャジェメント.ですの.example.org/испытание/is/index.GET_C200C_1970-01-01_001640000_0.htm`
        - `shupnq`      : `s/localhosts/ssss`
              - `https://example.org`, `https://example.org/` -> `https/example.org/index.htm`
              - `https://example.org/index.html` -> `https/example.org/index.html`
              - `https://example.org/media`, `https://example.org/media/` -> `https/example.org/media/index.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/example.org/view/index?one=1&two=2&three=3.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/königsgäßchen.example.org/index.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/ジャジェメント.ですの.example.org/испытание/is/index.htm`
        - `shupnq_n`    : `s/localhosts/sss.ds`
              - `https://example.org`, `https://example.org/` -> `https/example.org/index.0.htm`
              - `https://example.org/index.html` -> `https/example.org/index.0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `https/example.org/media/index.0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/example.org/view/index?one=1&two=2&three=3.0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/königsgäßchen.example.org/index.0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/ジャジェメント.ですの.example.org/испытание/is/index.0.htm`
        - `shupnq_tn`   : `s/localhosts/sss.d-02d-02d_02d02d02d03d_ds`
              - `https://example.org`, `https://example.org/` -> `https/example.org/index.1970-01-01_001640000_0.htm`
              - `https://example.org/index.html` -> `https/example.org/index.1970-01-01_001640000_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `https/example.org/media/index.1970-01-01_001640000_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/example.org/view/index?one=1&two=2&three=3.1970-01-01_001640000_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/königsgäßchen.example.org/index.1970-01-01_001640000_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/ジャジェメント.ですの.example.org/испытание/is/index.1970-01-01_001640000_0.htm`
        - `shupnq_msn`  : `s/localhosts/sss.s_s_ds`
              - `https://example.org`, `https://example.org/` -> `https/example.org/index.GET_C200C_0.htm`
              - `https://example.org/index.html` -> `https/example.org/index.GET_C200C_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `https/example.org/media/index.GET_C200C_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/example.org/view/index?one=1&two=2&three=3.GET_C200C_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/königsgäßchen.example.org/index.GET_C200C_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/ジャジェメント.ですの.example.org/испытание/is/index.GET_C200C_0.htm`
        - `shupnq_mstn` : `s/localhosts/sss.s_s_d-02d-02d_02d02d02d03d_ds`
              - `https://example.org`, `https://example.org/` -> `https/example.org/index.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/index.html` -> `https/example.org/index.GET_C200C_1970-01-01_001640000_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `https/example.org/media/index.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/example.org/view/index?one=1&two=2&three=3.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/königsgäßchen.example.org/index.GET_C200C_1970-01-01_001640000_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/ジャジェメント.ですの.example.org/испытание/is/index.GET_C200C_1970-01-01_001640000_0.htm`
        - `shupnq_mhs`  : `s/localhosts/sss.s_s_ss`
              - `https://example.org`, `https://example.org/` -> `https/example.org/index.GET_8198_C200C.htm`
              - `https://example.org/index.html` -> `https/example.org/index.GET_f0dc_C200C.html`
              - `https://example.org/media` -> `https/example.org/media/index.GET_086d_C200C.htm`
              - `https://example.org/media/` -> `https/example.org/media/index.GET_3fbb_C200C.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/example.org/view/index?one=1&two=2&three=3.GET_5658_C200C.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/königsgäßchen.example.org/index.GET_4f11_C200C.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/ジャジェメント.ですの.example.org/испытание/is/index.GET_c4ae_C200C.htm`
        - `shupnq_mhsn` : `s/localhosts/sss.s_s_s_ds`
              - `https://example.org`, `https://example.org/` -> `https/example.org/index.GET_8198_C200C_0.htm`
              - `https://example.org/index.html` -> `https/example.org/index.GET_f0dc_C200C_0.html`
              - `https://example.org/media` -> `https/example.org/media/index.GET_086d_C200C_0.htm`
              - `https://example.org/media/` -> `https/example.org/media/index.GET_3fbb_C200C_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/example.org/view/index?one=1&two=2&three=3.GET_5658_C200C_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/königsgäßchen.example.org/index.GET_4f11_C200C_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/ジャジェメント.ですの.example.org/испытание/is/index.GET_c4ae_C200C_0.htm`
        - `shupnq_mhstn`: `s/localhosts/sss.s_s_s_d-02d-02d_02d02d02d03d_ds`
              - `https://example.org`, `https://example.org/` -> `https/example.org/index.GET_8198_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/index.html` -> `https/example.org/index.GET_f0dc_C200C_1970-01-01_001640000_0.html`
              - `https://example.org/media` -> `https/example.org/media/index.GET_086d_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/media/` -> `https/example.org/media/index.GET_3fbb_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/example.org/view/index?one=1&two=2&three=3.GET_5658_C200C_1970-01-01_001640000_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/königsgäßchen.example.org/index.GET_4f11_C200C_1970-01-01_001640000_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/ジャジェメント.ですの.example.org/испытание/is/index.GET_c4ae_C200C_1970-01-01_001640000_0.htm`
        - `srhupq`      : `s/s/ssss`
              - `https://example.org`, `https://example.org/` -> `https/org.example/index.htm`
              - `https://example.org/index.html` -> `https/org.example/index.html`
              - `https://example.org/media`, `https://example.org/media/` -> `https/org.example/media/index.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/org.example/view/index?one=1&two=2&three=&three=3.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/org.example.königsgäßchen/index.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/org.example.ですの.ジャジェメント/испытание/is/index.htm`
        - `srhupq_n`    : `s/s/sss.ds`
              - `https://example.org`, `https://example.org/` -> `https/org.example/index.0.htm`
              - `https://example.org/index.html` -> `https/org.example/index.0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `https/org.example/media/index.0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/org.example/view/index?one=1&two=2&three=&three=3.0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/org.example.königsgäßchen/index.0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/org.example.ですの.ジャジェメント/испытание/is/index.0.htm`
        - `srhupq_tn`   : `s/s/sss.d-02d-02d_02d02d02d03d_ds`
              - `https://example.org`, `https://example.org/` -> `https/org.example/index.1970-01-01_001640000_0.htm`
              - `https://example.org/index.html` -> `https/org.example/index.1970-01-01_001640000_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `https/org.example/media/index.1970-01-01_001640000_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/org.example/view/index?one=1&two=2&three=&three=3.1970-01-01_001640000_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/org.example.königsgäßchen/index.1970-01-01_001640000_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/org.example.ですの.ジャジェメント/испытание/is/index.1970-01-01_001640000_0.htm`
        - `srhupq_msn`  : `s/s/sss.s_s_ds`
              - `https://example.org`, `https://example.org/` -> `https/org.example/index.GET_C200C_0.htm`
              - `https://example.org/index.html` -> `https/org.example/index.GET_C200C_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `https/org.example/media/index.GET_C200C_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/org.example/view/index?one=1&two=2&three=&three=3.GET_C200C_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/org.example.königsgäßchen/index.GET_C200C_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/org.example.ですの.ジャジェメント/испытание/is/index.GET_C200C_0.htm`
        - `srhupq_mstn` : `s/s/sss.s_s_d-02d-02d_02d02d02d03d_ds`
              - `https://example.org`, `https://example.org/` -> `https/org.example/index.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/index.html` -> `https/org.example/index.GET_C200C_1970-01-01_001640000_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `https/org.example/media/index.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/org.example/view/index?one=1&two=2&three=&three=3.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/org.example.königsgäßchen/index.GET_C200C_1970-01-01_001640000_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/org.example.ですの.ジャジェメント/испытание/is/index.GET_C200C_1970-01-01_001640000_0.htm`
        - `srhupnq`     : `s/s/ssss`
              - `https://example.org`, `https://example.org/` -> `https/org.example/index.htm`
              - `https://example.org/index.html` -> `https/org.example/index.html`
              - `https://example.org/media`, `https://example.org/media/` -> `https/org.example/media/index.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/org.example/view/index?one=1&two=2&three=3.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/org.example.königsgäßchen/index.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/org.example.ですの.ジャジェメント/испытание/is/index.htm`
        - `srhupnq_n`   : `s/s/sss.ds`
              - `https://example.org`, `https://example.org/` -> `https/org.example/index.0.htm`
              - `https://example.org/index.html` -> `https/org.example/index.0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `https/org.example/media/index.0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/org.example/view/index?one=1&two=2&three=3.0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/org.example.königsgäßchen/index.0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/org.example.ですの.ジャジェメント/испытание/is/index.0.htm`
        - `srhupnq_tn`  : `s/s/sss.d-02d-02d_02d02d02d03d_ds`
              - `https://example.org`, `https://example.org/` -> `https/org.example/index.1970-01-01_001640000_0.htm`
              - `https://example.org/index.html` -> `https/org.example/index.1970-01-01_001640000_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `https/org.example/media/index.1970-01-01_001640000_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/org.example/view/index?one=1&two=2&three=3.1970-01-01_001640000_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/org.example.königsgäßchen/index.1970-01-01_001640000_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/org.example.ですの.ジャジェメント/испытание/is/index.1970-01-01_001640000_0.htm`
        - `srhupnq_msn` : `s/s/sss.s_s_ds`
              - `https://example.org`, `https://example.org/` -> `https/org.example/index.GET_C200C_0.htm`
              - `https://example.org/index.html` -> `https/org.example/index.GET_C200C_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `https/org.example/media/index.GET_C200C_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/org.example/view/index?one=1&two=2&three=3.GET_C200C_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/org.example.königsgäßchen/index.GET_C200C_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/org.example.ですの.ジャジェメント/испытание/is/index.GET_C200C_0.htm`
        - `srhupnq_mstn`: `s/s/sss.s_s_d-02d-02d_02d02d02d03d_ds`
              - `https://example.org`, `https://example.org/` -> `https/org.example/index.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/index.html` -> `https/org.example/index.GET_C200C_1970-01-01_001640000_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `https/org.example/media/index.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/org.example/view/index?one=1&two=2&three=3.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/org.example.königsgäßchen/index.GET_C200C_1970-01-01_001640000_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/org.example.ですの.ジャジェメント/испытание/is/index.GET_C200C_1970-01-01_001640000_0.htm`
        - `srhupnq_mhs` : `s/s/sss.s_s_ss`
              - `https://example.org`, `https://example.org/` -> `https/org.example/index.GET_8198_C200C.htm`
              - `https://example.org/index.html` -> `https/org.example/index.GET_f0dc_C200C.html`
              - `https://example.org/media` -> `https/org.example/media/index.GET_086d_C200C.htm`
              - `https://example.org/media/` -> `https/org.example/media/index.GET_3fbb_C200C.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/org.example/view/index?one=1&two=2&three=3.GET_5658_C200C.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/org.example.königsgäßchen/index.GET_4f11_C200C.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/org.example.ですの.ジャジェメント/испытание/is/index.GET_c4ae_C200C.htm`
        - `srhupnq_mhsn`: `s/s/sss.s_s_s_ds`
              - `https://example.org`, `https://example.org/` -> `https/org.example/index.GET_8198_C200C_0.htm`
              - `https://example.org/index.html` -> `https/org.example/index.GET_f0dc_C200C_0.html`
              - `https://example.org/media` -> `https/org.example/media/index.GET_086d_C200C_0.htm`
              - `https://example.org/media/` -> `https/org.example/media/index.GET_3fbb_C200C_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/org.example/view/index?one=1&two=2&three=3.GET_5658_C200C_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/org.example.königsgäßchen/index.GET_4f11_C200C_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/org.example.ですの.ジャジェメント/испытание/is/index.GET_c4ae_C200C_0.htm`
        - `srhupnq_mhstn`: `s/s/sss.s_s_s_d-02d-02d_02d02d02d03d_ds`
              - `https://example.org`, `https://example.org/` -> `https/org.example/index.GET_8198_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/index.html` -> `https/org.example/index.GET_f0dc_C200C_1970-01-01_001640000_0.html`
              - `https://example.org/media` -> `https/org.example/media/index.GET_086d_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/media/` -> `https/org.example/media/index.GET_3fbb_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `https/org.example/view/index?one=1&two=2&three=3.GET_5658_C200C_1970-01-01_001640000_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `https/org.example.königsgäßchen/index.GET_4f11_C200C_1970-01-01_001640000_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `https/org.example.ですの.ジャジェメント/испытание/is/index.GET_c4ae_C200C_1970-01-01_001640000_0.htm`
        - `url`         : `s/sss`
              - `https://example.org`, `https://example.org/` -> `example.org/`
              - `https://example.org/index.html` -> `example.org/index.html`
              - `https://example.org/media`, `https://example.org/media/` -> `example.org/media`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view?one=1&two=2&three=&three=3`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание/is`
        - `url_msn`     : `s/sss__s_s_d`
              - `https://example.org`, `https://example.org/` -> `example.org/__GET_C200C_0`
              - `https://example.org/index.html` -> `example.org/index.html__GET_C200C_0`
              - `https://example.org/media`, `https://example.org/media/` -> `example.org/media__GET_C200C_0`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view?one=1&two=2&three=&three=3__GET_C200C_0`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.html__GET_C200C_0`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание/is__GET_C200C_0`
        - `url_mstn`    : `s/sss__s_s_d-02d-02d_02d02d02d03d_d`
              - `https://example.org`, `https://example.org/` -> `example.org/__GET_C200C_1970-01-01_001640000_0`
              - `https://example.org/index.html` -> `example.org/index.html__GET_C200C_1970-01-01_001640000_0`
              - `https://example.org/media`, `https://example.org/media/` -> `example.org/media__GET_C200C_1970-01-01_001640000_0`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view?one=1&two=2&three=&three=3__GET_C200C_1970-01-01_001640000_0`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.html__GET_C200C_1970-01-01_001640000_0`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание/is__GET_C200C_1970-01-01_001640000_0`
        - `hupq`        : `localhosts/ssss`
              - `https://example.org`, `https://example.org/` -> `example.org/index.htm`
              - `https://example.org/index.html` -> `example.org/index.html`
              - `https://example.org/media`, `https://example.org/media/` -> `example.org/media/index.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view/index?one=1&two=2&three=&three=3.htm`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание/is/index.htm`
        - `hupq_n`      : `localhosts/sss.ds`
              - `https://example.org`, `https://example.org/` -> `example.org/index.0.htm`
              - `https://example.org/index.html` -> `example.org/index.0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `example.org/media/index.0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view/index?one=1&two=2&three=&three=3.0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание/is/index.0.htm`
        - `hupq_tn`     : `localhosts/sss.d-02d-02d_02d02d02d03d_ds`
              - `https://example.org`, `https://example.org/` -> `example.org/index.1970-01-01_001640000_0.htm`
              - `https://example.org/index.html` -> `example.org/index.1970-01-01_001640000_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `example.org/media/index.1970-01-01_001640000_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view/index?one=1&two=2&three=&three=3.1970-01-01_001640000_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.1970-01-01_001640000_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание/is/index.1970-01-01_001640000_0.htm`
        - `hupq_msn`    : `localhosts/sss.s_s_ds`
              - `https://example.org`, `https://example.org/` -> `example.org/index.GET_C200C_0.htm`
              - `https://example.org/index.html` -> `example.org/index.GET_C200C_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `example.org/media/index.GET_C200C_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view/index?one=1&two=2&three=&three=3.GET_C200C_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.GET_C200C_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание/is/index.GET_C200C_0.htm`
        - `hupq_mstn`   : `localhosts/sss.s_s_d-02d-02d_02d02d02d03d_ds`
              - `https://example.org`, `https://example.org/` -> `example.org/index.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/index.html` -> `example.org/index.GET_C200C_1970-01-01_001640000_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `example.org/media/index.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view/index?one=1&two=2&three=&three=3.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.GET_C200C_1970-01-01_001640000_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание/is/index.GET_C200C_1970-01-01_001640000_0.htm`
        - `hupnq`       : `localhosts/ssss`
              - `https://example.org`, `https://example.org/` -> `example.org/index.htm`
              - `https://example.org/index.html` -> `example.org/index.html`
              - `https://example.org/media`, `https://example.org/media/` -> `example.org/media/index.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view/index?one=1&two=2&three=3.htm`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание/is/index.htm`
        - `hupnq_n`     : `localhosts/sss.ds`
              - `https://example.org`, `https://example.org/` -> `example.org/index.0.htm`
              - `https://example.org/index.html` -> `example.org/index.0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `example.org/media/index.0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view/index?one=1&two=2&three=3.0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание/is/index.0.htm`
        - `hupnq_tn`    : `localhosts/sss.d-02d-02d_02d02d02d03d_ds`
              - `https://example.org`, `https://example.org/` -> `example.org/index.1970-01-01_001640000_0.htm`
              - `https://example.org/index.html` -> `example.org/index.1970-01-01_001640000_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `example.org/media/index.1970-01-01_001640000_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view/index?one=1&two=2&three=3.1970-01-01_001640000_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.1970-01-01_001640000_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание/is/index.1970-01-01_001640000_0.htm`
        - `hupnq_msn`   : `localhosts/sss.s_s_ds`
              - `https://example.org`, `https://example.org/` -> `example.org/index.GET_C200C_0.htm`
              - `https://example.org/index.html` -> `example.org/index.GET_C200C_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `example.org/media/index.GET_C200C_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view/index?one=1&two=2&three=3.GET_C200C_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.GET_C200C_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание/is/index.GET_C200C_0.htm`
        - `hupnq_mstn`  : `localhosts/sss.s_s_d-02d-02d_02d02d02d03d_ds`
              - `https://example.org`, `https://example.org/` -> `example.org/index.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/index.html` -> `example.org/index.GET_C200C_1970-01-01_001640000_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `example.org/media/index.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view/index?one=1&two=2&three=3.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.GET_C200C_1970-01-01_001640000_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание/is/index.GET_C200C_1970-01-01_001640000_0.htm`
        - `hupnq_mhs`   : `localhosts/sss.s_s_ss`
              - `https://example.org`, `https://example.org/` -> `example.org/index.GET_8198_C200C.htm`
              - `https://example.org/index.html` -> `example.org/index.GET_f0dc_C200C.html`
              - `https://example.org/media` -> `example.org/media/index.GET_086d_C200C.htm`
              - `https://example.org/media/` -> `example.org/media/index.GET_3fbb_C200C.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view/index?one=1&two=2&three=3.GET_5658_C200C.htm`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.GET_4f11_C200C.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание/is/index.GET_c4ae_C200C.htm`
        - `hupnq_mhsn`  : `localhosts/sss.s_s_s_ds`
              - `https://example.org`, `https://example.org/` -> `example.org/index.GET_8198_C200C_0.htm`
              - `https://example.org/index.html` -> `example.org/index.GET_f0dc_C200C_0.html`
              - `https://example.org/media` -> `example.org/media/index.GET_086d_C200C_0.htm`
              - `https://example.org/media/` -> `example.org/media/index.GET_3fbb_C200C_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view/index?one=1&two=2&three=3.GET_5658_C200C_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.GET_4f11_C200C_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание/is/index.GET_c4ae_C200C_0.htm`
        - `hupnq_mhstn` : `localhosts/sss.s_s_s_d-02d-02d_02d02d02d03d_ds`
              - `https://example.org`, `https://example.org/` -> `example.org/index.GET_8198_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/index.html` -> `example.org/index.GET_f0dc_C200C_1970-01-01_001640000_0.html`
              - `https://example.org/media` -> `example.org/media/index.GET_086d_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/media/` -> `example.org/media/index.GET_3fbb_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view/index?one=1&two=2&three=3.GET_5658_C200C_1970-01-01_001640000_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.GET_4f11_C200C_1970-01-01_001640000_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание/is/index.GET_c4ae_C200C_1970-01-01_001640000_0.htm`
        - `rhupq`       : `s/ssss`
              - `https://example.org`, `https://example.org/` -> `org.example/index.htm`
              - `https://example.org/index.html` -> `org.example/index.html`
              - `https://example.org/media`, `https://example.org/media/` -> `org.example/media/index.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `org.example/view/index?one=1&two=2&three=&three=3.htm`
              - `https://königsgäßchen.example.org/index.html` -> `org.example.königsgäßchen/index.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `org.example.ですの.ジャジェメント/испытание/is/index.htm`
        - `rhupq_n`     : `s/sss.ds`
              - `https://example.org`, `https://example.org/` -> `org.example/index.0.htm`
              - `https://example.org/index.html` -> `org.example/index.0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `org.example/media/index.0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `org.example/view/index?one=1&two=2&three=&three=3.0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `org.example.königsgäßchen/index.0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `org.example.ですの.ジャジェメント/испытание/is/index.0.htm`
        - `rhupq_tn`    : `s/sss.d-02d-02d_02d02d02d03d_ds`
              - `https://example.org`, `https://example.org/` -> `org.example/index.1970-01-01_001640000_0.htm`
              - `https://example.org/index.html` -> `org.example/index.1970-01-01_001640000_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `org.example/media/index.1970-01-01_001640000_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `org.example/view/index?one=1&two=2&three=&three=3.1970-01-01_001640000_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `org.example.königsgäßchen/index.1970-01-01_001640000_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `org.example.ですの.ジャジェメント/испытание/is/index.1970-01-01_001640000_0.htm`
        - `rhupq_msn`   : `s/sss.s_s_ds`
              - `https://example.org`, `https://example.org/` -> `org.example/index.GET_C200C_0.htm`
              - `https://example.org/index.html` -> `org.example/index.GET_C200C_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `org.example/media/index.GET_C200C_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `org.example/view/index?one=1&two=2&three=&three=3.GET_C200C_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `org.example.königsgäßchen/index.GET_C200C_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `org.example.ですの.ジャジェメント/испытание/is/index.GET_C200C_0.htm`
        - `rhupq_mstn`  : `s/sss.s_s_d-02d-02d_02d02d02d03d_ds`
              - `https://example.org`, `https://example.org/` -> `org.example/index.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/index.html` -> `org.example/index.GET_C200C_1970-01-01_001640000_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `org.example/media/index.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `org.example/view/index?one=1&two=2&three=&three=3.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `org.example.königsgäßchen/index.GET_C200C_1970-01-01_001640000_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `org.example.ですの.ジャジェメント/испытание/is/index.GET_C200C_1970-01-01_001640000_0.htm`
        - `rhupnq`      : `s/ssss`
              - `https://example.org`, `https://example.org/` -> `org.example/index.htm`
              - `https://example.org/index.html` -> `org.example/index.html`
              - `https://example.org/media`, `https://example.org/media/` -> `org.example/media/index.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `org.example/view/index?one=1&two=2&three=3.htm`
              - `https://königsgäßchen.example.org/index.html` -> `org.example.königsgäßchen/index.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `org.example.ですの.ジャジェメント/испытание/is/index.htm`
        - `rhupnq_n`    : `s/sss.ds`
              - `https://example.org`, `https://example.org/` -> `org.example/index.0.htm`
              - `https://example.org/index.html` -> `org.example/index.0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `org.example/media/index.0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `org.example/view/index?one=1&two=2&three=3.0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `org.example.königsgäßchen/index.0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `org.example.ですの.ジャジェメント/испытание/is/index.0.htm`
        - `rhupnq_tn`   : `s/sss.d-02d-02d_02d02d02d03d_ds`
              - `https://example.org`, `https://example.org/` -> `org.example/index.1970-01-01_001640000_0.htm`
              - `https://example.org/index.html` -> `org.example/index.1970-01-01_001640000_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `org.example/media/index.1970-01-01_001640000_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `org.example/view/index?one=1&two=2&three=3.1970-01-01_001640000_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `org.example.königsgäßchen/index.1970-01-01_001640000_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `org.example.ですの.ジャジェメント/испытание/is/index.1970-01-01_001640000_0.htm`
        - `rhupnq_msn`  : `s/sss.s_s_ds`
              - `https://example.org`, `https://example.org/` -> `org.example/index.GET_C200C_0.htm`
              - `https://example.org/index.html` -> `org.example/index.GET_C200C_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `org.example/media/index.GET_C200C_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `org.example/view/index?one=1&two=2&three=3.GET_C200C_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `org.example.königsgäßchen/index.GET_C200C_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `org.example.ですの.ジャジェメント/испытание/is/index.GET_C200C_0.htm`
        - `rhupnq_mstn` : `s/sss.s_s_d-02d-02d_02d02d02d03d_ds`
              - `https://example.org`, `https://example.org/` -> `org.example/index.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/index.html` -> `org.example/index.GET_C200C_1970-01-01_001640000_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `org.example/media/index.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `org.example/view/index?one=1&two=2&three=3.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `org.example.königsgäßchen/index.GET_C200C_1970-01-01_001640000_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `org.example.ですの.ジャジェメント/испытание/is/index.GET_C200C_1970-01-01_001640000_0.htm`
        - `rhupnq_mhs`  : `s/sss.s_s_ss`
              - `https://example.org`, `https://example.org/` -> `org.example/index.GET_8198_C200C.htm`
              - `https://example.org/index.html` -> `org.example/index.GET_f0dc_C200C.html`
              - `https://example.org/media` -> `org.example/media/index.GET_086d_C200C.htm`
              - `https://example.org/media/` -> `org.example/media/index.GET_3fbb_C200C.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `org.example/view/index?one=1&two=2&three=3.GET_5658_C200C.htm`
              - `https://königsgäßchen.example.org/index.html` -> `org.example.königsgäßchen/index.GET_4f11_C200C.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `org.example.ですの.ジャジェメント/испытание/is/index.GET_c4ae_C200C.htm`
        - `rhupnq_mhsn` : `s/sss.s_s_s_ds`
              - `https://example.org`, `https://example.org/` -> `org.example/index.GET_8198_C200C_0.htm`
              - `https://example.org/index.html` -> `org.example/index.GET_f0dc_C200C_0.html`
              - `https://example.org/media` -> `org.example/media/index.GET_086d_C200C_0.htm`
              - `https://example.org/media/` -> `org.example/media/index.GET_3fbb_C200C_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `org.example/view/index?one=1&two=2&three=3.GET_5658_C200C_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `org.example.königsgäßchen/index.GET_4f11_C200C_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `org.example.ですの.ジャジェメント/испытание/is/index.GET_c4ae_C200C_0.htm`
        - `rhupnq_mhstn`: `s/sss.s_s_s_d-02d-02d_02d02d02d03d_ds`
              - `https://example.org`, `https://example.org/` -> `org.example/index.GET_8198_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/index.html` -> `org.example/index.GET_f0dc_C200C_1970-01-01_001640000_0.html`
              - `https://example.org/media` -> `org.example/media/index.GET_086d_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/media/` -> `org.example/media/index.GET_3fbb_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `org.example/view/index?one=1&two=2&three=3.GET_5658_C200C_1970-01-01_001640000_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `org.example.königsgäßchen/index.GET_4f11_C200C_1970-01-01_001640000_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `org.example.ですの.ジャジェメント/испытание/is/index.GET_c4ae_C200C_1970-01-01_001640000_0.htm`
        - `flat`        : `localhosts/ssss`
              - `https://example.org`, `https://example.org/` -> `example.org/index.htm`
              - `https://example.org/index.html` -> `example.org/index.html`
              - `https://example.org/media`, `https://example.org/media/` -> `example.org/media__index.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view__index?one=1&two=2&three=3.htm`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание__is__index.htm`
        - `flat_n`      : `localhosts/sss.ds`
              - `https://example.org`, `https://example.org/` -> `example.org/index.0.htm`
              - `https://example.org/index.html` -> `example.org/index.0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `example.org/media__index.0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view__index?one=1&two=2&three=3.0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание__is__index.0.htm`
        - `flat_tn`     : `localhosts/sss.d-02d-02d_02d02d02d03d_ds`
              - `https://example.org`, `https://example.org/` -> `example.org/index.1970-01-01_001640000_0.htm`
              - `https://example.org/index.html` -> `example.org/index.1970-01-01_001640000_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `example.org/media__index.1970-01-01_001640000_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view__index?one=1&two=2&three=3.1970-01-01_001640000_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.1970-01-01_001640000_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание__is__index.1970-01-01_001640000_0.htm`
        - `flat_ms`     : `localhosts/sss.s_ss`
              - `https://example.org`, `https://example.org/` -> `example.org/index.GET_C200C.htm`
              - `https://example.org/index.html` -> `example.org/index.GET_C200C.html`
              - `https://example.org/media`, `https://example.org/media/` -> `example.org/media__index.GET_C200C.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view__index?one=1&two=2&three=3.GET_C200C.htm`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.GET_C200C.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание__is__index.GET_C200C.htm`
        - `flat_msn`    : `localhosts/sss.s_s_ds`
              - `https://example.org`, `https://example.org/` -> `example.org/index.GET_C200C_0.htm`
              - `https://example.org/index.html` -> `example.org/index.GET_C200C_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `example.org/media__index.GET_C200C_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view__index?one=1&two=2&three=3.GET_C200C_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.GET_C200C_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание__is__index.GET_C200C_0.htm`
        - `flat_mstn`   : `localhosts/sss.s_s_d-02d-02d_02d02d02d03d_ds`
              - `https://example.org`, `https://example.org/` -> `example.org/index.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/index.html` -> `example.org/index.GET_C200C_1970-01-01_001640000_0.html`
              - `https://example.org/media`, `https://example.org/media/` -> `example.org/media__index.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view__index?one=1&two=2&three=3.GET_C200C_1970-01-01_001640000_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.GET_C200C_1970-01-01_001640000_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание__is__index.GET_C200C_1970-01-01_001640000_0.htm`
        - `flat_mhs`    : `localhosts/sss.s_s_ss`
              - `https://example.org`, `https://example.org/` -> `example.org/index.GET_8198_C200C.htm`
              - `https://example.org/index.html` -> `example.org/index.GET_f0dc_C200C.html`
              - `https://example.org/media` -> `example.org/media__index.GET_086d_C200C.htm`
              - `https://example.org/media/` -> `example.org/media__index.GET_3fbb_C200C.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view__index?one=1&two=2&three=3.GET_5658_C200C.htm`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.GET_4f11_C200C.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание__is__index.GET_c4ae_C200C.htm`
        - `flat_mhsn`   : `localhosts/sss.s_s_s_ds`
              - `https://example.org`, `https://example.org/` -> `example.org/index.GET_8198_C200C_0.htm`
              - `https://example.org/index.html` -> `example.org/index.GET_f0dc_C200C_0.html`
              - `https://example.org/media` -> `example.org/media__index.GET_086d_C200C_0.htm`
              - `https://example.org/media/` -> `example.org/media__index.GET_3fbb_C200C_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view__index?one=1&two=2&three=3.GET_5658_C200C_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.GET_4f11_C200C_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание__is__index.GET_c4ae_C200C_0.htm`
        - `flat_mhstn`  : `localhosts/sss.s_s_s_d-02d-02d_02d02d02d03d_ds`
              - `https://example.org`, `https://example.org/` -> `example.org/index.GET_8198_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/index.html` -> `example.org/index.GET_f0dc_C200C_1970-01-01_001640000_0.html`
              - `https://example.org/media` -> `example.org/media__index.GET_086d_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/media/` -> `example.org/media__index.GET_3fbb_C200C_1970-01-01_001640000_0.htm`
              - `https://example.org/view?one=1&two=2&three=&three=3#fragment` -> `example.org/view__index?one=1&two=2&three=3.GET_5658_C200C_1970-01-01_001640000_0.htm`
              - `https://königsgäßchen.example.org/index.html` -> `königsgäßchen.example.org/index.GET_4f11_C200C_1970-01-01_001640000_0.html`
              - `https://ジャジェメント.ですの.example.org/испытание/is/`, `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` -> `ジャジェメント.ですの.example.org/испытание__is__index.GET_c4ae_C200C_1970-01-01_001640000_0.htm`
      - available substitutions:
        - all expressions of `hoardy-web get --expr` (which see);
        - `num`: number of times the resulting output path was encountered before; adding this parameter to your `--output` format will ensure all generated file names will be unique

  - new `--output`s printing:
    - `--no-print`
    : don't print anything; default
    - `-l, --lf-terminated`
    : print absolute paths of newly produced or replaced files terminated with `\n` (LF) newline characters
    - `-z, --zero-terminated`
    : print absolute paths of newly produced or replaced files terminated with `\0` (NUL) bytes

  - updates to `--output`s:
    - `--no-overwrite`
    : disallow overwrites and replacements of any existing files under `OUTPUT_DESTINATION`, i.e. only ever create new files under `OUTPUT_DESTINATION`, producing errors instead of attempting any other updates; default;
      `--output` targets that are broken symlinks will be considered to be non-existent and will be replaced;
      when the operation's source is binary-eqivalent to the `--output` target, the operation will be permitted, but the disk write will be reduced to a noop, i.e. the results will be deduplicated;
      the `dirname` of a source file and the `--to` target directories can be the same, in that case the source file will be renamed to use new `--output` name, though renames that attempt to swap files will still fail
    - `--latest`
    : replace files under `OUTPUT_DESTINATION` with their latest version;
      this is only allowed in combination with `--symlink` at the moment;
      for each source `PATH` file, the destination `--output` file will be replaced with a symlink to the source if and only if `stime_ms` of the source reqres is newer than `stime_ms` of the reqres stored at the destination file



  Use specified parser to parse data in each `INPUT` `PATH` into (a sequence of) reqres and then generate and place their `WRR` dumps into separate `WRR` files under `OUTPUT_DESTINATION` with paths derived from their metadata.
  In short, this is `hoardy-web organize --copy` for `INPUT` files that use different files formats.

  - file formats:
    - `{wrrb,bundle,mitmproxy,mitmdump}`
      - `wrrb (bundle)`
      : convert `WRR` bundles into separate `WRR` files
      - `mitmproxy (mitmdump)`
      : convert `mitmproxy` stream dumps (files produced by `mitmdump`) into `WRR` files



  Parse each `INPUT` `PATH` as a `WRR` bundle (an optionally compressed sequence of `WRR` dumps) and then generate and place their `WRR` dumps into separate `WRR` files under `OUTPUT_DESTINATION` with paths derived from their metadata.

  - options:
    - `--dry-run`
    : perform a trial run without actually performing any changes
    - `-q, --quiet`
    : don't log computed updates and don't print end-of-filtering warnings to stderr

  - caching, deferring, and batching:
    - `--seen-number INT`
    : track at most this many distinct generated `--output` values; default: `16384`;
      making this larger improves disk performance at the cost of increased memory consumption;
      setting it to zero will force force `hoardy-web` to constantly re-check existence of `--output` files and force `hoardy-web` to execute  all IO actions immediately, disregarding `--defer-number` setting
    - `--cache-number INT`
    : cache `stat(2)` information about this many files in memory; default: `8192`;
      making this larger improves performance at the cost of increased memory consumption;
      setting this to a too small number will likely force `hoardy-web` into repeatedly performing lots of `stat(2)` system calls on the same files;
      setting this to a value smaller than `--defer-number` will not improve memory consumption very much since deferred IO actions also cache information about their own files
    - `--defer-number INT`
    : defer at most this many IO actions; default: `0`;
      making this larger improves performance at the cost of increased memory consumption;
      setting it to zero will force all IO actions to be applied immediately
    - `--batch-number INT`
    : queue at most this many deferred IO actions to be applied together in a batch; this queue will only be used if all other resource constraints are met; default: `1024`
    - `--max-memory INT`
    : the caches, the deferred actions queue, and the batch queue, all taken together, must not take more than this much memory in MiB; default: `1024`;
      making this larger improves performance;
      the actual maximum whole-program memory consumption is `O(<size of the largest reqres> + <--seen-number> + <sum of lengths of the last --seen-number generated --output paths> + <--cache-number> + <--defer-number> + <--batch-number> + <--max-memory>)`
    - `--lazy`
    : sets all of the above options to positive infinity;
      most useful when doing `hoardy-web organize --symlink --latest --output flat` or similar, where the number of distinct generated `--output` values and the amount of other data `hoardy-web` needs to keep in memory is small, in which case it will force `hoardy-web` to compute the desired file system state first and then perform all disk writes in a single batch

  - error handling:
    - `--errors {fail,skip,ignore}`
    : when an error occurs:
      - `fail`: report failure and stop the execution; default
      - `skip`: report failure but skip the reqres that produced it from the output and continue
      - `ignore`: `skip`, but don't report the failure

  - path ordering:
    - `--paths-given-order`
    : `argv` and `--stdin0` `PATH`s are processed in the order they are given; default
    - `--paths-sorted`
    : `argv` and `--stdin0` `PATH`s are processed in lexicographic order
    - `--paths-reversed`
    : `argv` and `--stdin0` `PATH`s are processed in reverse lexicographic order
    - `--walk-fs-order`
    : recursive file system walk is done in the order `readdir(2)` gives results
    - `--walk-sorted`
    : recursive file system walk is done in lexicographic order; default
    - `--walk-reversed`
    : recursive file system walk is done in reverse lexicographic order

  - input loading:
    - `--load-any`
    : for each given input `PATH`, decide which loader to use based on its file extension; default
    - `--load-wrr`
    : load all inputs using the single-`WRR` per-file loader
    - `--load-wrrb`
    : load all inputs using the `WRR` bundle loader, this will load separate `WRR` files as single-`WRR` bundles too
    - `--load-mitmproxy`
    : load inputs using the `mitmproxy` dump loader
    - `--stdin0`
    : read zero-terminated `PATH`s from stdin, these will be processed after `PATH`s specified as command-line arguments
    - `PATH`
    : inputs, can be a mix of files and directories (which will be traversed recursively)

  - `MIME` type sniffing; this controls the use of the [`mimesniff` algorithm](https://mimesniff.spec.whatwg.org/); for this sub-command this influeences generated file names because `filepath_parts` and `filepath_ext` of `hoardy-web get --expr` (which see) depend on both the original file extension present in the URL and the detected `MIME` type of its content:
    - `--sniff-default`
    : run `mimesniff` when the spec says it should be run; i.e. trust `Content-Type` `HTTP` headers most of the time; default
    - `--sniff-force`
    : run `mimesniff` regardless of what `Content-Type`  and `X-Content-Type-Options` `HTTP` headers say; i.e. for each reqres, run `mimesniff` algorithm on the `Content-Type` `HTTP` header and the actual contents of `(request|response).body` (depending on the first argument of `scrub`) to determine what the body actually contains, then interpret the data as intersection of what `Content-Type` and `mimesniff` claim it to be; e.g. if `Content-Type` says `text/plain` but `mimesniff` says `text/plain or text/javascript`, interpret it as `text/plain`
    - `--sniff-paranoid`
    : do what `--sniff-force` does, but interpret the results in the most paranoid way possible; e.g. if `Content-Type` says `text/plain` but `mimesniff` says `text/plain or text/javascript`, interpret it as `text/plain or text/javascript`; which, for instance, will then make `scrub` with `-scripts` censor it out, since it can be interpreted as a script

  - file outputs:
    - `-t OUTPUT_DESTINATION, --to OUTPUT_DESTINATION, --import-to OUTPUT_DESTINATION`
    : destination directory; required
    - `-o OUTPUT_FORMAT, --output OUTPUT_FORMAT`
    : format describing generated output paths, an alias name or "format:" followed by a custom pythonic  string; same expression format as `hoardy-web organize --output` (which see); default: `default`

  - new `--output`s printing:
    - `--no-print`
    : don't print anything; default
    - `-l, --lf-terminated`
    : print absolute paths of newly produced or replaced files terminated with `\n` (LF) newline characters
    - `-z, --zero-terminated`
    : print absolute paths of newly produced or replaced files terminated with `\0` (NUL) bytes

  - updates to `--output`s:
    - `--no-overwrite`
    : disallow overwrites and replacements of any existing files under `OUTPUT_DESTINATION`, i.e. only ever create new files under `OUTPUT_DESTINATION`, producing errors instead of attempting any other updates; default
    - `--overwrite-dangerously`
    : permit overwrites to files under `OUTPUT_DESTINATION`;
      DANGEROUS! not recommended, importing to a new `OUTPUT_DESTINATION` with the default `--no-overwrite` and then `rsync`ing some of the files over to the old `OUTPUT_DESTINATION` is a safer way to do this



  Parse each `INPUT` `PATH` as `mitmproxy` stream dump (by using `mitmproxy`'s own parser) into a sequence of reqres and then generate and place their `WRR` dumps into separate `WRR` files under `OUTPUT_DESTINATION` with paths derived from their metadata.

  - options:
    - `--dry-run`
    : perform a trial run without actually performing any changes
    - `-q, --quiet`
    : don't log computed updates and don't print end-of-filtering warnings to stderr

  - caching, deferring, and batching:
    - `--seen-number INT`
    : track at most this many distinct generated `--output` values; default: `16384`;
      making this larger improves disk performance at the cost of increased memory consumption;
      setting it to zero will force force `hoardy-web` to constantly re-check existence of `--output` files and force `hoardy-web` to execute  all IO actions immediately, disregarding `--defer-number` setting
    - `--cache-number INT`
    : cache `stat(2)` information about this many files in memory; default: `8192`;
      making this larger improves performance at the cost of increased memory consumption;
      setting this to a too small number will likely force `hoardy-web` into repeatedly performing lots of `stat(2)` system calls on the same files;
      setting this to a value smaller than `--defer-number` will not improve memory consumption very much since deferred IO actions also cache information about their own files
    - `--defer-number INT`
    : defer at most this many IO actions; default: `0`;
      making this larger improves performance at the cost of increased memory consumption;
      setting it to zero will force all IO actions to be applied immediately
    - `--batch-number INT`
    : queue at most this many deferred IO actions to be applied together in a batch; this queue will only be used if all other resource constraints are met; default: `1024`
    - `--max-memory INT`
    : the caches, the deferred actions queue, and the batch queue, all taken together, must not take more than this much memory in MiB; default: `1024`;
      making this larger improves performance;
      the actual maximum whole-program memory consumption is `O(<size of the largest reqres> + <--seen-number> + <sum of lengths of the last --seen-number generated --output paths> + <--cache-number> + <--defer-number> + <--batch-number> + <--max-memory>)`
    - `--lazy`
    : sets all of the above options to positive infinity;
      most useful when doing `hoardy-web organize --symlink --latest --output flat` or similar, where the number of distinct generated `--output` values and the amount of other data `hoardy-web` needs to keep in memory is small, in which case it will force `hoardy-web` to compute the desired file system state first and then perform all disk writes in a single batch

  - error handling:
    - `--errors {fail,skip,ignore}`
    : when an error occurs:
      - `fail`: report failure and stop the execution; default
      - `skip`: report failure but skip the reqres that produced it from the output and continue
      - `ignore`: `skip`, but don't report the failure

  - path ordering:
    - `--paths-given-order`
    : `argv` and `--stdin0` `PATH`s are processed in the order they are given; default
    - `--paths-sorted`
    : `argv` and `--stdin0` `PATH`s are processed in lexicographic order
    - `--paths-reversed`
    : `argv` and `--stdin0` `PATH`s are processed in reverse lexicographic order
    - `--walk-fs-order`
    : recursive file system walk is done in the order `readdir(2)` gives results
    - `--walk-sorted`
    : recursive file system walk is done in lexicographic order; default
    - `--walk-reversed`
    : recursive file system walk is done in reverse lexicographic order

  - input loading:
    - `--load-any`
    : for each given input `PATH`, decide which loader to use based on its file extension; default
    - `--load-wrr`
    : load all inputs using the single-`WRR` per-file loader
    - `--load-wrrb`
    : load all inputs using the `WRR` bundle loader, this will load separate `WRR` files as single-`WRR` bundles too
    - `--load-mitmproxy`
    : load inputs using the `mitmproxy` dump loader
    - `--stdin0`
    : read zero-terminated `PATH`s from stdin, these will be processed after `PATH`s specified as command-line arguments
    - `PATH`
    : inputs, can be a mix of files and directories (which will be traversed recursively)

  - `MIME` type sniffing; this controls the use of the [`mimesniff` algorithm](https://mimesniff.spec.whatwg.org/); for this sub-command this influeences generated file names because `filepath_parts` and `filepath_ext` of `hoardy-web get --expr` (which see) depend on both the original file extension present in the URL and the detected `MIME` type of its content:
    - `--sniff-default`
    : run `mimesniff` when the spec says it should be run; i.e. trust `Content-Type` `HTTP` headers most of the time; default
    - `--sniff-force`
    : run `mimesniff` regardless of what `Content-Type`  and `X-Content-Type-Options` `HTTP` headers say; i.e. for each reqres, run `mimesniff` algorithm on the `Content-Type` `HTTP` header and the actual contents of `(request|response).body` (depending on the first argument of `scrub`) to determine what the body actually contains, then interpret the data as intersection of what `Content-Type` and `mimesniff` claim it to be; e.g. if `Content-Type` says `text/plain` but `mimesniff` says `text/plain or text/javascript`, interpret it as `text/plain`
    - `--sniff-paranoid`
    : do what `--sniff-force` does, but interpret the results in the most paranoid way possible; e.g. if `Content-Type` says `text/plain` but `mimesniff` says `text/plain or text/javascript`, interpret it as `text/plain or text/javascript`; which, for instance, will then make `scrub` with `-scripts` censor it out, since it can be interpreted as a script

  - file outputs:
    - `-t OUTPUT_DESTINATION, --to OUTPUT_DESTINATION, --import-to OUTPUT_DESTINATION`
    : destination directory; required
    - `-o OUTPUT_FORMAT, --output OUTPUT_FORMAT`
    : format describing generated output paths, an alias name or "format:" followed by a custom pythonic  string; same expression format as `hoardy-web organize --output` (which see); default: `default`

  - new `--output`s printing:
    - `--no-print`
    : don't print anything; default
    - `-l, --lf-terminated`
    : print absolute paths of newly produced or replaced files terminated with `\n` (LF) newline characters
    - `-z, --zero-terminated`
    : print absolute paths of newly produced or replaced files terminated with `\0` (NUL) bytes

  - updates to `--output`s:
    - `--no-overwrite`
    : disallow overwrites and replacements of any existing files under `OUTPUT_DESTINATION`, i.e. only ever create new files under `OUTPUT_DESTINATION`, producing errors instead of attempting any other updates; default
    - `--overwrite-dangerously`
    : permit overwrites to files under `OUTPUT_DESTINATION`;
      DANGEROUS! not recommended, importing to a new `OUTPUT_DESTINATION` with the default `--no-overwrite` and then `rsync`ing some of the files over to the old `OUTPUT_DESTINATION` is a safer way to do this



  Generate a local offline static website mirror from given intuts, producing results similar to those of `wget -mpk`.

  Algorithm:

  - index all given inputs, for each input `PATH`:
    - load it;
    - check this reqres satisfies given filters and skip it if it does not,
    - if there are no root filters set or if it satisfies given root filters, queue it for mirroring;
    - either remember its location (or, for some types of files, its contents) for future use or forget about it (e.g., if running with `--latest` and this input is older than the already indexed one);
  - then, for each reqres in the queue, mirror it:
    - evaluate all `EXPR` expressions on the reqres (which, by default, takes its response body and rewrites all links to point to locally mirrored files);
    - if the document being mirrored has resource requisites, mirror them recursively,
    - if the document being mirrored references other documents and the current depth is smaller than `DEPTH`, queue those documents for mirroring too,
    - write the result of evaluating `EXPR`s into a separate file under `OUTPUT_DESTINATION` with its path derived from reqres' metadata.

  The end.

  Essentially, this is a combination of `hoardy-web organize --copy` followed by in-place `hoardy-web get` which has the advanced URL remapping capabilities of `(*|/|&)(jumps|actions|reqs)` options available in its `scrub` function.

  - options:
    - `--dry-run`
    : perform a trial run without actually performing any changes
    - `-q, --quiet`
    : don't log computed updates and don't print end-of-filtering warnings to stderr

  - caching:
    - `--max-memory INT`
    : the caches, all taken together, must not take more than this much memory in MiB; default: `1024`;
      making this larger improves performance;
      the actual maximum whole-program memory consumption is `O(<size of the largest reqres> + <numer of indexed files> + <sum of lengths of all their --output paths> + <--max-memory>)`

  - error handling:
    - `--errors {fail,skip,ignore}`
    : when an error occurs:
      - `fail`: report failure and stop the execution; default
      - `skip`: report failure but skip the reqres that produced it from the output and continue
      - `ignore`: `skip`, but don't report the failure

  - path ordering:
    - `--paths-given-order`
    : `argv` and `--stdin0` `PATH`s are processed in the order they are given; default
    - `--paths-sorted`
    : `argv` and `--stdin0` `PATH`s are processed in lexicographic order
    - `--paths-reversed`
    : `argv` and `--stdin0` `PATH`s are processed in reverse lexicographic order
    - `--walk-fs-order`
    : recursive file system walk is done in the order `readdir(2)` gives results
    - `--walk-sorted`
    : recursive file system walk is done in lexicographic order; default
    - `--walk-reversed`
    : recursive file system walk is done in reverse lexicographic order

  - input loading:
    - `--load-any`
    : for each given input `PATH`, decide which loader to use based on its file extension; default
    - `--load-wrr`
    : load all inputs using the single-`WRR` per-file loader
    - `--load-wrrb`
    : load all inputs using the `WRR` bundle loader, this will load separate `WRR` files as single-`WRR` bundles too
    - `--load-mitmproxy`
    : load inputs using the `mitmproxy` dump loader
    - `--stdin0`
    : read zero-terminated `PATH`s from stdin, these will be processed after `PATH`s specified as command-line arguments
    - `--boring PATH`
    : low-priority input `PATH`; boring `PATH`s will be processed after all `PATH`s specified as positional command-line arguments and those given via `--stdin0` and will not be queued as roots even when no `--root-*` options are specified
    - `PATH`
    : inputs, can be a mix of files and directories (which will be traversed recursively)

  - `MIME` type sniffing; this controls the use of the [`mimesniff` algorithm](https://mimesniff.spec.whatwg.org/); for this sub-command this influeences generated file names because `filepath_parts` and `filepath_ext` of `hoardy-web get --expr` (which see) depend on both the original file extension present in the URL and the detected `MIME` type of its content; also, higher values make the `scrub` function (which see) censor out more things when `-unknown`, `-styles`, or `-scripts` options are set; in particular, at the moment, with `--sniff-paranoid` and `-scripts` most plain text files will be censored out as potential `JavaScript`:
    - `--sniff-default`
    : run `mimesniff` when the spec says it should be run; i.e. trust `Content-Type` `HTTP` headers most of the time; default
    - `--sniff-force`
    : run `mimesniff` regardless of what `Content-Type`  and `X-Content-Type-Options` `HTTP` headers say; i.e. for each reqres, run `mimesniff` algorithm on the `Content-Type` `HTTP` header and the actual contents of `(request|response).body` (depending on the first argument of `scrub`) to determine what the body actually contains, then interpret the data as intersection of what `Content-Type` and `mimesniff` claim it to be; e.g. if `Content-Type` says `text/plain` but `mimesniff` says `text/plain or text/javascript`, interpret it as `text/plain`
    - `--sniff-paranoid`
    : do what `--sniff-force` does, but interpret the results in the most paranoid way possible; e.g. if `Content-Type` says `text/plain` but `mimesniff` says `text/plain or text/javascript`, interpret it as `text/plain or text/javascript`; which, for instance, will then make `scrub` with `-scripts` censor it out, since it can be interpreted as a script

  - default input filters:
    - `--ignore-bad-inputs`
    : initialize input filters to `--status-re ".200C"`; default
    - `--index-all-inputs`
    : do not set any input filters by default; note, however, that this sub-command expects input filters to be at least as restrictive as the above default and is likely to produce broken outputs if those filters are unset; use this option at your own risk

  - expression evaluation:
    - `-e EXPR, --expr EXPR`
    : an expression to compute, same expression format and semantics as `hoardy-web get --expr` (which see); can be specified multiple times; the default depends on `--remap-*` options below

  - rendering of `--expr` values:
    - `--not-separated`
    : render `--expr` values into outputs without separating them with anything, just concatenate them
    - `--lf-separated`
    : render `--expr` values into outputs separated with `\n` (LF) newline characters; default
    - `--zero-separated`
    : render `--expr` values into outputs separated with `\0` (NUL) bytes

  - default value of `--expr`:
    - `--raw-qbody`
    : set the default value of `--expr` to `request.body|eb`; i.e. produce the raw request body
    - `--raw-sbody, --no-remap`
    : set the default value of `--expr` to `response.body|eb`; i.e. produce the raw response body
    - `--remap-id`
    : set the default value of `--expr` to `response.body|eb|scrub response +all_refs`; i.e. `scrub` response body as follows: remap all URLs with an identity function (which, as a whole, is NOT an identity function, it will transform all relative URLs into absolute ones), censor out all dynamic content (e.g. `JavaScript`); results will NOT be self-contained
    - `--remap-void`
    : set the default value of `--expr` to `response.body|eb|scrub response -all_refs`; i.e. `scrub` response body as follows: remap all URLs into `javascript:void(0)` and empty `data:` URLs, censor out all dynamic content; results will be self-contained
    - `--remap-open, -k, --convert-links`
    : set the default value of `--expr` to `response.body|eb|scrub response *all_refs`; i.e. `scrub` response body as follows: remap all URLs present in input `PATH`s and reachable from `--root-*`s in no more that `--depth` steps to their corresponding `--output` paths, remap all other URLs like `--remap-id` does, censor out all dynamic content; results almost certainly will NOT be self-contained
    - `--remap-closed`
    : set the default value of `--expr` to `response.body|eb|scrub response /all_refs`; i.e. `scrub` response body as follows: remap all URLs present in input `PATH`s and reachable from `--root-*`s in no more that `--depth` steps to their corresponding `--output` paths, remap all other URLs like `--remap-void` does, censor out all dynamic content; results will be self-contained
    - `--remap-semi`
    : set the default value of `--expr` to `response.body|eb|scrub response *jumps,/actions,/reqs`; i.e. `scrub` response body as follows: remap all jump links like `--remap-open` does, remap action links and references to page requisites like `--remap-closed` does, censor out all dynamic content; this is a better version of `--remap-open` which keeps the `mirror`s self-contained with respect to page requisites, i.e. generated pages can be opened in a web browser without it trying to access the Internet, but all navigations to missing and unreachable URLs will still point to the original URLs; results will be semi-self-contained
    - `--remap-all`
    : set the default value of `--expr` to `response.body|eb|scrub response &all_refs`; i.e. `scrub` response body as follows: remap all links and references like `--remap-closed` does, except, instead of voiding missing and unreachable URLs, replace them with fallback URLs whenever possble, censor out all dynamic content; results will be self-contained; default

      `hoardy-web mirror` uses `--output` paths of trivial `GET <URL> -> 200 OK` as fallbacks for `&(jumps|actions|reqs)` options of `scrub`.
      This will remap links pointing to missing and unreachable URLs to missing files.
      However, for simple `--output` formats (like the default `hupq`), those files can later be generated by running `hoardy-web mirror` with `WRR` files containing those missing or unreachable URLs as inputs.
      I.e. this behaviour allows you to add new data to an already existing mirror without regenerating old files that reference newly added URLs.
      I.e. this allows `hoardy-web mirror` to be used incrementally.

      Note however, that using fallbacks when the `--output` format depends on anything but the URL itself (e.g. if it mentions timestamps) will produce a mirror with unrecoverably broken links.

  - link conversions:
    - `--relative`
    : when remapping URLs to local files, produce links and references with relative URLs (relative to the `--output` files under `OUTPUT_DESTINATION`); default when `--copy` or `--hardlink`
    - `--absolute`
    : when remapping URLs to local files, produce links and references with absolute URLs; default when `--symlink`

  - what gets mirrored:
    - `--oldest`
    : for each URL, mirror its oldest available version
    - `--oldest-hybrid`
    : for each URL, mirror its oldest available version, except, for each URL that is a requisite resource, mirror a version that is time-closest to the referencing document; i.e., this will make each mirrored page refer to requisites (images, media, `CSS`, fonts, etc) that were archived around the time the page itself was archived, even if those requisite resources changed in time; this produces results that are as close to the original web page as possible at the cost of much more memory to `mirror`
    - `--nearest INTERVAL_DATE`
    : for each URL, mirror an available version that is closest to the given `INTERVAL_DATE` value; the `INTERVAL_DATE` is parsed as a time interval the middle point of which is taken as target value; e.g., `2024` becomes `2024-07-02 00:00:00` (which is the exact middle point of that year), `2024-12-31` becomes `2024-12-31 12:00:00`, `2024-12-31 12` -> `2024-12-31 12:30:00`, `2024-12-31 12:00` -> `2024-12-31 12:00:30`, `2024-12-31 12:00:01` -> `2024-12-31 12:00:01.5`, etc
    - `--nearest-hybrid INTERVAL_DATE`
    : for each URL, mirror an available version that is closest to the given `INTERVAL_DATE` value; the `INTERVAL_DATE` format and semantics is the same as above, except, for each URL that is a requisite resource, mirror a version that is time-closest to the referencing document; see `--oldest-hybrid` above for more info
    - `--latest`
    : for each URL, mirror its latest available version; default
    - `--latest-hybrid`
    : for each URL, mirror its latest available version, except, for each URL that is a requisite resource, mirror a version that is time-closest to the referencing document; see `--oldest-hybrid` above for more info
    - `--all`
    : mirror all available versions of all available URLs; this is likely to take a lot of time and eat a lot of memory!

  - file outputs:
    - `-t OUTPUT_DESTINATION, --to OUTPUT_DESTINATION, --mirror-to OUTPUT_DESTINATION`
    : destination directory; required
    - `-o OUTPUT_FORMAT, --output OUTPUT_FORMAT`
    : format describing generated output paths, an alias name or "format:" followed by a custom pythonic  string; same expression format as `hoardy-web organize --output` (which see); default: `hupq_n`

  - new `--output`s printing:
    - `--no-print`
    : don't print anything; default
    - `-l, --lf-terminated`
    : print absolute paths of newly produced or replaced files terminated with `\n` (LF) newline characters
    - `-z, --zero-terminated`
    : print absolute paths of newly produced or replaced files terminated with `\0` (NUL) bytes

  - updates to `--output`s:
    - `--no-overwrite`
    : disallow overwrites and replacements of any existing files under `OUTPUT_DESTINATION`, i.e. only ever create new files under `OUTPUT_DESTINATION`, producing errors instead of attempting any other updates; default;
      repeated `mirror`s of the same targets with the same parameters (which, therefore, will produce the same `--output` data) are allowed and will be reduced to noops;
      however, trying to overwrite existing files under `OUTPUT_DESTINATION` with any new data will produce errors;
      this allows reusing the `OUTPUT_DESTINATION` between unrelated `mirror`s and between `mirror`s that produce the same data on disk in their common parts
    - `--skip-existing, --partial`
    : skip rendering of targets which have a corresponding file under `OUTPUT_DESTINATION`, use the contents of such files instead;
      using this together with `--depth` is likely to produce a partially broken result, since skipping of a document will also skip all of the things it references;
      on the other hand, this is quite useful when growing a partial mirror generated with `--remap-all`
    - `--overwrite-dangerously`
    : mirror all targets while permitting overwriting of old `--output` files under `OUTPUT_DESTINATION`;
      DANGEROUS! not recommended, mirroring to a new `OUTPUT_DESTINATION` with the default `--no-overwrite` and then `rsync`ing some of the files over to the old `OUTPUT_DESTINATION` is a safer way to do this

  - content-addressed file output mode:
    - `--copy`
    : do not use content-addressed outputs, simply write rendered output data to files under `OUTPUT_DESTINATION`
    - `--hardlink`
    : write rendered output data to files under `CONTENT_DESTINATION`, then hardlink them to paths under `OUTPUT_DESTINATION`; default
    - `--symlink`
    : write rendered output data to files under `CONTENT_DESTINATION`, then symlink them to paths under `OUTPUT_DESTINATION`

  - content-addressed file output settings:
    - `--content-to CONTENT_DESTINATION`
    : content-addressed destination directory; if not specified, reuses `OUTPUT_DESTINATION`
    - `--content-output CONTENT_FORMAT`
    : format describing generated content-addressed output paths, an alias name or "format:" followed by a custom pythonic  string:
      - available aliases and corresponding :
        - `default`     : `_content/sha256/s/ss`; the default
      - available substitutions:
        - all expressions of `hoardy-web get --expr` (which see);
        - `content`: rendered content
        - `content_sha256`: alias for `content|sha256`

  - recursion root filters; if none are specified, then all URLs available from input `PATH`s will be treated as roots (except for those given via `--boring`); can be specified multiple times in arbitrary combinations; the resulting logical expression that will be checked is `all_of(before) and all_of(not_before) and all_of(after) and all_of(not_after) and any_of(protocol) and not any_of(not_protcol) and any_of(request_method) and not any_of(not_request_method) ... and any_of(grep) and not any_of(not_grep) and all_of(and_grep) and not all_of(not_and_grep) and all_of(ands) and any_of(ors)`:
    - `--root-before DATE`
    : take reqres as a root when its `stime` is smaller than this; the `DATE` can be specified either as a number of seconds since UNIX epoch using `@<number>` format where `<number>` can be a floating point, or using one of the following formats:`YYYY-mm-DD HH:MM:SS[.NN*] (+|-)HHMM`, `YYYY-mm-DD HH:MM:SS[.NN*]`, `YYYY-mm-DD HH:MM:SS`, `YYYY-mm-DD HH:MM`, `YYYY-mm-DD`, `YYYY-mm`, `YYYY`; if no `(+|-)HHMM` part is specified, the `DATE` is assumed to be in local time; if other parts are unspecified they are inherited from `<year>-01-01 00:00:00.0`
    - `--root-not-before DATE`
    : take reqres as a root when its `stime` is larger or equal than this; the `DATE` format is the same as above
    - `--root-after DATE`
    : take reqres as a root when its `stime` is larger than this; the `DATE` format is the same as above
    - `--root-not-after DATE`
    : take reqres as a root when its `stime` is smaller or equal than this; the `DATE` format is the same as above
    - `--root-protocol PROTOCOL`
    : take reqres as a root when one of the given `PROTOCOL` option arguments is equal to its `protocol` (of `hoardy-web get --expr`, which see); in short, this option defines a whitelisted element rule
    - `--root-protocol-prefix PROTOCOL_PREFIX`
    : take reqres as a root when one of the given `PROTOCOL_PREFIX` option arguments is a prefix of its `protocol` (of `hoardy-web get --expr`, which see); in short, this option defines a whitelisted element rule
    - `--root-protocol-re PROTOCOL_RE`
    : take reqres as a root when one of the given `PROTOCOL_RE` regular expressions matches its `protocol` (of `hoardy-web get --expr`, which see); this option matches the given regular expression against the whole input value; to match against any part of the input value, use `.*<re>.*` or `^.*<re>.*$`; in short, this option defines a whitelisted element rule
    - `--root-not-protocol NOT_PROTOCOL`
    : take reqres as a root when none of the given `NOT_PROTOCOL` option arguments are equal to its `protocol` (of `hoardy-web get --expr`, which see); in short, this option defines a blacklisted element rule
    - `--root-not-protocol-prefix NOT_PROTOCOL_PREFIX`
    : take reqres as a root when none of the given `NOT_PROTOCOL_PREFIX` option arguments are a prefix of its `protocol` (of `hoardy-web get --expr`, which see); in short, this option defines a blacklisted element rule
    - `--root-not-protocol-re NOT_PROTOCOL_RE`
    : take reqres as a root when none of the given `NOT_PROTOCOL_RE` regular expressions match its `protocol` (of `hoardy-web get --expr`, which see); this option matches the given regular expression against the whole input value; to match against any part of the input value, use `.*<re>.*` or `^.*<re>.*$`; in short, this option defines a blacklisted element rule
    - `--root-request-method REQUEST_METHOD, --root-method REQUEST_METHOD`
    : take reqres as a root when one of the given `REQUEST_METHOD` option arguments is equal to its `request.method` (of `hoardy-web get --expr`, which see); in short, this option defines a whitelisted element rule
    - `--root-request-method-prefix REQUEST_METHOD_PREFIX, --root-method-prefix REQUEST_METHOD_PREFIX`
    : take reqres as a root when one of the given `REQUEST_METHOD_PREFIX` option arguments is a prefix of its `request.method` (of `hoardy-web get --expr`, which see); in short, this option defines a whitelisted element rule
    - `--root-request-method-re REQUEST_METHOD_RE, --root-method-re REQUEST_METHOD_RE`
    : take reqres as a root when one of the given `REQUEST_METHOD_RE` regular expressions matches its `request.method` (of `hoardy-web get --expr`, which see); this option matches the given regular expression against the whole input value; to match against any part of the input value, use `.*<re>.*` or `^.*<re>.*$`; in short, this option defines a whitelisted element rule
    - `--root-not-request-method NOT_REQUEST_METHOD, --root-not-method NOT_REQUEST_METHOD`
    : take reqres as a root when none of the given `NOT_REQUEST_METHOD` option arguments are equal to its `request.method` (of `hoardy-web get --expr`, which see); in short, this option defines a blacklisted element rule
    - `--root-not-request-method-prefix NOT_REQUEST_METHOD_PREFIX`
    : take reqres as a root when none of the given `NOT_REQUEST_METHOD_PREFIX` option arguments are a prefix of its `request.method` (of `hoardy-web get --expr`, which see); in short, this option defines a blacklisted element rule
    - `--root-not-request-method-re NOT_REQUEST_METHOD_RE`
    : take reqres as a root when none of the given `NOT_REQUEST_METHOD_RE` regular expressions match its `request.method` (of `hoardy-web get --expr`, which see); this option matches the given regular expression against the whole input value; to match against any part of the input value, use `.*<re>.*` or `^.*<re>.*$`; in short, this option defines a blacklisted element rule
    - `--root-status STATUS`
    : take reqres as a root when one of the given `STATUS` option arguments is equal to its `status` (of `hoardy-web get --expr`, which see); in short, this option defines a whitelisted element rule
    - `--root-status-prefix STATUS_PREFIX`
    : take reqres as a root when one of the given `STATUS_PREFIX` option arguments is a prefix of its `status` (of `hoardy-web get --expr`, which see); in short, this option defines a whitelisted element rule
    - `--root-status-re STATUS_RE`
    : take reqres as a root when one of the given `STATUS_RE` regular expressions matches its `status` (of `hoardy-web get --expr`, which see); this option matches the given regular expression against the whole input value; to match against any part of the input value, use `.*<re>.*` or `^.*<re>.*$`; in short, this option defines a whitelisted element rule
    - `--root-not-status NOT_STATUS`
    : take reqres as a root when none of the given `NOT_STATUS` option arguments are equal to its `status` (of `hoardy-web get --expr`, which see); in short, this option defines a blacklisted element rule
    - `--root-not-status-prefix NOT_STATUS_PREFIX`
    : take reqres as a root when none of the given `NOT_STATUS_PREFIX` option arguments are a prefix of its `status` (of `hoardy-web get --expr`, which see); in short, this option defines a blacklisted element rule
    - `--root-not-status-re NOT_STATUS_RE`
    : take reqres as a root when none of the given `NOT_STATUS_RE` regular expressions match its `status` (of `hoardy-web get --expr`, which see); this option matches the given regular expression against the whole input value; to match against any part of the input value, use `.*<re>.*` or `^.*<re>.*$`; in short, this option defines a blacklisted element rule
    - `--root-url URL`
    : take reqres as a root when one of the given `URL` option arguments is equal to its `net_url` (of `hoardy-web get --expr`, which see); Punycode UTS46 IDNAs, plain UNICODE IDNAs, percent-encoded URL components, and UNICODE URL components in arbitrary mixes and combinations are allowed; e.g. `https://xn--hck7aa9d8fj9i.ですの.example.org/исп%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/` will be silently normalized into its Punycode UTS46 and percent-encoded version of `https://xn--hck7aa9d8fj9i.xn--88j1aw.example.org/%D0%B8%D1%81%D0%BF%D1%8B%D1%82%D0%B0%D0%BD%D0%B8%D0%B5/is/`, which will then be matched against; in short, this option defines a whitelisted element rule
    - `--root-url-prefix URL_PREFIX, --root URL_PREFIX, -r URL_PREFIX`
    : take reqres as a root when one of the given `URL_PREFIX` option arguments is a prefix of its `net_url` (of `hoardy-web get --expr`, which see); similarly to the previous option, arbitrary mixes of URL encodinds are allowed; in short, this option defines a whitelisted element rule
    - `--root-url-re URL_RE`
    : take reqres as a root when one of the given `URL_RE` regular expressions matches its `net_url` or `pretty_net_url` (of `hoardy-web get --expr`, which see); only Punycode UTS46 IDNAs with percent-encoded URL components or plain UNICODE IDNAs with UNICODE URL components are allowed; regular expressions that use mixes of differently encoded parts will fail to match properly; this option matches the given regular expression against the whole input value; to match against any part of the input value, use `.*<re>.*` or `^.*<re>.*$`; in short, this option defines a whitelisted element rule
    - `--root-not-url NOT_URL`
    : take reqres as a root when none of the given `NOT_URL` option arguments are equal to its `net_url` (of `hoardy-web get --expr`, which see); option argument format and caveats are idential to the `not-`less option above; in short, this option defines a blacklisted element rule
    - `--root-not-url-prefix NOT_URL_PREFIX`
    : take reqres as a root when none of the given `NOT_URL_PREFIX` option arguments are a prefix of its `net_url` (of `hoardy-web get --expr`, which see); option argument format and caveats are idential to the `not-`less option above; in short, this option defines a blacklisted element rule
    - `--root-not-url-re NOT_URL_RE`
    : take reqres as a root when none of the given `NOT_URL_RE` regular expressions match its `net_url` or `pretty_net_url` (of `hoardy-web get --expr`, which see); option argument format and caveats are idential to the `not-`less option above; in short, this option defines a blacklisted element rule
    - `--root-request-headers-or-grep OR_PATTERN, --root-request-headers-grep OR_PATTERN`
    : take reqres as a root when at least one of the given `OR_PATTERN` option arguments is a substring of at least one of the elements of the list containing all `request.headers` (of `hoardy-web get --expr`, which see); each `HTTP` header of `*.headers` is matched as a single `<header_name>: <header_value>` value; at the moment, binary values are matched against given option arguments by encoding the latter into `UTF-8` first, which means that `*.headers` and `*.body` values that use encodings other than `UTF-8` are not guaranteed to match properly; in short, this option defines a whitelisted element rule
    - `--root-request-headers-or-grep-re OR_PATTERN_RE, --root-request-headers-grep-re OR_PATTERN_RE`
    : take reqres as a root when at least one of the given `OR_PATTERN_RE` regular expressions matches a substring of at least one of the elements of the above list; matching caveats are the same as above; in short, this option defines a whitelisted element rule
    - `--root-not-request-headers-or-grep NOT_OR_PATTERN, --root-not-request-headers-grep NOT_OR_PATTERN`
    : take reqres as a root when none of the given `NOT_OR_PATTERN` option arguments are substrings of any of the elements of the above list; matching caveats are the same as above; in short, this option defines a blacklisted element rule
    - `--root-not-request-headers-or-grep-re NOT_OR_PATTERN_RE, --root-not-request-headers-grep-re NOT_OR_PATTERN_RE`
    : take reqres as a root when none of the given `NOT_OR_PATTERN_RE` regular expressions match any substrings of any of the elements of the above list; matching caveats are the same as above; in short, this option defines a blacklisted element rule
    - `--root-request-headers-and-grep AND_PATTERN`
    : take reqres as a root when each of the given `AND_PATTERN` option arguments is a substring of some element of the above list; matching caveats are the same as above
    - `--root-request-headers-and-grep-re AND_PATTERN_RE`
    : take reqres as a root when each of the given `AND_PATTERN_RE` regular expressions matches a substring of some element of the above list; matching caveats are the same as above
    - `--root-not-request-headers-and-grep NOT_AND_PATTERN`
    : take reqres as a root when one or more of the given `NOT_AND_PATTERN` option arguments is not a substring of the elements of the above list; matching caveats are the same as above
    - `--root-not-request-headers-and-grep-re NOT_AND_PATTERN_RE`
    : take reqres as a root when one or more of the given `NOT_AND_PATTERN_RE` regular expressions fails to match any substrings of the elements of the above list; matching caveats are the same as above
    - `--root-request-body-or-grep OR_PATTERN, --root-request-body-grep OR_PATTERN`
    : take reqres as a root when at least one of the given `OR_PATTERN` option arguments is a substring of `request.body` (of `hoardy-web get --expr`, which see); at the moment, binary values are matched against given option arguments by encoding the latter into `UTF-8` first, which means that `*.headers` and `*.body` values that use encodings other than `UTF-8` are not guaranteed to match properly; in short, this option defines a whitelisted element rule
    - `--root-request-body-or-grep-re OR_PATTERN_RE, --root-request-body-grep-re OR_PATTERN_RE`
    : take reqres as a root when at least one of the given `OR_PATTERN_RE` regular expressions matches a substring of `request.body`; matching caveats are the same as above; in short, this option defines a whitelisted element rule
    - `--root-not-request-body-or-grep NOT_OR_PATTERN, --root-not-request-body-grep NOT_OR_PATTERN`
    : take reqres as a root when none of the given `NOT_OR_PATTERN` option arguments are substrings of `request.body`; matching caveats are the same as above; in short, this option defines a blacklisted element rule
    - `--root-not-request-body-or-grep-re NOT_OR_PATTERN_RE, --root-not-request-body-grep-re NOT_OR_PATTERN_RE`
    : take reqres as a root when none of the given `NOT_OR_PATTERN_RE` regular expressions match any substrings of `request.body`; matching caveats are the same as above; in short, this option defines a blacklisted element rule
    - `--root-request-body-and-grep AND_PATTERN`
    : take reqres as a root when each of the given `AND_PATTERN` option arguments is a substring of `request.body`; matching caveats are the same as above
    - `--root-request-body-and-grep-re AND_PATTERN_RE`
    : take reqres as a root when each of the given `AND_PATTERN_RE` regular expressions matches a substring of `request.body`; matching caveats are the same as above
    - `--root-not-request-body-and-grep NOT_AND_PATTERN`
    : take reqres as a root when one or more of the given `NOT_AND_PATTERN` option arguments is not a substring of `request.body`; matching caveats are the same as above
    - `--root-not-request-body-and-grep-re NOT_AND_PATTERN_RE`
    : take reqres as a root when one or more of the given `NOT_AND_PATTERN_RE` regular expressions fails to match any substrings of `request.body`; matching caveats are the same as above
    - `--root-request-mime REQUEST_MIME`
    : take reqres as a root when one of the given `REQUEST_MIME` option arguments is equal to its `request_mime` (of `hoardy-web get --expr`, which see); both canonical and non-canonical MIME types are allowed; e.g., giving `application/x-grip` or `application/gzip` will produce the same predicate; in short, this option defines a whitelisted element rule
    - `--root-request-mime-prefix REQUEST_MIME_PREFIX`
    : take reqres as a root when one of the given `REQUEST_MIME_PREFIX` option arguments is a prefix of its `request_mime` (of `hoardy-web get --expr`, which see); given prefixes will only ever be matched against canonicalized MIME types; in short, this option defines a whitelisted element rule
    - `--root-request-mime-re REQUEST_MIME_RE`
    : take reqres as a root when one of the given `REQUEST_MIME_RE` regular expressions matches its `request_mime` (of `hoardy-web get --expr`, which see); given regular expressions will only ever be matched against canonicalized MIME types; this option matches the given regular expression against the whole input value; to match against any part of the input value, use `.*<re>.*` or `^.*<re>.*$`; in short, this option defines a whitelisted element rule
    - `--root-not-request-mime NOT_REQUEST_MIME`
    : take reqres as a root when none of the given `NOT_REQUEST_MIME` option arguments are equal to its `request_mime` (of `hoardy-web get --expr`, which see); option argument format and caveats are idential to the `not-`less option above; in short, this option defines a blacklisted element rule
    - `--root-not-request-mime-prefix NOT_REQUEST_MIME_PREFIX`
    : take reqres as a root when none of the given `NOT_REQUEST_MIME_PREFIX` option arguments are a prefix of its `request_mime` (of `hoardy-web get --expr`, which see); option argument format and caveats are idential to the `not-`less option above; in short, this option defines a blacklisted element rule
    - `--root-not-request-mime-re NOT_REQUEST_MIME_RE`
    : take reqres as a root when none of the given `NOT_REQUEST_MIME_RE` regular expressions match its `request_mime` (of `hoardy-web get --expr`, which see); option argument format and caveats are idential to the `not-`less option above; in short, this option defines a blacklisted element rule
    - `--root-response-headers-or-grep OR_PATTERN, --root-response-headers-grep OR_PATTERN`
    : take reqres as a root when at least one of the given `OR_PATTERN` option arguments is a substring of at least one of the elements of the list containing all `response.headers` (of `hoardy-web get --expr`, which see); each `HTTP` header of `*.headers` is matched as a single `<header_name>: <header_value>` value; at the moment, binary values are matched against given option arguments by encoding the latter into `UTF-8` first, which means that `*.headers` and `*.body` values that use encodings other than `UTF-8` are not guaranteed to match properly; in short, this option defines a whitelisted element rule
    - `--root-response-headers-or-grep-re OR_PATTERN_RE, --root-response-headers-grep-re OR_PATTERN_RE`
    : take reqres as a root when at least one of the given `OR_PATTERN_RE` regular expressions matches a substring of at least one of the elements of the above list; matching caveats are the same as above; in short, this option defines a whitelisted element rule
    - `--root-not-response-headers-or-grep NOT_OR_PATTERN, --root-not-response-headers-grep NOT_OR_PATTERN`
    : take reqres as a root when none of the given `NOT_OR_PATTERN` option arguments are substrings of any of the elements of the above list; matching caveats are the same as above; in short, this option defines a blacklisted element rule
    - `--root-not-response-headers-or-grep-re NOT_OR_PATTERN_RE, --root-not-response-headers-grep-re NOT_OR_PATTERN_RE`
    : take reqres as a root when none of the given `NOT_OR_PATTERN_RE` regular expressions match any substrings of any of the elements of the above list; matching caveats are the same as above; in short, this option defines a blacklisted element rule
    - `--root-response-headers-and-grep AND_PATTERN`
    : take reqres as a root when each of the given `AND_PATTERN` option arguments is a substring of some element of the above list; matching caveats are the same as above
    - `--root-response-headers-and-grep-re AND_PATTERN_RE`
    : take reqres as a root when each of the given `AND_PATTERN_RE` regular expressions matches a substring of some element of the above list; matching caveats are the same as above
    - `--root-not-response-headers-and-grep NOT_AND_PATTERN`
    : take reqres as a root when one or more of the given `NOT_AND_PATTERN` option arguments is not a substring of the elements of the above list; matching caveats are the same as above
    - `--root-not-response-headers-and-grep-re NOT_AND_PATTERN_RE`
    : take reqres as a root when one or more of the given `NOT_AND_PATTERN_RE` regular expressions fails to match any substrings of the elements of the above list; matching caveats are the same as above
    - `--root-response-body-or-grep OR_PATTERN, --root-response-body-grep OR_PATTERN`
    : take reqres as a root when at least one of the given `OR_PATTERN` option arguments is a substring of `response.body` (of `hoardy-web get --expr`, which see); at the moment, binary values are matched against given option arguments by encoding the latter into `UTF-8` first, which means that `*.headers` and `*.body` values that use encodings other than `UTF-8` are not guaranteed to match properly; in short, this option defines a whitelisted element rule
    - `--root-response-body-or-grep-re OR_PATTERN_RE, --root-response-body-grep-re OR_PATTERN_RE`
    : take reqres as a root when at least one of the given `OR_PATTERN_RE` regular expressions matches a substring of `response.body`; matching caveats are the same as above; in short, this option defines a whitelisted element rule
    - `--root-not-response-body-or-grep NOT_OR_PATTERN, --root-not-response-body-grep NOT_OR_PATTERN`
    : take reqres as a root when none of the given `NOT_OR_PATTERN` option arguments are substrings of `response.body`; matching caveats are the same as above; in short, this option defines a blacklisted element rule
    - `--root-not-response-body-or-grep-re NOT_OR_PATTERN_RE, --root-not-response-body-grep-re NOT_OR_PATTERN_RE`
    : take reqres as a root when none of the given `NOT_OR_PATTERN_RE` regular expressions match any substrings of `response.body`; matching caveats are the same as above; in short, this option defines a blacklisted element rule
    - `--root-response-body-and-grep AND_PATTERN`
    : take reqres as a root when each of the given `AND_PATTERN` option arguments is a substring of `response.body`; matching caveats are the same as above
    - `--root-response-body-and-grep-re AND_PATTERN_RE`
    : take reqres as a root when each of the given `AND_PATTERN_RE` regular expressions matches a substring of `response.body`; matching caveats are the same as above
    - `--root-not-response-body-and-grep NOT_AND_PATTERN`
    : take reqres as a root when one or more of the given `NOT_AND_PATTERN` option arguments is not a substring of `response.body`; matching caveats are the same as above
    - `--root-not-response-body-and-grep-re NOT_AND_PATTERN_RE`
    : take reqres as a root when one or more of the given `NOT_AND_PATTERN_RE` regular expressions fails to match any substrings of `response.body`; matching caveats are the same as above
    - `--root-response-mime RESPONSE_MIME`
    : take reqres as a root when one of the given `RESPONSE_MIME` option arguments is equal to its `response_mime` (of `hoardy-web get --expr`, which see); both canonical and non-canonical MIME types are allowed; e.g., giving `application/x-grip` or `application/gzip` will produce the same predicate; in short, this option defines a whitelisted element rule
    - `--root-response-mime-prefix RESPONSE_MIME_PREFIX`
    : take reqres as a root when one of the given `RESPONSE_MIME_PREFIX` option arguments is a prefix of its `response_mime` (of `hoardy-web get --expr`, which see); given prefixes will only ever be matched against canonicalized MIME types; in short, this option defines a whitelisted element rule
    - `--root-response-mime-re RESPONSE_MIME_RE`
    : take reqres as a root when one of the given `RESPONSE_MIME_RE` regular expressions matches its `response_mime` (of `hoardy-web get --expr`, which see); given regular expressions will only ever be matched against canonicalized MIME types; this option matches the given regular expression against the whole input value; to match against any part of the input value, use `.*<re>.*` or `^.*<re>.*$`; in short, this option defines a whitelisted element rule
    - `--root-not-response-mime NOT_RESPONSE_MIME`
    : take reqres as a root when none of the given `NOT_RESPONSE_MIME` option arguments are equal to its `response_mime` (of `hoardy-web get --expr`, which see); option argument format and caveats are idential to the `not-`less option above; in short, this option defines a blacklisted element rule
    - `--root-not-response-mime-prefix NOT_RESPONSE_MIME_PREFIX`
    : take reqres as a root when none of the given `NOT_RESPONSE_MIME_PREFIX` option arguments are a prefix of its `response_mime` (of `hoardy-web get --expr`, which see); option argument format and caveats are idential to the `not-`less option above; in short, this option defines a blacklisted element rule
    - `--root-not-response-mime-re NOT_RESPONSE_MIME_RE`
    : take reqres as a root when none of the given `NOT_RESPONSE_MIME_RE` regular expressions match its `response_mime` (of `hoardy-web get --expr`, which see); option argument format and caveats are idential to the `not-`less option above; in short, this option defines a blacklisted element rule
    - `--root-or-grep OR_PATTERN, --root-grep OR_PATTERN`
    : take reqres as a root when at least one of the given `OR_PATTERN` option arguments is a substring of at least one of the elements of the list containing `raw_url`, `url`, `pretty_url`, all `request.headers`, `request.body`, all `response.headers`, and `response.body` (of `hoardy-web get --expr`, which see); each `HTTP` header of `*.headers` is matched as a single `<header_name>: <header_value>` value; at the moment, binary values are matched against given option arguments by encoding the latter into `UTF-8` first, which means that `*.headers` and `*.body` values that use encodings other than `UTF-8` are not guaranteed to match properly; in short, this option defines a whitelisted element rule
    - `--root-or-grep-re OR_PATTERN_RE, --root-grep-re OR_PATTERN_RE`
    : take reqres as a root when at least one of the given `OR_PATTERN_RE` regular expressions matches a substring of at least one of the elements of the above list; matching caveats are the same as above; in short, this option defines a whitelisted element rule
    - `--root-not-or-grep NOT_OR_PATTERN, --root-not-grep NOT_OR_PATTERN`
    : take reqres as a root when none of the given `NOT_OR_PATTERN` option arguments are substrings of any of the elements of the above list; matching caveats are the same as above; in short, this option defines a blacklisted element rule
    - `--root-not-or-grep-re NOT_OR_PATTERN_RE, --root-not-grep-re NOT_OR_PATTERN_RE`
    : take reqres as a root when none of the given `NOT_OR_PATTERN_RE` regular expressions match any substrings of any of the elements of the above list; matching caveats are the same as above; in short, this option defines a blacklisted element rule
    - `--root-and-grep AND_PATTERN`
    : take reqres as a root when each of the given `AND_PATTERN` option arguments is a substring of some element of the above list; matching caveats are the same as above
    - `--root-and-grep-re AND_PATTERN_RE`
    : take reqres as a root when each of the given `AND_PATTERN_RE` regular expressions matches a substring of some element of the above list; matching caveats are the same as above
    - `--root-not-and-grep NOT_AND_PATTERN`
    : take reqres as a root when one or more of the given `NOT_AND_PATTERN` option arguments is not a substring of the elements of the above list; matching caveats are the same as above
    - `--root-not-and-grep-re NOT_AND_PATTERN_RE`
    : take reqres as a root when one or more of the given `NOT_AND_PATTERN_RE` regular expressions fails to match any substrings of the elements of the above list; matching caveats are the same as above
    - `--root-and EXPR`
    : take reqres as a root when all of the given expressions of the same format as `hoardy-web get --expr` (which see) evaluate to `true`
    - `--root-or EXPR`
    : take reqres as a root when some of the given expressions of the same format as `hoardy-web get --expr` (which see) evaluate to `true`

  - recursion depth:
    - `-d DEPTH, --depth DEPTH`
    : maximum recursion depth level; the default is `0`, which means "`--root-*` documents and their requisite resources only"; setting this to `1` will also mirror one level of documents referenced via jump and action links, if those are being remapped to local files with `--remap-*`; higher values will mean even more recursion



  Run an archiving server and/or serve given input files for replay over HTTP.

  Algorithm:

  - index all given inputs, for each input `PATH`:
    - load it;
    - check this reqres satisfies given filters and skip it if it does not,
    - remember its location (or, for some types of files, its contents) for future use;
  - start listering on given host and port for:
    - replay requests on `GET /web/<selector>/<url>`;
  - for each replay request:
    - if `selector` ends with `*`:
      - interpret `selector` as a time interval;
      - interpret `url` as glob pattern;
      - show a page with all indexed visits to URLs matching the pattern in the interval;
    - otherwise:
      - if `url` has indexed visits, respond with data most closely matching the given `selector`;
      - otherwise:
        - if `url` contains `*`, interpret it as a glob pattern;
        - otherwise, generate a glob pattern by chopping away less important parts of the current `url`;
        - show a `Not Found` page with a list of similar URLs and visits matching the pattern.

  The end.

  - options:
    - `-q, --quiet`
    : don't don't print end-of-filtering warnings, don't print optional informational messages, and don't log HTTP requests to stderr

  - caching:
    - `--max-memory INT`
    : the caches, all taken together, must not take more than this much memory in MiB; default: `1024`;
      making this larger improves performance;
      the actual maximum whole-program memory consumption is `O(<size of the largest reqres> + <numer of indexed files> + <sum of lengths of all their --output paths> + <--max-memory>)`

  - error handling:
    - `--errors {fail,skip,ignore}`
    : when an error occurs:
      - `fail`: report failure and stop the execution; default
      - `skip`: report failure but skip the reqres that produced it from the output and continue
      - `ignore`: `skip`, but don't report the failure

  - path ordering:
    - `--paths-given-order`
    : `argv` and `--stdin0` `PATH`s are processed in the order they are given; default
    - `--paths-sorted`
    : `argv` and `--stdin0` `PATH`s are processed in lexicographic order
    - `--paths-reversed`
    : `argv` and `--stdin0` `PATH`s are processed in reverse lexicographic order
    - `--walk-fs-order`
    : recursive file system walk is done in the order `readdir(2)` gives results
    - `--walk-sorted`
    : recursive file system walk is done in lexicographic order; default
    - `--walk-reversed`
    : recursive file system walk is done in reverse lexicographic order

  - input loading:
    - `--load-any`
    : for each given input `PATH`, decide which loader to use based on its file extension; default
    - `--load-wrr`
    : load all inputs using the single-`WRR` per-file loader
    - `--load-wrrb`
    : load all inputs using the `WRR` bundle loader, this will load separate `WRR` files as single-`WRR` bundles too
    - `--load-mitmproxy`
    : load inputs using the `mitmproxy` dump loader
    - `--stdin0`
    : read zero-terminated `PATH`s from stdin, these will be processed after `PATH`s specified as command-line arguments
    - `PATH`
    : inputs, can be a mix of files and directories (which will be traversed recursively)

  - `MIME` type sniffing; this controls the use of the [`mimesniff` algorithm](https://mimesniff.spec.whatwg.org/); for this sub-command higher values make the `scrub` function (which see) censor out more things when `-unknown`, `-styles`, or `-scripts` options are set; in particular, at the moment, with `--sniff-paranoid` and `-scripts` most plain text files will be censored out as potential `JavaScript`:
    - `--sniff-default`
    : run `mimesniff` when the spec says it should be run; i.e. trust `Content-Type` `HTTP` headers most of the time; default
    - `--sniff-force`
    : run `mimesniff` regardless of what `Content-Type`  and `X-Content-Type-Options` `HTTP` headers say; i.e. for each reqres, run `mimesniff` algorithm on the `Content-Type` `HTTP` header and the actual contents of `(request|response).body` (depending on the first argument of `scrub`) to determine what the body actually contains, then interpret the data as intersection of what `Content-Type` and `mimesniff` claim it to be; e.g. if `Content-Type` says `text/plain` but `mimesniff` says `text/plain or text/javascript`, interpret it as `text/plain`
    - `--sniff-paranoid`
    : do what `--sniff-force` does, but interpret the results in the most paranoid way possible; e.g. if `Content-Type` says `text/plain` but `mimesniff` says `text/plain or text/javascript`, interpret it as `text/plain or text/javascript`; which, for instance, will then make `scrub` with `-scripts` censor it out, since it can be interpreted as a script

  - default input filters:
    - `--ignore-bad-inputs`
    : initialize input filters to `--status-re ".200C"`; default
    - `--index-all-inputs`
    : do not set any input filters by default; note, however, that this sub-command expects input filters to be at least as restrictive as the above default and is likely to produce broken outputs if those filters are unset; use this option at your own risk

  - `HTTP` server options:
    - `--host HOST`
    : listen on what host/IP; default: `127.0.0.1`
    - `--port PORT`
    : listen on what port; default: `3210`
    - `--debug-bottle`
    : run with `bottle`'s debugging enabled

  - expression evaluation:
    - `-e EXPR, --expr EXPR`
    : an expression to compute, same expression format and semantics as `hoardy-web get --expr` (which see); can be specified multiple times; the default depends on `--remap-*` options below

  - rendering of `--expr` values:
    - `--not-separated`
    : render `--expr` values into outputs without separating them with anything, just concatenate them
    - `--lf-separated`
    : render `--expr` values into outputs separated with `\n` (LF) newline characters; default
    - `--zero-separated`
    : render `--expr` values into outputs separated with `\0` (NUL) bytes

  - default value of `--expr`:
    - `--raw-qbody`
    : set the default value of `--expr` to `request.body|eb`; i.e. produce the raw request body
    - `--raw-sbody, --no-remap`
    : set the default value of `--expr` to `response.body|eb`; i.e. produce the raw response body
    - `--remap-id`
    : set the default value of `--expr` to `response.body|eb|scrub response +all_refs`; i.e. `scrub` response body as follows: remap all URLs with an identity function (which, as a whole, is NOT an identity function, it will transform all relative URLs into absolute ones), censor out all dynamic content (e.g. `JavaScript`); results will NOT be self-contained
    - `--remap-void`
    : set the default value of `--expr` to `response.body|eb|scrub response -all_refs`; i.e. `scrub` response body as follows: remap all URLs into `javascript:void(0)` and empty `data:` URLs, censor out all dynamic content; results will be self-contained
    - `--remap-semi`
    : set the default value of `--expr` to `response.body|eb|scrub response *jumps,/actions,/reqs`; i.e. `scrub` response body as follows: keeps all jump links pointing to unarchived URLs as-is, remap all other links and references to their replay URLs, censor out all dynamic content; results will be self-contained
    - `--remap-all`
    : set the default value of `--expr` to `response.body|eb|scrub response &all_refs`; i.e. `scrub` response body as follows: remap all links and references to their replay URLs, even when they are not available in the index, censor out all dynamic content; results will be self-contained; default

  - buckets:
    - `--default-bucket NAME, --default-profile NAME`
    : default bucket name to use when a client does not specify any; default: `default`
    - `--ignore-buckets, --ignore-profiles`
    : ignore bucket names specified by clients and always use `--default-bucket` instead

  - file output options:
    - `--compress`
    : compress new archivals before dumping them to disk; default
    - `--no-compress, --uncompressed`
    : dump new archivals to disk without compression

  - file outputs:
    - `-t ARCHIVE_DESTINATION, --to ARCHIVE_DESTINATION, --archive-to ARCHIVE_DESTINATION`
    : archiving destination directory; if left unset, which is the default, then archiving server support will be disabled
    - `-i, --implicit`
    : prepend `ARCHIVE_DESTINATION` to the list of input `PATH`s
    - `-o OUTPUT_FORMAT, --output OUTPUT_FORMAT`
    : format describing generated output paths, an alias name or "format:" followed by a custom pythonic  string; same expression format as `hoardy-web organize --output` (which see); default: `default`

  - new `--output`s printing:
    - `--no-print`
    : don't print anything; default
    - `-l, --lf-terminated`
    : print absolute paths of newly produced or replaced files terminated with `\n` (LF) newline characters
    - `-z, --zero-terminated`
    : print absolute paths of newly produced or replaced files terminated with `\0` (NUL) bytes

  - replay options:
    - `--no-replay`
    : disable replay functionality, makes this into an archive-only server, like `hoardy-web-sas` is
    - `--latest`
    : index and replay only the latest visit for each URL; if `--to` is set, archiving a new visit for a URL will replace the indexed and replayable version
    - `--all`
    : index and replay all visits to all available URLs; if `--to` is given, archiving a new visit for a URL will update the index and make the new visit available for replay; default



  - Pretty-print all reqres in `../simple_server/pwebarc-dump` using an abridged (for ease of reading and rendering) verbose textual representation:
    ```
    hoardy-web pprint ../simple_server/pwebarc-dump
    ```

  - Pipe raw response body from a given `WRR` file to stdout:
    ```
    hoardy-web get ../simple_server/pwebarc-dump/path/to/file.wrr
    ```

  - Pipe response body scrubbed of dynamic content from a given `WRR` file to stdout:
    ```
    hoardy-web get -e "response.body|eb|scrub response defaults" ../simple_server/pwebarc-dump/path/to/file.wrr
    ```

  - Get first 2 bytes (4 characters) of a hex digest of sha256 hash computed on the URL without the fragment/hash part:
    ```
    hoardy-web get -e "net_url|to_ascii|sha256|take_prefix 2|to_hex" ../simple_server/pwebarc-dump/path/to/file.wrr
    ```

  - Pipe response body from a given `WRR` file to stdout, but less efficiently, by generating a temporary file and giving it to `cat`:
    ```
    hoardy-web run cat ../simple_server/pwebarc-dump/path/to/file.wrr
    ```

    Thus `hoardy-web run` can be used to do almost anything you want, e.g.

    ```
    hoardy-web run less ../simple_server/pwebarc-dump/path/to/file.wrr
    ```

    ```
    hoardy-web run -- sort -R ../simple_server/pwebarc-dump/path/to/file.wrr
    ```

    ```
    hoardy-web run -n 2 -- diff -u ../simple_server/pwebarc-dump/path/to/file-v1.wrr ../simple_server/pwebarc-dump/path/to/file-v2.wrr
    ```

  - List paths of all `WRR` files from `../simple_server/pwebarc-dump` that contain complete `200 OK` responses with `text/html` bodies larger than 1K:
    ```
    hoardy-web find --status-re .200C --response-mime text/html --and "response.body|len|> 1024" ../simple_server/pwebarc-dump
    ```

  - Rename all `WRR` files in `../simple_server/pwebarc-dump/default` according to their metadata using `--output default` (see the `hoardy-web organize` section for its definition, the `default` format is designed to be human-readable while causing almost no collisions, thus making `num` substitution parameter to almost always stay equal to `0`, making things nice and deterministic):
    ```
    hoardy-web organize ../simple_server/pwebarc-dump/default
    ```

    alternatively, just show what would be done

    ```
    hoardy-web organize --dry-run ../simple_server/pwebarc-dump/default
    ```



  - Pretty-print all reqres in `../simple_server/pwebarc-dump` by dumping their whole structure into an abridged Pythonic Object Representation (repr):
    ```
    hoardy-web stream --expr . ../simple_server/pwebarc-dump
    ```

    ```
    hoardy-web stream -e . ../simple_server/pwebarc-dump
    ```

  - Pretty-print all reqres in `../simple_server/pwebarc-dump` using the unabridged verbose textual representation:
    ```
    hoardy-web pprint --unabridged ../simple_server/pwebarc-dump
    ```

    ```
    hoardy-web pprint -u ../simple_server/pwebarc-dump
    ```

  - Pretty-print all reqres in `../simple_server/pwebarc-dump` by dumping their whole structure into the unabridged Pythonic Object Representation (repr) format:
    ```
    hoardy-web stream --unabridged --expr . ../simple_server/pwebarc-dump
    ```

    ```
    hoardy-web stream -ue . ../simple_server/pwebarc-dump
    ```

  - Produce a `JSON` list of `[<file path>, <time it finished loading in seconds since UNIX epoch>, <URL>]` tuples (one per reqres) and pipe it into `jq` for indented and colored output:
    ```
    hoardy-web stream --format=json -ue fs_path -e finished_at -e request.url ../simple_server/pwebarc-dump | jq .
    ```

  - Similarly, but produce a `CBOR` output:
    ```
    hoardy-web stream --format=cbor -ue fs_path -e finished_at -e request.url ../simple_server/pwebarc-dump | less
    ```

  - Concatenate all response bodies of all the requests in `../simple_server/pwebarc-dump`:
    ```
    hoardy-web stream --format=raw --not-terminated -ue "response.body|eb" ../simple_server/pwebarc-dump | less
    ```

  - Print all unique visited URLs, one per line:
    ```
    hoardy-web stream --format=raw --lf-terminated -ue request.url ../simple_server/pwebarc-dump | sort | uniq
    ```

  - Same idea, but using NUL bytes, with some post-processing, and two URLs per line:
    ```
    hoardy-web stream --format=raw --zero-terminated -ue request.url ../simple_server/pwebarc-dump | sort -z | uniq -z | xargs -0 -n2 echo
    ```



  Trying to use response bodies produced by `hoardy-web stream --format=json` is likely to result garbled data as `JSON` can't represent raw sequences of bytes, thus binary data will have to be encoded into UNICODE using replacement characters:

  ```
  hoardy-web stream --format=json -ue . ../simple_server/pwebarc-dump/path/to/file.wrr | jq .
  ```

  The most generic solution to this is to use `--format=cbor` instead, which would produce a verbose `CBOR` representation equivalent to the one used by `--format=json` but with binary data preserved as-is:

  ```
  hoardy-web stream --format=cbor -ue . ../simple_server/pwebarc-dump/path/to/file.wrr | less
  ```

  Or you could just dump raw response bodies separately:

  ```
  hoardy-web stream --format=raw -ue response.body ../simple_server/pwebarc-dump/path/to/file.wrr | less
  ```

  ```
  hoardy-web get ../simple_server/pwebarc-dump/path/to/file.wrr | less
  ```



  Sanity check and test `hoardy-web` command-line interface.



  - Run tests on each of given WRR bundles:

    ```
    ./test-cli.sh ~/Downloads/Hoardy-Web-export-*.wrrb
    ```

  - Run tests on all WRR files in a given directory:

    ```
    ./test-cli.sh ~/hoardy-web/latest/archiveofourown.org
    ```

  - Run tests on a random subset of WRR files in a given directory:

    ```
    ./test-cli.sh --subset 100 ~/hoardy-web/raw
    ```

  - Run tests on each of given WRR bundles, except run long tests on a small subset of each:

    ```
    ./test-cli.sh --short 16 ~/Downloads/Hoardy-Web-export-*.wrrb
    ```

  - Make `--stdin0` input and test on it, as if it was a WRR bundle:

    ```
    hoardy-web find -z ~/hoardy-web/latest/archiveofourown.org ~/hoardy-web/latest/example.org > ./bunch.wrrtest
    ./test-cli.sh ./bunch.wrrtest
    ```


