# packages

pkgId: f46ec1a97119ab380fff3fa0c31fb28d31881d26f4084fd2dca5e2df63a45c1d
name: python-compliance-trestle
arch: src
version: 3.6.0
epoch: 0
release: 1
summary: Tools to manage & autogenerate python objects representing the OSCAL layers/models
url: https://oscal-compass.github.io/compliance-trestle
rpm_license: Apache Software License v2
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-compliance-trestle-3.6.0-1.src.rpm
checksum_type: sha256
description:
  ![[OS Compatibility](#prerequisites)](https://img.shields.io/badge/platform-osx%20%7C%20linux%20%7C%20windows-orange.svg)
  ![PyPI - Python Version](https://img.shields.io/pypi/pyversions/compliance-trestle)
  ![[Pre-commit](https://github.com/pre-commit/pre-commit)](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)
  ![[Code Coverage](https://sonarcloud.io/dashboard?id=compliance-trestle)](https://sonarcloud.io/api/project_badges/measure?project=compliance-trestle&metric=coverage)
  ![[Quality gate](https://sonarcloud.io/dashboard?id=compliance-trestle)](https://sonarcloud.io/api/project_badges/measure?project=compliance-trestle&metric=alert_status)
  ![[Pypi](https://pypi.org/project/compliance-trestle/)](https://img.shields.io/pypi/dm/compliance-trestle)
  ![GitHub Actions status](https://github.com/oscal-compass/compliance-trestle/actions/workflows/python-test.yml/badge.svg?branch=develop)
  ![OpenSSF Best Practices](https://www.bestpractices.dev/projects/9408/badge)

  Trestle is an ensemble of tools that enable the creation, validation, and governance of documentation artifacts for compliance needs. It leverages NIST's [OSCAL](https://pages.nist.gov/OSCAL/) as a standard data format for interchange between tools and people, and provides an opinionated approach to OSCAL adoption.

  Trestle is designed to operate as a CICD pipeline running on top of compliance artifacts in `git`, to provide transparency for the state of compliance across multiple stakeholders in an environment friendly to developers. Trestle passes the generated artifacts onto tools that orchestrate the enforcement, measurement, and reporting of compliance.

  It also provides tooling to manage OSCAL documents in a more human-friendly manner. By splitting large OSCAL data structures into smaller and easier to edit sub-structures, creation and maintenance of these artifacts can follow normal `git` workflows including peer review via pull request, versioning, releases/tagging.

  Trestle provides three separate but related functions in the compliance space:

  - Manage OSCAL documents to allow editing and manipulation while making sure the schemas are enforced
  - Transform documents from other formats to OSCAL
  - Provide support and governance to author compliance content as markdown and drawio.

  Trestle provides tooling to help orchestrate the compliance process across a number of dimensions:

  - Help manage OSCAL documents in a more human-friendly manner by expanding the large OSCAL data structures into smaller and easier to edit sub-structures while making sure the schemas are enforced.
  - Transform documents from other formats to OSCAL
  - Provide governance for markdown documents and enforce consistency of format and content based on specified templates
  - Tooling manage authoring and governance of markdown and drawio files within a repository.
  - Support within trestle to streamline management within a managed git environment.
  - An underlying object model that supports developers interacting with OSCAL artifacts.



  Compliance suffers from being a complex topic that is hard to articulate simply. It involves complete and accurate execution of multiple procedures across many disciplines (e.g. IT, HR, management) with periodic verification and audit of those procedures against controls.

  While it is possible to manage the description of controls and how an organisation implements them in ad hoc ways with general tools (spreadsheets, documents), this is hard to maintain for multiple accreditations and, in the IT domain at least, creates a barrier between the compliance efforts and the people doing daily work (DevOps staff).

  Trestle aims to reduce or remove this barrier by bringing the maintenance of control descriptions into the DevOps domain. The goal is to have changes to the system (for example, updates to configuration management) easily related to the controls impacted, and to enable modification of those controls as required in concert with the system changes.

  Trestle implicitly provides a core opinionated workflow driven by its pipeline to allow standardized interlocks with other compliance tooling platforms.



  Compliance activities at scale, whether size of estate or number of accreditations, require automation to be successful and repeatable. OSCAL as a standard allows teams to bridge between the "Governance" layer and operational tools.

  By building human managed artifacts into OSCAL, Trestle is not only able to validate the integrity of the artifacts that people generate - it also enables reuse and sharing of artifacts, and furthermore can provide suitable input into tools that automate operational compliance.



  `trestle` implicitly supports all OSCAL schemas for use within the object model. The development roadmap for `trestle` includes adding workflow around specific elements / objects that is opinionated.



  OSCAL supports `xml`, `json` and `yaml` with their [metaschema](https://github.com/usnistgov/metaschema) tooling. Trestle
  natively supports only `json` and `yaml` formats at this time.

  Future roadmap anticipates that support for xml [import](https://github.com/oscal-compass/compliance-trestle/issues/177) and [upstream references](https://github.com/oscal-compass/compliance-trestle/issues/178) will be enabled. However, it is expected
  that full support will remain only for `json` and  `yaml`.

  Users needing to import XML OSCAL artifacts are recommended to look at NIST's XML to json conversion page [here](https://github.com/usnistgov/OSCAL/blob/main/build/README.md#converters).



  Trestle runs on almost all Python platforms (e.g. Linux, Mac, Windows), is available on PyPi and can be easily installed via pip. It is under active development and new releases are made available regularly.\
  To install run: `pip install compliance-trestle`\
  See [Install trestle in a python virtual environment](https://oscal-compass.github.io/compliance-trestle/python_trestle_setup/) for the full installation guide.



  Complete documentation, tutorials, and background on compliance can be found [here](https://oscal-compass.github.io/compliance-trestle).



  A trestle-based agile authoring repository setup tool, documentation and tutorial can be found [here](https://github.com/oscal-compass/compliance-trestle-agile-authoring).

  Agile authoring comprises the following beneficial features:

  - based on OSCAL documents behind-the-scenes
  - employs GIT for document control and access
  - exposes text (markdown) and spread sheets (csv) to ease management of compliance artifacts
  - implements compliance digitization for improved audit readiness and cost effectiveness



  A collection of demos utilizing trestle can be found in the related project [compliance-trestle-demos](https://github.com/oscal-compass/compliance-trestle-demos).





  - supports NIST OSCAL 1.1.2 as well as previous versions



  - supports NIST OSCAL 1.0.4 as well as previous versions



  Please refer to the community [README](https://github.com/oscal-compass/community/blob/main/README.md) for communication details.



  Our project welcomes external contributions. Please consult [contributing](https://oscal-compass.github.io/compliance-trestle/contributing/mkdocs_contributing/) to get started.



  Participation in the OSCAL Compass community is governed by the [Code of Conduct](https://github.com/oscal-compass/community/blob/main/CODE_OF_CONDUCT.md).



  If you would like to see the detailed LICENSE click [here](LICENSE).
  Consult [contributors](https://github.com/oscal-compass/compliance-trestle/graphs/contributors) for a list of authors and [maintainers](MAINTAINERS.md) for the core team.

  ```text













  ```

  ______________________________________________________________________

  We are a Cloud Native Computing Foundation sandbox project.

  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="https://www.cncf.io/wp-content/uploads/2022/07/cncf-white-logo.svg">
    <img src="https://www.cncf.io/wp-content/uploads/2022/07/cncf-color-bg.svg" width=300 />
  </picture>

  The Linux Foundation® (TLF) has registered trademarks and uses trademarks. For a list of TLF trademarks, see [Trademark Usage](https://www.linuxfoundation.org/legal/trademark-usage)".

  *Trestle was originally created by IBM.*


pkgId: c911c0070d76ae54d1e6f0668a9c6e1d6593b3ad1763211fc27c2b7f7337ebdb
name: python-compliant-social-app-django
arch: src
version: 5.4.1
epoch: 0
release: 1
summary: Python Social Authentication, Django integration, Google compliant fork
url: https://github.com/RightlyGroup/compliant-social-app-django
rpm_license: BSD
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-compliant-social-app-django-5.4.1-1.src.rpm
checksum_type: sha256
description:
  This is a Google compliant fork of the [Django](https://www.djangoproject.com/) component of the
  [python-social-auth ecosystem](https://github.com/python-social-auth/social-core),
  it implements the needed functionality to integrate
  [social-auth-core](https://github.com/python-social-auth/social-core)
  in a Django based project.



  Python Social Auth is an easy to setup social authentication/registration
  mechanism with support for several frameworks and auth providers.



  This project will focus on the currently supported Django releases as
  stated on the [Django Project Supported Versions table](https://www.djangoproject.com/download/#supported-versions).

  Backward compatibility with unsupported versions won't be enforced.



  Compliant Social App Django provides three key features required to comply with Google's standards for applications that
  ask for restricted scopes such as the ability to read gmail inboxes:

  1. Storing the access and refresh tokens encrypted in the database.
  2. The encryption of those tokens being backed by KMS.
  3. Audit logs that register the access to and revocation of those same tokens.

  An Audit Logger object must be provided to the app, and must inherit from the abstract base audit logger provided. We
  also provide an alternative Google backend that uses the audit logger.

  KMS must be set up independently, and you must specify a KMS_FIELD_KEY in settings; typically the alias of your KMS key.

  If moving to this package from the main social-app-django package the db migrations will also handle the migration of
  tokens, with encryption, into their appropriate fields.

  Core project documentation is available at https://python-social-auth.readthedocs.org/.



  ```shell
  $ pip install compliant-social-app-django
  ```



  This project follows [Semantic Versioning 2.0.0](http://semver.org/spec/v2.0.0.html).

  This fork's versioning will match the major version of the original package version upon which it is based



  To release a new version use these steps:

  1. Increment the version number in `__init.py__`
  2. Update the `CHANGELOG.md` with description of the changes in the release (and commit)
  3. Remove the `build` and `dist` directories completely
     ```bash
     rm -rf build dist
     ```
  4. Build the source distribution:
     ```bash
     python setup.py sdist
     ```
  5. Push the distribution. You'll need to configure your `~/.pypirc` file to
     authenticate with Pypi using an API token before running this.
     ```bash
     python -m twine upload --repository compliant-social-app-django dist/*
     ```
  6. Tag the release and push the tag



  This project follows the BSD license. See the [LICENSE](LICENSE) for details.


pkgId: 826a0305fb9a93e9bbeecddf689b1a6ca89a9b4bfddc8b33616b34276cd943de
name: python-component-contribution
arch: src
version: 0.6.0
epoch: 0
release: 1
summary: A method for estimating the standard reaction Gibbs energy of biochemical reactions
url: https://pypi.org/project/component-contribution/
rpm_license: MIT License
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-component-contribution-0.6.0-1.src.rpm
checksum_type: sha256
description:
  [![PyPI version](https://badge.fury.io/py/component-contribution.svg)](https://badge.fury.io/py/component-contribution)
  [![Anaconda-Server Badge](https://anaconda.org/conda-forge/component-contribution/badges/version.svg)](https://anaconda.org/conda-forge/component-contribution)
  [![Python version](https://img.shields.io/pypi/pyversions/component-contribution.svg)](https://www.python.org/downloads)
  [![MIT license](https://img.shields.io/pypi/l/component-contribution.svg)](https://mit-license.org/)

  [![pipeline status](https://gitlab.com/elad.noor/component-contribution/badges/develop/pipeline.svg)](https://gitlab.com/elad.noor/component-contribution/commits/develop)
  [![codecov](https://codecov.io/gl/equilibrator/component-contribution/branch/develop/graph/badge.svg?token=OxxaCqgaLs)](https://codecov.io/gl/equilibrator/component-contribution)
  [![Join our Google group](https://img.shields.io/badge/google_group-equilibrator_users-blue)](https://groups.google.com/g/equilibrator-users)
  [![Documentation Status](https://readthedocs.org/projects/equilibrator/badge/?version=latest)](https://equilibrator.readthedocs.io/en/latest/?badge=latest)

  A method for estimating the standard reaction Gibbs energy of biochemical reactions.



  For more information on the method behind component-contribution, please view our open
  access paper:

  Noor E, Haraldsdóttir HS, Milo R, Fleming RMT (2013)
  [Consistent Estimation of Gibbs Energy Using Component Contributions](http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003098),
  PLoS Comput Biol 9:e1003098, DOI: 10.1371/journal.pcbi.1003098

  Please, cite this paper if you publish work that uses `component-contribution`.



  * `pip install component-contribution`



  * Python 3.9+
  * PyPI dependencies for prediction:
    - equilibrator-cache
    - numpy
    - scipy
    - pandas
    - pint
    - path
    - periodictable
    - uncertainties
  * PyPI dependencies for training a new model:
    - openbabel
    - equilibrator-assets



  * [Training data for the component contribution method](https://zenodo.org/record/3978440)
  * [Chemical group definitions for the component-contribution method](https://zenodo.org/record/4010930)


pkgId: a6789865e0c291627004352fc6a007f466cf64a0a562a6f1135e000763f0ea9e
name: python-componentes
arch: src
version: 0.2.6
epoch: 0
release: 1
summary: Componetes de uso de teste
url: https://pypi.org/project/componentes/
rpm_license: UNKNOWN
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-componentes-0.2.6-1.src.rpm
checksum_type: sha256
description:
  ___


  <details>
    <summary>Criando ou atualizando componente</summary>










      - Abra o terminal dentro do diretorio relacionado
      - Utilizar comando `poetry new nome_projeto`
      - Acesse a pasta do projeto criado (usando cd)
      - registre o token gerado no projeto com o comando `poetry config pypi-token.pypi token_gerado`
      - Alterando informações no arquivo `pyproject.toml`
          -- Adicione o nome, descrição, autor(ficara visivel no Pypi) e informações que desejar
          -- Adicione as dependencias usando no terminal `poetry add nome_pacote` ou adicione dentro de `pyproject.toml` abaixo da linha tool.poetry.dependencies
          -- Altere a versão caso seja uma atualização
      - No arquivo _init_.py altere a versão caso seja uma atualização
      - Na pasta do projeto: Adicione os arquivos necessários
      - No terminal: digite `poetry build` para compilar e gerar o diretorio dist com os arquivos zipados
      - No terminal: digite `poetry publish` e o pacote será criado ou atualizado
    Obs: É importante alterar a versão no `pyproject.toml` sempre que for alterar o projeto
  </details>

  ___


pkgId: 5af3aa9e0f231fac9aad2ef1f25c5e0304a312c486372c3014ae58783cd4f34f
name: python-componentsDjangoType
arch: src
version: 0.0.4
epoch: 0
release: 1
summary: A Django app for creating HTML components
url: https://github.com/jose-CR/componentsDjangoType
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-componentsDjangoType-0.0.4-1.src.rpm
checksum_type: sha256
description:
  <div style="center">
    <img src="https://img.shields.io/badge/django-%23092E20.svg?style=for-the-badge&logo=django&logoColor=white" alt="Django">
    <img src="https://img.shields.io/badge/python-3670A0?style=for-the-badge&logo=python&logoColor=ffdd54" alt="Python">
    <img src="https://img.shields.io/badge/pypi-%23ececec.svg?style=for-the-badge&logo=pypi&logoColor=1f73b7" alt="PyPi">
  </div>



  Este comando de Django te permite crear un componente HTML dentro del directorio `templates` o `static` de una aplicación específica.



  1. Instala Django si no lo has hecho ya: `pip install django`
  2. Luego instala el paquete pip `pip install componentsDjangoType`
  3. Agrega `componentsDjangoType` a la lista de aplicaciones instaladas (`INSTALLED_APPS`) en el archivo `settings.py` de tu proyecto de Django.



  1. Ejecuta el comando en tu terminal: `python manage.py create_html_component <nombre_de_aplicación> <ruta_del_archivo> --type=<template|static>`
  2. Reemplaza `<nombre_de_aplicación>` con el nombre de la aplicación donde deseas crear el componente.
  3. Reemplaza `<ruta_del_archivo>` con la ruta donde deseas crear el componente dentro del directorio `templates` o `static` de la aplicación.
  4. La bandera `--type` especifica si deseas crear un componente en el directorio `templates` o `static`. Por defecto, se establece en `template`.



  Para crear un componente HTML en el directorio `templates` de la aplicación `myapp` con la ruta `components/my_component.html`, ejecuta:
  ```
  python manage.py myapp components/my_component.html --type static
  ```



  - El comando creará los directorios necesarios si no existen.
  - Si el archivo del componente ya existe, el comando mostrará un mensaje de advertencia.
  - El comando creará un contenido predeterminado para el componente según su extensión de archivo:
      - `.html`: `<div>Este es el contenido del componente.</div>`
      - `.js`: `console.log("Este es el contenido del componente.");`
      - `.css`: `/* Este es el contenido del componente */`
  - Si buscas el paquete de pip para descargarlo, aquí está: [Pypi](https://pypi.org/project/componentsDjangoType/)



  Este proyecto está licenciado bajo la Licencia MIT. Consulta el archivo [LICENSE](LICENSE) para obtener más detalles.


pkgId: 5aeee6c1b85c162d88fd294b6c851751217427a277f6b8d5a07b7360fcd3b39b
name: python-componentsDjangoType
arch: src
version: 2.0.14
epoch: 0
release: 1
summary: Comandos para crear archivos html, css y js
url: https://github.com/jose-CR/componentsDjangoType
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-componentsDjangoType-2.0.14-1.src.rpm
checksum_type: sha256
description:
  <div style="center">
    <img src="https://img.shields.io/badge/django-%23092E20.svg?style=for-the-badge&logo=django&logoColor=white" alt="Django">
    <img src="https://img.shields.io/badge/python-3670A0?style=for-the-badge&logo=python&logoColor=ffdd54" alt="Python">
    <img src="https://img.shields.io/badge/pypi-%23ececec.svg?style=for-the-badge&logo=pypi&logoColor=1f73b7" alt="PyPi">
  </div>



  Este comando de Django te permite crear un componente HTML dentro del directorio `templates` o `static` de una aplicación específica.



  1. Instala Django si no lo has hecho ya: `pip install django`
  2. Luego instala el paquete pip `pip install componentsDjangoType`
  3. Agrega `componentsDjangoType` a la lista de aplicaciones instaladas (`INSTALLED_APPS`) en el archivo `settings.py` de tu proyecto de Django.





  1. Ejecuta el comando en tu terminal: `python manage.py myapp createcomponent <carpeta/archivo> --type=<template|static>`
  2. Reemplaza `<nombre_de_aplicación>`con el nombre de la aplicación donde deseas crear el componente.
  3. Reemplaza `<ruta_del_archivo>` con la ruta donde deseas crear el componente dentro del directorio templates o static de la aplicación.
  4. La bandera `--type` especifica si deseas crear un componente en el directorio templates o static. Por defecto, se establece en template.



  1. Ejecuta el comando en tu terminal: `python manage.py createApp`
  2. En este punto te mostrara un mensaje para poner el nombre de la app principal de tu apicacion
  la App principal es la que tiene el archivo settings.py de tu aplicacion



  Para crear un componente HTML en el directorio `templates` de la aplicación `myapp` con la ruta `components/my_component.html`, ejecuta:
  ```
  python manage.py myapp components/my_component.html --type static
  ```



  - El comando creará los directorios necesarios si no existen.
  - Si el archivo del componente ya existe, el comando mostrará un mensaje de advertencia.
  - El comando creará un contenido predeterminado para el componente según su extensión de archivo:
      - `.html`: `<div>Este es el contenido del componente.</div>`
      - `.js`: `console.log("Este es el contenido del componente.");`
      - `.css`: `/* Este es el contenido del componente */`
  - Si buscas el paquete de pip para descargarlo, aquí está: [Pypi](https://pypi.org/project/componentsDjangoType/)



  Este proyecto está licenciado bajo la Licencia MIT. Consulta el archivo [LICENSE](LICENSE) para obtener más detalles.


pkgId: bba2f73b29693256ee6571d97d1a7dc59ba130ba0d45391a2e132643bb968415
name: python-compose
arch: src
version: 1.5.0
epoch: 0
release: 1
summary: The classic ``compose``, with all the Pythonic features.
url: https://github.com/mentalisttraceur/python-compose
rpm_license: 0BSD
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-compose-1.5.0-1.src.rpm
checksum_type: sha256
description:
  The classic ``compose``, with all the Pythonic features.
  This ``compose`` follows the lead of ``functools.partial``
  and returns callable ``compose`` objects which:
  * have a regular and unambiguous ``repr``,
  * retain correct signature introspection,
  * allow introspection of the composed callables,
  * can be type-checked,
  * can be weakly referenced,
  * can have attributes,
  * will merge when nested, and
  * can be pickled (if all composed callables can be pickled).
  For ``async``/``await`` support, different variants of
  ``compose`` are included.


pkgId: 057f741866e546fb1326deb07d70b6fa9bf3ac0727bb6fd5d0ae2ca4c444ef42
name: python-compose-flow
arch: src
version: 4.0.0
epoch: 0
release: 1
summary: codified workflows for docker compose
url: https://github.com/openslate/compose-flow
rpm_license: UNKNOWN
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-compose-flow-4.0.0-1.src.rpm
checksum_type: sha256
description:
  This utility is built on top of [Docker Compose](https://docs.docker.com/compose/) and [Swarm Mode](https://docs.docker.com/engine/swarm/).  It establishes workflow conventions that are easily shared between team members -- and butlers -- who need to manage and deploy services, including:

  - managing [Stacks](https://docs.docker.com/get-started/part5/#prerequisites) across multiple Swarms (e.g. separate dev and prod Swarms)
  - connecting to and working with service containers
  - building and publishing images
  - sharing service configuration between team members




  ```
  pip install compose-flow
  ```




  Create the file `~/.compose/config.yml` with the following sections.

  - Build
  - Remotes
  - Options (for local dev)

  ex:
  ```yaml
  build:
    image_prefix: registry.osslabs.net
  remotes:
    test:
      backend: rancher
      rancher:
        project: Ops
        cluster: prod
    dev:
      backend: rancher
    prod:
      backend: rancher
  options:
    local:
      dirty_working_copy_okay: True
  ```

  With this in place you're ready to go onto your project setup.




  This is the most basic file to get started.

  Place this at `compose/compose-flow.yml` in your project directory:

  ```
  profiles:
    local:
      - docker-compose.yml
  ```

  Alongside it, place the file `compose/docker-compose.yml`:

  ```
  version: '3.7'
  services:
    app:
      build: ..
      image: ${DOCKER_IMAGE}
  ```

  For building, run: `compose-flow build`.

  For publishing: `compose-flow publish`.

  For deploying as configured above: `compose-flow -e local deploy`.

  More information at [docs/advanced.md](docs/advanced.md)


pkgId: 9cc4df9669301f94a51aaf7000602dc704f0ca8739eecc1ef0d855b5e2d369e4
name: python-compose-plantuml
arch: src
version: 0.2.1
epoch: 0
release: 1
summary: converts docker-compose into plantuml
url: http://github.com/funkwerk/compose_plantuml
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-compose-plantuml-0.2.1-1.src.rpm
checksum_type: sha256
description:
  Generate Plantuml graphs from docker-compose files
  Note that version 2 as well as the legacy version is supported.


pkgId: eeb0c9c4f2fb6f346705893dc058ec46a9177135846a6d24d7848e3861bf750d
name: python-compose-runner
arch: src
version: 0.4.6
epoch: 0
release: 1
summary: A package for running neurosynth-compose analyses
url: https://pypi.org/project/compose-runner/
rpm_license: BSD 3-Clause License
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-compose-runner-0.4.6-1.src.rpm
checksum_type: sha256
description:
  Python package to execute meta-analyses created using neurosynth compose and NiMARE
  as the meta-analysis execution engine.


pkgId: 3687fc7573696f3222a561c163bdab1dfd426d68a58692312c622cc3fe1bc883
name: python-compose-sdk
arch: src
version: 0.21.0
epoch: 0
release: 1
summary: please add a summary manually as the author left a blank one
url: https://pypi.org/project/compose-sdk/
rpm_license: LGPL-3.0-or-later
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-compose-sdk-0.21.0-1.src.rpm
checksum_type: sha256
description:
  Compose is a framework for building robust internal apps in just a couple minutes. Create useful apps without writing a single line of frontend code, API endpoints, authentication, validation, or other boilerplate. Just write your backend logic, while Compose abstracts the rest away.

  Learn more at [composehq.com](https://composehq.com).


pkgId: 75a20797c9004aaa8fe00e79f2498ecac59fb53a3d9ea0338854674ad3c3c0fd
name: python-composeml
arch: src
version: 0.10.1
epoch: 0
release: 1
summary: a framework for automated prediction engineering
url: https://pypi.org/project/composeml/
rpm_license: BSD 3-clause
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-composeml-0.10.1-1.src.rpm
checksum_type: sha256
description:
  <p align="center"><img width=50% src="https://raw.githubusercontent.com/alteryx/compose/main/docs/source/images/compose.png" alt="Compose" /></p>
  <p align="center"><i>"Build better training examples in a fraction of the time."</i></p>
  <p align="center">
      <a href="https://github.com/alteryx/compose/actions?query=workflow%3ATests" target="_blank">
          <img src="https://github.com/alteryx/compose/workflows/Tests/badge.svg" alt="Tests" />
      </a>
      <a href="https://codecov.io/gh/alteryx/compose">
          <img src="https://codecov.io/gh/alteryx/compose/branch/main/graph/badge.svg?token=mDz4ueTUEO"/>
      </a>
      <a href="https://compose.alteryx.com/en/stable/?badge=stable" target="_blank">
          <img src="https://readthedocs.com/projects/feature-labs-inc-compose/badge/?version=stable&token=5c3ace685cdb6e10eb67828a4dc74d09b20bb842980c8ee9eb4e9ed168d05b00"
              alt="ReadTheDocs" />
      </a>
      <a href="https://badge.fury.io/py/composeml" target="_blank">
          <img src="https://badge.fury.io/py/composeml.svg?maxAge=2592000" alt="PyPI Version" />
      </a>
      <a href="https://stackoverflow.com/questions/tagged/compose-ml" target="_blank">
          <img src="https://img.shields.io/badge/questions-on_stackoverflow-blue.svg?" alt="StackOverflow" />
      </a>
      <a href="https://pepy.tech/project/composeml" target="_blank">
          <img src="https://pepy.tech/badge/composeml/month" alt="PyPI Downloads" />
      </a>
  </p>
  <hr>

  [Compose](https://compose.alteryx.com) is a machine learning tool for automated prediction engineering. It allows you to structure prediction problems and generate labels for supervised learning. An end user defines an outcome of interest by writing a *labeling function*, then runs a search to automatically extract training examples from historical data. Its result is then provided to [Featuretools](https://docs.featuretools.com/) for automated feature engineering and subsequently to [EvalML](https://evalml.alteryx.com/) for automated machine learning. The workflow of an applied machine learning engineer then becomes:

  <br><p align="center"><img width=90% src="https://raw.githubusercontent.com/alteryx/compose/main/docs/source/images/workflow.png" alt="Compose" /></p><br>

  By automating the early stage of the machine learning pipeline, our end user can easily define a task and solve it. See the [documentation](https://compose.alteryx.com) for more information.


  Install with pip

  ```
  python -m pip install composeml
  ```

  or from the Conda-forge channel on [conda](https://anaconda.org/conda-forge/composeml):

  ```
  conda install -c conda-forge composeml
  ```



  **Update checker** - Receive automatic notifications of new Compose releases

  ```
  python -m pip install "composeml[update_checker]"
  ```


  > Will a customer spend more than 300 in the next hour of transactions?

  In this example, we automatically generate new training examples from a historical dataset of transactions.

  ```python
  import composeml as cp
  df = cp.demos.load_transactions()
  df = df[df.columns[:7]]
  df.head()
  ```

  <table border="0" class="dataframe">
    <thead>
      <tr style="text-align: right;">
        <th>transaction_id</th>
        <th>session_id</th>
        <th>transaction_time</th>
        <th>product_id</th>
        <th>amount</th>
        <th>customer_id</th>
        <th>device</th>
      </tr>
    </thead>
    <tbody>
      <tr>
        <td>298</td>
        <td>1</td>
        <td>2014-01-01 00:00:00</td>
        <td>5</td>
        <td>127.64</td>
        <td>2</td>
        <td>desktop</td>
      </tr>
      <tr>
        <td>10</td>
        <td>1</td>
        <td>2014-01-01 00:09:45</td>
        <td>5</td>
        <td>57.39</td>
        <td>2</td>
        <td>desktop</td>
      </tr>
      <tr>
        <td>495</td>
        <td>1</td>
        <td>2014-01-01 00:14:05</td>
        <td>5</td>
        <td>69.45</td>
        <td>2</td>
        <td>desktop</td>
      </tr>
      <tr>
        <td>460</td>
        <td>10</td>
        <td>2014-01-01 02:33:50</td>
        <td>5</td>
        <td>123.19</td>
        <td>2</td>
        <td>tablet</td>
      </tr>
      <tr>
        <td>302</td>
        <td>10</td>
        <td>2014-01-01 02:37:05</td>
        <td>5</td>
        <td>64.47</td>
        <td>2</td>
        <td>tablet</td>
      </tr>
    </tbody>
  </table>

  First, we represent the prediction problem with a labeling function and a label maker.

  ```python
  def total_spent(ds):
      return ds['amount'].sum()

  label_maker = cp.LabelMaker(
      target_dataframe_index="customer_id",
      time_index="transaction_time",
      labeling_function=total_spent,
      window_size="1h",
  )
  ```

  Then, we run a search to automatically generate the training examples.

  ```python
  label_times = label_maker.search(
      df.sort_values('transaction_time'),
      num_examples_per_instance=2,
      minimum_data='2014-01-01',
      drop_empty=False,
      verbose=False,
  )

  label_times = label_times.threshold(300)
  label_times.head()
  ```

  <table border="0" class="dataframe">
    <thead>
      <tr style="text-align: right;">
        <th>customer_id</th>
        <th>time</th>
        <th>total_spent</th>
      </tr>
    </thead>
    <tbody>
      <tr>
        <td>1</td>
        <td>2014-01-01 00:00:00</td>
        <td>True</td>
      </tr>
      <tr>
        <td>1</td>
        <td>2014-01-01 01:00:00</td>
        <td>True</td>
      </tr>
      <tr>
        <td>2</td>
        <td>2014-01-01 00:00:00</td>
        <td>False</td>
      </tr>
      <tr>
        <td>2</td>
        <td>2014-01-01 01:00:00</td>
        <td>False</td>
      </tr>
      <tr>
        <td>3</td>
        <td>2014-01-01 00:00:00</td>
        <td>False</td>
      </tr>
    </tbody>
  </table>

  We now have labels that are ready to use in [Featuretools](https://docs.featuretools.com/) to generate features.



  The Innovation Labs open source community is happy to provide support to users of Compose. Project support can be found in three places depending on the type of question:

  1. For usage questions, use [Stack Overflow](https://stackoverflow.com/questions/tagged/compose-ml) with the `composeml` tag.
  2. For bugs, issues, or feature requests start a Github [issue](https://github.com/alteryx/compose/issues/new).
  3. For discussion regarding development on the core library, use [Slack](https://join.slack.com/t/alteryx-oss/shared_invite/zt-182tyvuxv-NzIn6eiCEf8TBziuKp0bNA).
  4. For everything else, the core developers can be reached by email at open_source_support@alteryx.com


  Compose is built upon a newly defined part of the machine learning process — prediction engineering. If you use Compose, please consider citing this paper:
  James Max Kanter, Gillespie, Owen, Kalyan Veeramachaneni. [Label, Segment,Featurize: a cross domain framework for prediction engineering.](https://dai.lids.mit.edu/wp-content/uploads/2017/10/Pred_eng1.pdf) IEEE DSAA 2016.

  BibTeX entry:

  ```bibtex
  @inproceedings{kanter2016label,
    title={Label, segment, featurize: a cross domain framework for prediction engineering},
    author={Kanter, James Max and Gillespie, Owen and Veeramachaneni, Kalyan},
    booktitle={2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA)},
    pages={430--439},
    year={2016},
    organization={IEEE}
  }
  ```



  The open source development has been supported in part by DARPA's Data driven discovery of models program (D3M).



  **Compose** is an open source project maintained by [Alteryx](https://www.alteryx.com). We developed Compose to enable flexible definition of the machine learning task. To see the other open source projects we’re working on visit [Alteryx Open Source](https://www.alteryx.com/open-source). If building impactful data science pipelines is important to you or your business, please get in touch.

  <p align="center">
    <a href="https://www.alteryx.com/open-source">
      <img src="https://alteryx-oss-web-images.s3.amazonaws.com/OpenSource_Logo-01.png" alt="Alteryx Open Source" width="800"/>
    </a>
  </p>


pkgId: 78050b4f2dd7b42c9dd3051e48148d55c7cc9e3ea840508580e7a9415ede9a3a
name: python-composer
arch: src
version: 0.25.0
epoch: 0
release: 1
url: https://github.com/mosaicml/composer
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-composer-0.25.0-1.src.rpm
checksum_type: sha256
summary:
  Composer is a PyTorch library that enables you to train neural networks faster, at lower cost, and to higher accuracy.

description:
  <br />
  <p align="center">
      <a href="https://github.com/mosaicml/composer#gh-light-mode-only" class="only-light">
        <img src="https://storage.googleapis.com/docs.mosaicml.com/images/header_light.svg" width="50%"/>
      </a>

  </p>

  <h2><p align="center">Supercharge your Model Training</p></h2>
  <h3><p align="center">Deep Learning Framework for Training at Scale</p></h3>

  <h4><p align='center'>
  <a href="https://www.mosaicml.com">[Website]</a>
  - <a href="https://docs.mosaicml.com/projects/composer/en/stable/getting_started/installation.html">[Getting Started]</a>
  - <a href="https://docs.mosaicml.com/projects/composer/">[Docs]</a>
  - <a href="https://www.databricks.com/company/careers/open-positions?department=Mosaic%20AI&location=all">[We're Hiring!]</a>
  </p></h4>

  <p align="center">
      <a href="https://pypi.org/project/mosaicml/">
          <img alt="PyPi Version" src="https://img.shields.io/pypi/pyversions/mosaicml">
      </a>
      <a href="https://pypi.org/project/mosaicml/">
          <img alt="PyPi Package Version" src="https://img.shields.io/pypi/v/mosaicml">
      </a>
      <a href="https://pepy.tech/project/mosaicml/">
          <img alt="PyPi Downloads" src="https://static.pepy.tech/personalized-badge/mosaicml?period=month&units=international_system&left_color=grey&right_color=blue&left_text=Downloads/month">
      </a>
      <a href="https://docs.mosaicml.com/projects/composer/en/stable/">
          <img alt="Documentation" src="https://readthedocs.org/projects/composer/badge/?version=stable">
      </a>
      <a href="https://dub.sh/mcomm">
          <img alt="Chat @ Slack" src="https://img.shields.io/badge/slack-chat-2eb67d.svg?logo=slack">
      </a>
      <a href="https://github.com/mosaicml/composer/blob/dev/LICENSE">
          <img alt="License" src="https://img.shields.io/badge/License-Apache%202.0-green.svg?logo=slack">
      </a>
  </p>
  <br />



  Composer is an open-source deep learning training library by [MosaicML](https://www.mosaicml.com/). Built on top of PyTorch, the Composer library makes it easier to implement distributed training workflows on large-scale clusters.

  We built Composer to be **optimized for scalability and usability**, integrating best practices for efficient, multi-node training. By abstracting away low-level complexities like parallelism techniques, distributed data loading, and memory optimization, you can focus on training modern ML models and running experiments without slowing down.

  We recommend using Composer to speedup your experimentation workflow if you’re training neural networks of any size, including:

  - Large Language Models (LLMs)
  - Diffusion models
  - Embedding models (e.g. BERT)
  - Transformer-based models
  - Convolutional Neural Networks (CNNs)

  Composer is heavily used by the MosaicML research team to train state-of-the-art models like MPT, and we open-sourced this library to enable the ML community to do the same. This framework is used by organizations in both the tech industry and the academic sphere and is continually updated with new features, bug fixes, and stability improvements for production workloads.


  ![Composer is to give you better workflows with the ability to maximize scale and customizability.](docs/source/_static/images/key_features.png)

  We designed Composer from the ground up for modern deep learning workloads. Gone are the days of AlexNet and ResNet, when state-of-the-art models could be trained on a couple of desktop GPUs. Today, developing the latest and greatest deep learning models often requires cluster-scale hardware — but with Composer’s help, you’ll hardly notice the difference.

  The heart of Composer is our Trainer abstraction: a highly optimized PyTorch training loop designed to allow both you and your model to iterate faster. Our trainer has simple ways for you to configure your parallelization scheme, data loaders, metrics, loggers, and more.



  Whether you’re training on 1 GPU or 512 GPUs, 50MB or 10TB of data - Composer is built to keep your workflow simple.

  - [**FSDP**](https://docs.mosaicml.com/projects/composer/en/stable/notes/distributed_training.html#fullyshardeddataparallel-fsdp): For large models that are too large to fit on GPUs, Composer has integrated PyTorch [FullyShardedDataParallelism](https://docs.mosaicml.com/projects/composer/en/stable/notes/distributed_training.html#fullyshardeddataparallel-fsdp) into our trainer and made it simple to efficiently parallelize custom models. We’ve found FSDP is competitive performance-wise with much more complex parallelism strategies. Alternatively, Composer also supports standard PyTorch distributed data parallelism (DDP) and Deepspeed execution.
  - [**Elastic sharded checkpointing**](https://docs.mosaicml.com/projects/composer/en/stable/notes/distributed_training.html#saving-and-loading-sharded-checkpoints-with-fsdp): Save on eight GPUs, resume on sixteen. Composer supports elastic sharded checkpointing, so you never have to worry if your sharded saved state is compatible with your new hardware setup.
  - **Data streaming:** Working with large datasets? Download datasets from cloud blob storage on the fly by integrating with MosaicML [StreamingDataset](https://github.com/mosaicml/streaming) during model training.



  Other high-level deep learning trainers provide simplicity at the cost of rigidity. When you want to add your own features, their abstractions get in your way. Composer, on the other hand, provides simple ways for you to customize our Trainer to your needs.

  ![Composer’s training loop has a series of events that occur at each stage in the training process.](docs/source/_static/images/traning_loop.png)

  ***Fig. 1:** Composer’s training loop has a series of events that occur at each stage in the training process. Callbacks are functions that users write to run at specific events. For example, our [Learning Rate Monitor Callback](https://docs.mosaicml.com/projects/composer/en/stable/api_reference/generated/composer.callbacks.LRMonitor.html#composer.callbacks.LRMonitor) logs the learning rate at every BATCH_END event.*

  - [**Callbacks**](https://docs.mosaicml.com/projects/composer/en/stable/trainer/callbacks.html): Composer’s callback system allows you to insert custom logic at any point in the training loop. We’ve written callbacks to monitor memory usage, log and visualize images, and estimate your model’s remaining training time, to name a few. This feature is popular among researchers who want to implement and experiment with custom training techniques.
  - [**Speedup algorithms**](https://docs.mosaicml.com/projects/composer/en/stable/examples/custom_speedup_methods.html): We draw from the latest research to create a collection of algorithmic speedups. Stack these speedups into MosaicML recipes to boost your training speeds. Our team has open-sourced the optimal combinations of speedups for different types of models.
      - **8x speedup: Stable Diffusion**
          - $200k original SD2 cost —> $50k ([Blog](https://www.mosaicml.com/blog/diffusion))
      - **7x speedup: ResNet-50 on ImageNet**
          - 3h33m —> 25m on 8xA100 ([Blog](https://www.mosaicml.com/blog/mosaic-resnet))
      - **8.8x speedup: BERT-Base Pretraining**
          - 10h —> 1.13h on 8xA100 ([Blog](https://www.mosaicml.com/blog/mosaicbert))
      - **5.4x speedup: DeepLab v3 on ADE20K**
          - 3h30m —> 39m on 8xA100 ([Blog](https://www.mosaicml.com/blog/behind-the-scenes))



  Composer is built to automate away low-level pain points and headaches so you can focus on the important (and fun) parts of deep learning and iterate faster.

  - [**Auto-resumption**](https://docs.mosaicml.com/projects/composer/en/stable/notes/resumption.html): Failed training run? Have no fear — just re-run your code, and Composer will automatically resume from your latest saved checkpoint.
  - [**CUDA OOM Prevention**](https://docs.mosaicml.com/projects/composer/en/stable/examples/auto_microbatching.html): Say goodbye to out-of-memory errors. Set your microbatch size to “auto”, and Composer will automatically select the biggest one that fits on your GPUs.
  - **[Time Abstractions](https://docs.mosaicml.com/projects/composer/en/latest/trainer/time.html):** Ever messed up your conversion between update steps, epochs, samples, and tokens? Specify your training duration with custom units (epochs, batches, samples, and tokens) in your training loop with our `Time` class.



  Integrate with the tools you know and love for experiment tracking and data streaming.

  - **Cloud integrations**: Our Checkpointing and logging features have first-class support for remote storage and loading from Cloud bucket (OCI, GCP, AWS S3).
  - **********Experiment tracking:********** Weights and Biases, MLFlow, CometML, and neptune.ai — the choice is yours, easily log your data to your favorite platform.





  Composer is designed for users who are comfortable with Python and have basic familiarity with deep learning fundamentals and PyTorch.

  **********************************************Software requirements:**********************************************  A recent version of PyTorch.

  **********************************************Hardware requirements:**********************************************  System with CUDA-compatible GPUs (AMD + RoCM coming soon!). Composer can run on CPUs, but for full benefits, we recommend using it on hardware accelerators.



  Composer can be installed with `pip`:

  <!--pytest.mark.skip-->
  ```bash
  pip install mosaicml
  ```

  To simplify the environment setup for Composer, we also provide a set of [pre-built Docker images](https://docs.mosaicml.com/projects/composer/en/stable/getting_started/installation.html#docker). We *highly recommend* you use our Docker images.



  Here is a code snippet demonstrating our Trainer on the MNIST dataset.

  <!--pytest.mark.filterwarnings(r'ignore:Some targets have less than 1 total probability:UserWarning')-->
  <!--pytest.mark.filterwarnings('ignore:Cannot split tensor of length .* into batches of size 128.*:UserWarning')-->
  ```python
  import torch
  import torch.nn as nn
  import torch.nn.functional as F
  from torchvision import datasets, transforms
  from torch.utils.data import DataLoader

  from composer import Trainer
  from composer.models import ComposerClassifier
  from composer.algorithms import LabelSmoothing, CutMix, ChannelsLast

  class Model(nn.Module):
      """Toy convolutional neural network architecture in pytorch for MNIST."""

      def __init__(self, num_classes: int = 10):
          super().__init__()

          self.num_classes = num_classes

          self.conv1 = nn.Conv2d(1, 16, (3, 3), padding=0)
          self.conv2 = nn.Conv2d(16, 32, (3, 3), padding=0)
          self.bn = nn.BatchNorm2d(32)
          self.fc1 = nn.Linear(32 * 16, 32)
          self.fc2 = nn.Linear(32, num_classes)

      def forward(self, x):
          out = self.conv1(x)
          out = F.relu(out)
          out = self.conv2(out)
          out = self.bn(out)
          out = F.relu(out)
          out = F.adaptive_avg_pool2d(out, (4, 4))
          out = torch.flatten(out, 1, -1)
          out = self.fc1(out)
          out = F.relu(out)
          return self.fc2(out)

  transform = transforms.Compose([transforms.ToTensor()])
  dataset = datasets.MNIST("data", train=True, download=True, transform=transform)
  train_dataloader = DataLoader(dataset, batch_size=128)

  trainer = Trainer(
      model=ComposerClassifier(module=Model(), num_classes=10),
      train_dataloader=train_dataloader,
      max_duration="2ep",
      algorithms=[
          LabelSmoothing(smoothing=0.1),
          CutMix(alpha=1.0),
          ChannelsLast(),
      ],
  )
  trainer.fit()
  ```

  Next, check out our [Getting Started Colab](https://colab.research.google.com/github/mosaicml/composer/blob/9f594876f957c912758e540598ac9f47a468c39d/examples/getting_started.ipynb) for a walk-through of Composer’s main features. In this tutorial, we will cover the basics of the Composer Trainer:

  - Dataloader
  - Trainer
  - Optimizer and Scheduler
  - Logging
  - Training a baseline model
  - Speeding up training



  Once you’ve completed the Quick Start, you can go through the below tutorials or our [documentation](https://docs.mosaicml.com/projects/composer/en/stable/) to further familiarize yourself with Composer.

  If you have any questions, please feel free to reach out to us on our [Community Slack](https://dub.sh/mcomm)!

  Here are some resources actively maintained by the Composer community to help you get started:
  <table>
  <thead>
    <tr>
        <th><b>Resource</b></th>
        <th><b>Details</b></th>
    </tr>
  </thead>
  <tbody>
      <tr>
      <td><a href="https://colab.research.google.com/github/mosaicml/composer/blob/dev/examples/finetune_huggingface.ipynb" target="_blank" rel="noopener noreferrer">Training BERTs with Composer and 🤗 </a></td>
      <td>A Colab Notebook showing how to train BERT models with Composer and 🤗!</td>
    </tr>
    <tr>
      <td><a href="https://github.com/mosaicml/llm-foundry/blob/main/TUTORIAL.md" target="_blank" rel="noopener noreferrer">Pretraining and Finetuning an LLM Tutorial</a></td>
      <td>A tutorial from MosaicML’s LLM Foundry, using MosaicML Composer, StreamingDataset, and MCLI on training and evaluating LLMs.</td>
    </tr>
    <tr>
      <td><a href="https://docs.mosaicml.com/projects/composer/en/stable/examples/migrate_from_ptl.html" target="_blank" rel="noopener noreferrer">Migrating from PyTorch Lightning</a></td>
      <td>A tutorial is to illustrating a path from working in PyTorch Lightning to working in Composer.</td>
    </tr>
    <tr>
      <td><a href="https://docs.mosaicml.com/projects/composer/en/stable/examples/finetune_huggingface.html" target="_blank" rel="noopener noreferrer">Finetuning and Pretraining HuggingFace Models</a></td>
      <td>Want to use Hugging Face models with Composer? No problem. Here, we’ll walk through using Composer to fine-tune a pretrained Hugging Face BERT model.</td>
    </tr>
    <tr>
      <td><a href="https://colab.research.google.com/github/mosaicml/composer/blob/dev/examples/custom_speedup_methods.ipynb" target="_blank" rel="noopener noreferrer">Building Speedup Methods</a></td>
      <td>A Colab Notebook showing how to build new training modifications on top of Composer</td>
    </tr>

  </tbody>
  </table>



  Composer can be used on its own, but for the smoothest experience we recommend using it in combination with other components of the MosaicML ecosystem:

  ![We recommend that you train models with Composer, MosaicML StreamingDatasets, and Mosaic AI training.](docs/source/_static/images/ecosystem.png)

  - [**Mosaic AI training**](https://www.databricks.com/product/machine-learning/mosaic-ai-training) (MCLI)- Our proprietary Command Line Interface (CLI) and Python SDK for orchestrating, scaling, and monitoring the GPU nodes and container images executing training and deployment. Used by our customers for training their own Generative AI models.
      - **To get started, [reach out here](https://www.databricks.com/company/contact) and check out our [Training](https://www.databricks.com/product/machine-learning/mosaic-ai-training) product pages**
  - [**MosaicML LLM Foundry**](https://github.com/mosaicml/llm-foundry) - This open source repository contains code for training, finetuning, evaluating, and preparing LLMs for inference with [Composer](https://github.com/mosaicml/composer). Designed to be easy to use, efficient and flexible, this codebase is designed to enable rapid experimentation with the latest techniques.
  - [**MosaicML StreamingDataset**](https://github.com/mosaicml/streaming) - Open-source library for fast, accurate streaming from cloud storage.
  - [**MosaicML Diffusion**](https://github.com/mosaicml/diffusion) - Open-source code to train your own Stable Diffusion model on your own data.  Learn more via our blogs: ([Results](https://www.mosaicml.com/blog/stable-diffusion-2) , [Speedup Details](https://www.mosaicml.com/blog/diffusion))



  Here are some projects and experiments that used Composer. Got something to add? Share in our [Community Slack](https://dub.sh/mcomm)!

  - [**MPT Foundation Series:**](https://www.mosaicml.com/mpt) Commercially usable open source LLMs, optimized for fast training and inference and trained with Composer.
      - [MPT-7B Blog](https://www.mosaicml.com/blog/mpt-7b)
      - [MPT-7B-8k Blog](https://www.mosaicml.com/blog/long-context-mpt-7b-8k)
      - [MPT-30B Blog](https://www.mosaicml.com/blog/mpt-30b)
  - [**Mosaic Diffusion Models**](https://www.mosaicml.com/blog/training-stable-diffusion-from-scratch-costs-160k): see how we trained a stable diffusion model from scratch for <$50k
  - [**replit-code-v1-3b**](https://huggingface.co/replit/replit-code-v1-3b): A 2.7B Causal Language Model focused on **Code Completion,** trained by Replit on Mosaic AI training in 10 days.
  - **BabyLLM:** the first LLM to support both Arabic and English. This 7B model was trained by MetaDialog on the world’s largest Arabic/English dataset to improve customer support workflows ([Blog](https://blogs.nvidia.com/blog/2023/08/31/generative-ai-startups-africa-middle-east/))
  - [**BioMedLM**](https://www.mosaicml.com/blog/introducing-pubmed-gpt): a domain-specific LLM for Bio Medicine built by MosaicML and [Stanford CRFM](https://crfm.stanford.edu/)



  Composer is part of the broader Machine Learning community, and we welcome any contributions, pull requests, or issues!

  To start contributing, see our [Contributing](https://github.com/mosaicml/composer/blob/dev/CONTRIBUTING.md) page.

  P.S.: [We're hiring](https://www.databricks.com/company/careers/open-positions?department=Mosaic%20AI&location=all)!



  - **What is the best tech stack you recommend when training large models?**
      - We recommend that users combine components of the MosaicML ecosystem for the smoothest experience:
          - Composer
          - [StreamingDataset](https://github.com/mosaicml/streaming)
          - [MCLI](https://www.databricks.com/product/machine-learning/mosaic-ai-training) (Databricks Mosaic AI Training)
  - **How can I get community support for using Composer?**
      - You can join our [Community Slack](https://dub.sh/mcomm)!
  - **How does Composer compare to other trainers like NeMo Megatron and PyTorch Lightning?**
      - We built Composer to be optimized for both simplicity and efficiency. Community users have shared that they enjoy Composer for its capabilities and ease of use compared to alternative libraries.
  - **How do I use Composer to train graph neural networks (GNNs), or Generative Adversarial Networks (GANs), or models for reinforcement learning (RL)?**
      - We recommend you use alternative libraries for if you want to train these types of models - a lot of assumptions we made when designing Composer are suboptimal for GNNs, RL, and GANs


  ```
  @misc{mosaicml2022composer,
      author = {The Mosaic ML Team},
      title = {composer},
      year = {2021},
      howpublished = {\url{https://github.com/mosaicml/composer/}},
  }
  ```


pkgId: 82e6cb6ef4b30031ec5ec2abe23dbec0826e182037ab78ddcd9ca6463f43e4cb
name: python-composer
arch: src
version: 0.27.0
epoch: 0
release: 1
url: https://github.com/mosaicml/composer
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-composer-0.27.0-1.src.rpm
checksum_type: sha256
summary:
  Composer is a PyTorch library that enables you to train neural networks faster, at lower cost, and to higher accuracy.

description:
  <br />
  <p align="center">
      <a href="https://github.com/mosaicml/composer#gh-light-mode-only" class="only-light">
        <img src="./docs/source/_static/logo-light-mode.png" width="50%"/>
      </a>

  </p>

  <h2><p align="center">Supercharge your Model Training</p></h2>
  <h3><p align="center">Deep Learning Framework for Training at Scale</p></h3>

  <h4><p align='center'>
  <a href="https://www.mosaicml.com">[Website]</a>
  - <a href="https://docs.mosaicml.com/projects/composer/en/stable/getting_started/installation.html">[Getting Started]</a>
  - <a href="https://docs.mosaicml.com/projects/composer/">[Docs]</a>
  - <a href="https://www.databricks.com/company/careers/open-positions?department=Mosaic%20AI&location=all">[We're Hiring!]</a>
  </p></h4>

  <p align="center">
      <a href="https://pypi.org/project/mosaicml/">
          <img alt="PyPi Version" src="https://img.shields.io/pypi/pyversions/mosaicml">
      </a>
      <a href="https://pypi.org/project/mosaicml/">
          <img alt="PyPi Package Version" src="https://img.shields.io/pypi/v/mosaicml">
      </a>
      <a href="https://pepy.tech/project/mosaicml/">
          <img alt="PyPi Downloads" src="https://static.pepy.tech/personalized-badge/mosaicml?period=month&units=international_system&left_color=grey&right_color=blue&left_text=Downloads/month">
      </a>
      <a href="https://docs.mosaicml.com/projects/composer/en/stable/">
          <img alt="Documentation" src="https://readthedocs.org/projects/composer/badge/?version=stable">
      </a>
      <a href="https://dub.sh/mcomm">
          <img alt="Chat @ Slack" src="https://img.shields.io/badge/slack-chat-2eb67d.svg?logo=slack">
      </a>
      <a href="https://github.com/mosaicml/composer/blob/dev/LICENSE">
          <img alt="License" src="https://img.shields.io/badge/License-Apache%202.0-green.svg?logo=slack">
      </a>
  </p>
  <br />



  Composer is an open-source deep learning training library by [MosaicML](https://www.mosaicml.com/). Built on top of PyTorch, the Composer library makes it easier to implement distributed training workflows on large-scale clusters.

  We built Composer to be **optimized for scalability and usability**, integrating best practices for efficient, multi-node training. By abstracting away low-level complexities like parallelism techniques, distributed data loading, and memory optimization, you can focus on training modern ML models and running experiments without slowing down.

  We recommend using Composer to speedup your experimentation workflow if you’re training neural networks of any size, including:

  - Large Language Models (LLMs)
  - Diffusion models
  - Embedding models (e.g. BERT)
  - Transformer-based models
  - Convolutional Neural Networks (CNNs)

  Composer is heavily used by the MosaicML research team to train state-of-the-art models like MPT, and we open-sourced this library to enable the ML community to do the same. This framework is used by organizations in both the tech industry and the academic sphere and is continually updated with new features, bug fixes, and stability improvements for production workloads.


  ![Composer is to give you better workflows with the ability to maximize scale and customizability.](docs/source/_static/images/key_features.png)

  We designed Composer from the ground up for modern deep learning workloads. Gone are the days of AlexNet and ResNet, when state-of-the-art models could be trained on a couple of desktop GPUs. Today, developing the latest and greatest deep learning models often requires cluster-scale hardware — but with Composer’s help, you’ll hardly notice the difference.

  The heart of Composer is our Trainer abstraction: a highly optimized PyTorch training loop designed to allow both you and your model to iterate faster. Our trainer has simple ways for you to configure your parallelization scheme, data loaders, metrics, loggers, and more.



  Whether you’re training on 1 GPU or 512 GPUs, 50MB or 10TB of data - Composer is built to keep your workflow simple.

  - [**FSDP**](https://docs.mosaicml.com/projects/composer/en/stable/notes/distributed_training.html#fullyshardeddataparallel-fsdp): For large models that are too large to fit on GPUs, Composer has integrated PyTorch [FullyShardedDataParallelism](https://docs.mosaicml.com/projects/composer/en/stable/notes/distributed_training.html#fullyshardeddataparallel-fsdp) into our trainer and made it simple to efficiently parallelize custom models. We’ve found FSDP is competitive performance-wise with much more complex parallelism strategies. Alternatively, Composer also supports standard PyTorch distributed data parallelism (DDP) and Deepspeed execution.
  - [**Elastic sharded checkpointing**](https://docs.mosaicml.com/projects/composer/en/stable/notes/distributed_training.html#saving-and-loading-sharded-checkpoints-with-fsdp): Save on eight GPUs, resume on sixteen. Composer supports elastic sharded checkpointing, so you never have to worry if your sharded saved state is compatible with your new hardware setup.
  - **Data streaming:** Working with large datasets? Download datasets from cloud blob storage on the fly by integrating with MosaicML [StreamingDataset](https://github.com/mosaicml/streaming) during model training.



  Other high-level deep learning trainers provide simplicity at the cost of rigidity. When you want to add your own features, their abstractions get in your way. Composer, on the other hand, provides simple ways for you to customize our Trainer to your needs.

  ![Composer’s training loop has a series of events that occur at each stage in the training process.](docs/source/_static/images/traning_loop.png)

  ***Fig. 1:** Composer’s training loop has a series of events that occur at each stage in the training process. Callbacks are functions that users write to run at specific events. For example, our [Learning Rate Monitor Callback](https://docs.mosaicml.com/projects/composer/en/stable/api_reference/generated/composer.callbacks.LRMonitor.html#composer.callbacks.LRMonitor) logs the learning rate at every BATCH_END event.*

  - [**Callbacks**](https://docs.mosaicml.com/projects/composer/en/stable/trainer/callbacks.html): Composer’s callback system allows you to insert custom logic at any point in the training loop. We’ve written callbacks to monitor memory usage, log and visualize images, and estimate your model’s remaining training time, to name a few. This feature is popular among researchers who want to implement and experiment with custom training techniques.
  - [**Speedup algorithms**](https://docs.mosaicml.com/projects/composer/en/stable/examples/custom_speedup_methods.html): We draw from the latest research to create a collection of algorithmic speedups. Stack these speedups into MosaicML recipes to boost your training speeds. Our team has open-sourced the optimal combinations of speedups for different types of models.
      - **8x speedup: Stable Diffusion**
          - $200k original SD2 cost —> $50k ([Blog](https://www.mosaicml.com/blog/diffusion))
      - **7x speedup: ResNet-50 on ImageNet**
          - 3h33m —> 25m on 8xA100 ([Blog](https://www.mosaicml.com/blog/mosaic-resnet))
      - **8.8x speedup: BERT-Base Pretraining**
          - 10h —> 1.13h on 8xA100 ([Blog](https://www.mosaicml.com/blog/mosaicbert))
      - **5.4x speedup: DeepLab v3 on ADE20K**
          - 3h30m —> 39m on 8xA100 ([Blog](https://www.mosaicml.com/blog/behind-the-scenes))



  Composer is built to automate away low-level pain points and headaches so you can focus on the important (and fun) parts of deep learning and iterate faster.

  - [**Auto-resumption**](https://docs.mosaicml.com/projects/composer/en/stable/notes/resumption.html): Failed training run? Have no fear — just re-run your code, and Composer will automatically resume from your latest saved checkpoint.
  - [**CUDA OOM Prevention**](https://docs.mosaicml.com/projects/composer/en/stable/examples/auto_microbatching.html): Say goodbye to out-of-memory errors. Set your microbatch size to “auto”, and Composer will automatically select the biggest one that fits on your GPUs.
  - **[Time Abstractions](https://docs.mosaicml.com/projects/composer/en/latest/trainer/time.html):** Ever messed up your conversion between update steps, epochs, samples, and tokens? Specify your training duration with custom units (epochs, batches, samples, and tokens) in your training loop with our `Time` class.



  Integrate with the tools you know and love for experiment tracking and data streaming.

  - **Cloud integrations**: Our Checkpointing and logging features have first-class support for remote storage and loading from Cloud bucket (OCI, GCP, AWS S3).
  - **********Experiment tracking:********** Weights and Biases, MLFlow, CometML, and neptune.ai — the choice is yours, easily log your data to your favorite platform.





  Composer is designed for users who are comfortable with Python and have basic familiarity with deep learning fundamentals and PyTorch.

  **********************************************Software requirements:**********************************************  A recent version of PyTorch.

  **********************************************Hardware requirements:**********************************************  System with CUDA-compatible GPUs (AMD + RoCM coming soon!). Composer can run on CPUs, but for full benefits, we recommend using it on hardware accelerators.



  Composer can be installed with `pip`:

  <!--pytest.mark.skip-->
  ```bash
  pip install mosaicml
  ```

  To simplify the environment setup for Composer, we also provide a set of [pre-built Docker images](https://docs.mosaicml.com/projects/composer/en/stable/getting_started/installation.html#docker). We *highly recommend* you use our Docker images.



  Here is a code snippet demonstrating our Trainer on the MNIST dataset.

  <!--pytest.mark.filterwarnings(r'ignore:Some targets have less than 1 total probability:UserWarning')-->
  <!--pytest.mark.filterwarnings('ignore:Cannot split tensor of length .* into batches of size 128.*:UserWarning')-->
  ```python
  import torch
  import torch.nn as nn
  import torch.nn.functional as F
  from torchvision import datasets, transforms
  from torch.utils.data import DataLoader

  from composer import Trainer
  from composer.models import ComposerClassifier
  from composer.algorithms import LabelSmoothing, CutMix, ChannelsLast

  class Model(nn.Module):
      """Toy convolutional neural network architecture in pytorch for MNIST."""

      def __init__(self, num_classes: int = 10):
          super().__init__()

          self.num_classes = num_classes

          self.conv1 = nn.Conv2d(1, 16, (3, 3), padding=0)
          self.conv2 = nn.Conv2d(16, 32, (3, 3), padding=0)
          self.bn = nn.BatchNorm2d(32)
          self.fc1 = nn.Linear(32 * 16, 32)
          self.fc2 = nn.Linear(32, num_classes)

      def forward(self, x):
          out = self.conv1(x)
          out = F.relu(out)
          out = self.conv2(out)
          out = self.bn(out)
          out = F.relu(out)
          out = F.adaptive_avg_pool2d(out, (4, 4))
          out = torch.flatten(out, 1, -1)
          out = self.fc1(out)
          out = F.relu(out)
          return self.fc2(out)

  transform = transforms.Compose([transforms.ToTensor()])
  dataset = datasets.MNIST("data", train=True, download=True, transform=transform)
  train_dataloader = DataLoader(dataset, batch_size=128)

  trainer = Trainer(
      model=ComposerClassifier(module=Model(), num_classes=10),
      train_dataloader=train_dataloader,
      max_duration="2ep",
      algorithms=[
          LabelSmoothing(smoothing=0.1),
          CutMix(alpha=1.0),
          ChannelsLast(),
      ],
  )
  trainer.fit()
  ```

  Next, check out our [Getting Started Colab](https://colab.research.google.com/github/mosaicml/composer/blob/9f594876f957c912758e540598ac9f47a468c39d/examples/getting_started.ipynb) for a walk-through of Composer’s main features. In this tutorial, we will cover the basics of the Composer Trainer:

  - Dataloader
  - Trainer
  - Optimizer and Scheduler
  - Logging
  - Training a baseline model
  - Speeding up training



  Once you’ve completed the Quick Start, you can go through the below tutorials or our [documentation](https://docs.mosaicml.com/projects/composer/en/stable/) to further familiarize yourself with Composer.

  If you have any questions, please feel free to reach out to us on our [Community Slack](https://dub.sh/mcomm)!

  Here are some resources actively maintained by the Composer community to help you get started:
  <table>
  <thead>
    <tr>
        <th><b>Resource</b></th>
        <th><b>Details</b></th>
    </tr>
  </thead>
  <tbody>
      <tr>
      <td><a href="https://colab.research.google.com/github/mosaicml/composer/blob/dev/examples/finetune_huggingface.ipynb" target="_blank" rel="noopener noreferrer">Training BERTs with Composer and 🤗 </a></td>
      <td>A Colab Notebook showing how to train BERT models with Composer and 🤗!</td>
    </tr>
    <tr>
      <td><a href="https://github.com/mosaicml/llm-foundry/blob/main/TUTORIAL.md" target="_blank" rel="noopener noreferrer">Pretraining and Finetuning an LLM Tutorial</a></td>
      <td>A tutorial from MosaicML’s LLM Foundry, using MosaicML Composer, StreamingDataset, and MCLI on training and evaluating LLMs.</td>
    </tr>
    <tr>
      <td><a href="https://docs.mosaicml.com/projects/composer/en/stable/examples/migrate_from_ptl.html" target="_blank" rel="noopener noreferrer">Migrating from PyTorch Lightning</a></td>
      <td>A tutorial is to illustrating a path from working in PyTorch Lightning to working in Composer.</td>
    </tr>
    <tr>
      <td><a href="https://docs.mosaicml.com/projects/composer/en/stable/examples/finetune_huggingface.html" target="_blank" rel="noopener noreferrer">Finetuning and Pretraining HuggingFace Models</a></td>
      <td>Want to use Hugging Face models with Composer? No problem. Here, we’ll walk through using Composer to fine-tune a pretrained Hugging Face BERT model.</td>
    </tr>
    <tr>
      <td><a href="https://colab.research.google.com/github/mosaicml/composer/blob/dev/examples/custom_speedup_methods.ipynb" target="_blank" rel="noopener noreferrer">Building Speedup Methods</a></td>
      <td>A Colab Notebook showing how to build new training modifications on top of Composer</td>
    </tr>

  </tbody>
  </table>



  Composer can be used on its own, but for the smoothest experience we recommend using it in combination with other components of the MosaicML ecosystem:

  ![We recommend that you train models with Composer, MosaicML StreamingDatasets, and Mosaic AI training.](docs/source/_static/images/ecosystem.png)

  - [**Mosaic AI training**](https://www.databricks.com/product/machine-learning/mosaic-ai-training) (MCLI)- Our proprietary Command Line Interface (CLI) and Python SDK for orchestrating, scaling, and monitoring the GPU nodes and container images executing training and deployment. Used by our customers for training their own Generative AI models.
      - **To get started, [reach out here](https://www.databricks.com/company/contact) and check out our [Training](https://www.databricks.com/product/machine-learning/mosaic-ai-training) product pages**
  - [**MosaicML LLM Foundry**](https://github.com/mosaicml/llm-foundry) - This open source repository contains code for training, finetuning, evaluating, and preparing LLMs for inference with [Composer](https://github.com/mosaicml/composer). Designed to be easy to use, efficient and flexible, this codebase is designed to enable rapid experimentation with the latest techniques.
  - [**MosaicML StreamingDataset**](https://github.com/mosaicml/streaming) - Open-source library for fast, accurate streaming from cloud storage.
  - [**MosaicML Diffusion**](https://github.com/mosaicml/diffusion) - Open-source code to train your own Stable Diffusion model on your own data.  Learn more via our blogs: ([Results](https://www.mosaicml.com/blog/stable-diffusion-2) , [Speedup Details](https://www.mosaicml.com/blog/diffusion))



  Here are some projects and experiments that used Composer. Got something to add? Share in our [Community Slack](https://dub.sh/mcomm)!

  - [**MPT Foundation Series:**](https://www.mosaicml.com/mpt) Commercially usable open source LLMs, optimized for fast training and inference and trained with Composer.
      - [MPT-7B Blog](https://www.mosaicml.com/blog/mpt-7b)
      - [MPT-7B-8k Blog](https://www.mosaicml.com/blog/long-context-mpt-7b-8k)
      - [MPT-30B Blog](https://www.mosaicml.com/blog/mpt-30b)
  - [**Mosaic Diffusion Models**](https://www.mosaicml.com/blog/training-stable-diffusion-from-scratch-costs-160k): see how we trained a stable diffusion model from scratch for <$50k
  - [**replit-code-v1-3b**](https://huggingface.co/replit/replit-code-v1-3b): A 2.7B Causal Language Model focused on **Code Completion,** trained by Replit on Mosaic AI training in 10 days.
  - **BabyLLM:** the first LLM to support both Arabic and English. This 7B model was trained by MetaDialog on the world’s largest Arabic/English dataset to improve customer support workflows ([Blog](https://blogs.nvidia.com/blog/2023/08/31/generative-ai-startups-africa-middle-east/))
  - [**BioMedLM**](https://www.mosaicml.com/blog/introducing-pubmed-gpt): a domain-specific LLM for Bio Medicine built by MosaicML and [Stanford CRFM](https://crfm.stanford.edu/)



  Composer is part of the broader Machine Learning community, and we welcome any contributions, pull requests, or issues!

  To start contributing, see our [Contributing](https://github.com/mosaicml/composer/blob/dev/CONTRIBUTING.md) page.

  P.S.: [We're hiring](https://www.databricks.com/company/careers/open-positions?department=Mosaic%20AI&location=all)!



  - **What is the best tech stack you recommend when training large models?**
      - We recommend that users combine components of the MosaicML ecosystem for the smoothest experience:
          - Composer
          - [StreamingDataset](https://github.com/mosaicml/streaming)
          - [MCLI](https://www.databricks.com/product/machine-learning/mosaic-ai-training) (Databricks Mosaic AI Training)
  - **How can I get community support for using Composer?**
      - You can join our [Community Slack](https://dub.sh/mcomm)!
  - **How does Composer compare to other trainers like NeMo Megatron and PyTorch Lightning?**
      - We built Composer to be optimized for both simplicity and efficiency. Community users have shared that they enjoy Composer for its capabilities and ease of use compared to alternative libraries.
  - **How do I use Composer to train graph neural networks (GNNs), or Generative Adversarial Networks (GANs), or models for reinforcement learning (RL)?**
      - We recommend you use alternative libraries for if you want to train these types of models - a lot of assumptions we made when designing Composer are suboptimal for GNNs, RL, and GANs


  ```
  @misc{mosaicml2022composer,
      author = {The Mosaic ML Team},
      title = {composer},
      year = {2021},
      howpublished = {\url{https://github.com/mosaicml/composer/}},
  }
  ```


pkgId: a127c85734d04c37cd7287cf6a041f0b427e8ae51e386670cbc2b45876f1ee26
name: python-composer
arch: src
version: 0.28.0
epoch: 0
release: 1
url: https://github.com/mosaicml/composer
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-composer-0.28.0-1.src.rpm
checksum_type: sha256
summary:
  Composer is a PyTorch library that enables you to train neural networks faster, at lower cost, and to higher accuracy.

description:
  <br />
  <p align="center">
      <a href="https://github.com/mosaicml/composer#gh-light-mode-only" class="only-light">
        <img src="./docs/source/_static/logo-light-mode.png" width="50%"/>
      </a>

  </p>

  <h2><p align="center">Supercharge your Model Training</p></h2>
  <h3><p align="center">Deep Learning Framework for Training at Scale</p></h3>

  <h4><p align='center'>
  <a href="https://www.mosaicml.com">[Website]</a>
  - <a href="https://docs.mosaicml.com/projects/composer/en/stable/getting_started/installation.html">[Getting Started]</a>
  - <a href="https://docs.mosaicml.com/projects/composer/">[Docs]</a>
  - <a href="https://www.databricks.com/company/careers/open-positions?department=Mosaic%20AI&location=all">[We're Hiring!]</a>
  </p></h4>

  <p align="center">
      <a href="https://pypi.org/project/mosaicml/">
          <img alt="PyPi Version" src="https://img.shields.io/pypi/pyversions/mosaicml">
      </a>
      <a href="https://pypi.org/project/mosaicml/">
          <img alt="PyPi Package Version" src="https://img.shields.io/pypi/v/mosaicml">
      </a>
      <a href="https://pepy.tech/project/mosaicml/">
          <img alt="PyPi Downloads" src="https://static.pepy.tech/personalized-badge/mosaicml?period=month&units=international_system&left_color=grey&right_color=blue&left_text=Downloads/month">
      </a>
      <a href="https://docs.mosaicml.com/projects/composer/en/stable/">
          <img alt="Documentation" src="https://readthedocs.org/projects/composer/badge/?version=stable">
      </a>
      <a href="https://dub.sh/mcomm">
          <img alt="Chat @ Slack" src="https://img.shields.io/badge/slack-chat-2eb67d.svg?logo=slack">
      </a>
      <a href="https://github.com/mosaicml/composer/blob/dev/LICENSE">
          <img alt="License" src="https://img.shields.io/badge/License-Apache%202.0-green.svg?logo=slack">
      </a>
  </p>
  <br />



  Composer is an open-source deep learning training library by [MosaicML](https://www.mosaicml.com/). Built on top of PyTorch, the Composer library makes it easier to implement distributed training workflows on large-scale clusters.

  We built Composer to be **optimized for scalability and usability**, integrating best practices for efficient, multi-node training. By abstracting away low-level complexities like parallelism techniques, distributed data loading, and memory optimization, you can focus on training modern ML models and running experiments without slowing down.

  We recommend using Composer to speedup your experimentation workflow if you’re training neural networks of any size, including:

  - Large Language Models (LLMs)
  - Diffusion models
  - Embedding models (e.g. BERT)
  - Transformer-based models
  - Convolutional Neural Networks (CNNs)

  Composer is heavily used by the MosaicML research team to train state-of-the-art models like MPT, and we open-sourced this library to enable the ML community to do the same. This framework is used by organizations in both the tech industry and the academic sphere and is continually updated with new features, bug fixes, and stability improvements for production workloads.


  ![Composer is to give you better workflows with the ability to maximize scale and customizability.](docs/source/_static/images/key_features.png)

  We designed Composer from the ground up for modern deep learning workloads. Gone are the days of AlexNet and ResNet, when state-of-the-art models could be trained on a couple of desktop GPUs. Today, developing the latest and greatest deep learning models often requires cluster-scale hardware — but with Composer’s help, you’ll hardly notice the difference.

  The heart of Composer is our Trainer abstraction: a highly optimized PyTorch training loop designed to allow both you and your model to iterate faster. Our trainer has simple ways for you to configure your parallelization scheme, data loaders, metrics, loggers, and more.



  Whether you’re training on 1 GPU or 512 GPUs, 50MB or 10TB of data - Composer is built to keep your workflow simple.

  - [**FSDP**](https://docs.mosaicml.com/projects/composer/en/stable/notes/distributed_training.html#fullyshardeddataparallel-fsdp): For large models that are too large to fit on GPUs, Composer has integrated PyTorch [FullyShardedDataParallelism](https://docs.mosaicml.com/projects/composer/en/stable/notes/distributed_training.html#fullyshardeddataparallel-fsdp) into our trainer and made it simple to efficiently parallelize custom models. We’ve found FSDP is competitive performance-wise with much more complex parallelism strategies. Alternatively, Composer also supports standard PyTorch distributed data parallelism (DDP) execution.
  - [**Elastic sharded checkpointing**](https://docs.mosaicml.com/projects/composer/en/stable/notes/distributed_training.html#saving-and-loading-sharded-checkpoints-with-fsdp): Save on eight GPUs, resume on sixteen. Composer supports elastic sharded checkpointing, so you never have to worry if your sharded saved state is compatible with your new hardware setup.
  - **Data streaming:** Working with large datasets? Download datasets from cloud blob storage on the fly by integrating with MosaicML [StreamingDataset](https://github.com/mosaicml/streaming) during model training.



  Other high-level deep learning trainers provide simplicity at the cost of rigidity. When you want to add your own features, their abstractions get in your way. Composer, on the other hand, provides simple ways for you to customize our Trainer to your needs.

  ![Composer’s training loop has a series of events that occur at each stage in the training process.](docs/source/_static/images/traning_loop.png)

  ***Fig. 1:** Composer’s training loop has a series of events that occur at each stage in the training process. Callbacks are functions that users write to run at specific events. For example, our [Learning Rate Monitor Callback](https://docs.mosaicml.com/projects/composer/en/stable/api_reference/generated/composer.callbacks.LRMonitor.html#composer.callbacks.LRMonitor) logs the learning rate at every BATCH_END event.*

  - [**Callbacks**](https://docs.mosaicml.com/projects/composer/en/stable/trainer/callbacks.html): Composer’s callback system allows you to insert custom logic at any point in the training loop. We’ve written callbacks to monitor memory usage, log and visualize images, and estimate your model’s remaining training time, to name a few. This feature is popular among researchers who want to implement and experiment with custom training techniques.
  - [**Speedup algorithms**](https://docs.mosaicml.com/projects/composer/en/stable/examples/custom_speedup_methods.html): We draw from the latest research to create a collection of algorithmic speedups. Stack these speedups into MosaicML recipes to boost your training speeds. Our team has open-sourced the optimal combinations of speedups for different types of models.
      - **8x speedup: Stable Diffusion**
          - $200k original SD2 cost —> $50k ([Blog](https://www.mosaicml.com/blog/diffusion))
      - **7x speedup: ResNet-50 on ImageNet**
          - 3h33m —> 25m on 8xA100 ([Blog](https://www.mosaicml.com/blog/mosaic-resnet))
      - **8.8x speedup: BERT-Base Pretraining**
          - 10h —> 1.13h on 8xA100 ([Blog](https://www.mosaicml.com/blog/mosaicbert))
      - **5.4x speedup: DeepLab v3 on ADE20K**
          - 3h30m —> 39m on 8xA100 ([Blog](https://www.mosaicml.com/blog/behind-the-scenes))



  Composer is built to automate away low-level pain points and headaches so you can focus on the important (and fun) parts of deep learning and iterate faster.

  - [**Auto-resumption**](https://docs.mosaicml.com/projects/composer/en/stable/notes/resumption.html): Failed training run? Have no fear — just re-run your code, and Composer will automatically resume from your latest saved checkpoint.
  - [**CUDA OOM Prevention**](https://docs.mosaicml.com/projects/composer/en/stable/examples/auto_microbatching.html): Say goodbye to out-of-memory errors. Set your microbatch size to “auto”, and Composer will automatically select the biggest one that fits on your GPUs.
  - **[Time Abstractions](https://docs.mosaicml.com/projects/composer/en/latest/trainer/time.html):** Ever messed up your conversion between update steps, epochs, samples, and tokens? Specify your training duration with custom units (epochs, batches, samples, and tokens) in your training loop with our `Time` class.



  Integrate with the tools you know and love for experiment tracking and data streaming.

  - **Cloud integrations**: Our Checkpointing and logging features have first-class support for remote storage and loading from Cloud bucket (OCI, GCP, AWS S3).
  - **********Experiment tracking:********** Weights and Biases, MLFlow, CometML, and neptune.ai — the choice is yours, easily log your data to your favorite platform.





  Composer is designed for users who are comfortable with Python and have basic familiarity with deep learning fundamentals and PyTorch.

  **********************************************Software requirements:**********************************************  A recent version of PyTorch.

  **********************************************Hardware requirements:**********************************************  System with CUDA-compatible GPUs (AMD + RoCM coming soon!). Composer can run on CPUs, but for full benefits, we recommend using it on hardware accelerators.



  Composer can be installed with `pip`:

  <!--pytest.mark.skip-->
  ```bash
  pip install mosaicml
  ```

  To simplify the environment setup for Composer, we also provide a set of [pre-built Docker images](https://docs.mosaicml.com/projects/composer/en/stable/getting_started/installation.html#docker). We *highly recommend* you use our Docker images.



  Here is a code snippet demonstrating our Trainer on the MNIST dataset.

  <!--pytest.mark.filterwarnings(r'ignore:Some targets have less than 1 total probability:UserWarning')-->
  <!--pytest.mark.filterwarnings('ignore:Cannot split tensor of length .* into batches of size 128.*:UserWarning')-->
  ```python
  import torch
  import torch.nn as nn
  import torch.nn.functional as F
  from torchvision import datasets, transforms
  from torch.utils.data import DataLoader

  from composer import Trainer
  from composer.models import ComposerClassifier
  from composer.algorithms import LabelSmoothing, CutMix, ChannelsLast

  class Model(nn.Module):
      """Toy convolutional neural network architecture in pytorch for MNIST."""

      def __init__(self, num_classes: int = 10):
          super().__init__()

          self.num_classes = num_classes

          self.conv1 = nn.Conv2d(1, 16, (3, 3), padding=0)
          self.conv2 = nn.Conv2d(16, 32, (3, 3), padding=0)
          self.bn = nn.BatchNorm2d(32)
          self.fc1 = nn.Linear(32 * 16, 32)
          self.fc2 = nn.Linear(32, num_classes)

      def forward(self, x):
          out = self.conv1(x)
          out = F.relu(out)
          out = self.conv2(out)
          out = self.bn(out)
          out = F.relu(out)
          out = F.adaptive_avg_pool2d(out, (4, 4))
          out = torch.flatten(out, 1, -1)
          out = self.fc1(out)
          out = F.relu(out)
          return self.fc2(out)

  transform = transforms.Compose([transforms.ToTensor()])
  dataset = datasets.MNIST("data", train=True, download=True, transform=transform)
  train_dataloader = DataLoader(dataset, batch_size=128)

  trainer = Trainer(
      model=ComposerClassifier(module=Model(), num_classes=10),
      train_dataloader=train_dataloader,
      max_duration="2ep",
      algorithms=[
          LabelSmoothing(smoothing=0.1),
          CutMix(alpha=1.0),
          ChannelsLast(),
      ],
  )
  trainer.fit()
  ```

  Next, check out our [Getting Started Colab](https://colab.research.google.com/github/mosaicml/composer/blob/9f594876f957c912758e540598ac9f47a468c39d/examples/getting_started.ipynb) for a walk-through of Composer’s main features. In this tutorial, we will cover the basics of the Composer Trainer:

  - Dataloader
  - Trainer
  - Optimizer and Scheduler
  - Logging
  - Training a baseline model
  - Speeding up training



  Once you’ve completed the Quick Start, you can go through the below tutorials or our [documentation](https://docs.mosaicml.com/projects/composer/en/stable/) to further familiarize yourself with Composer.

  If you have any questions, please feel free to reach out to us on our [Community Slack](https://dub.sh/mcomm)!

  Here are some resources actively maintained by the Composer community to help you get started:
  <table>
  <thead>
    <tr>
        <th><b>Resource</b></th>
        <th><b>Details</b></th>
    </tr>
  </thead>
  <tbody>
      <tr>
      <td><a href="https://colab.research.google.com/github/mosaicml/composer/blob/dev/examples/finetune_huggingface.ipynb" target="_blank" rel="noopener noreferrer">Training BERTs with Composer and 🤗 </a></td>
      <td>A Colab Notebook showing how to train BERT models with Composer and 🤗!</td>
    </tr>
    <tr>
      <td><a href="https://github.com/mosaicml/llm-foundry/blob/main/TUTORIAL.md" target="_blank" rel="noopener noreferrer">Pretraining and Finetuning an LLM Tutorial</a></td>
      <td>A tutorial from MosaicML’s LLM Foundry, using MosaicML Composer, StreamingDataset, and MCLI on training and evaluating LLMs.</td>
    </tr>
    <tr>
      <td><a href="https://docs.mosaicml.com/projects/composer/en/stable/examples/migrate_from_ptl.html" target="_blank" rel="noopener noreferrer">Migrating from PyTorch Lightning</a></td>
      <td>A tutorial is to illustrating a path from working in PyTorch Lightning to working in Composer.</td>
    </tr>
    <tr>
      <td><a href="https://docs.mosaicml.com/projects/composer/en/stable/examples/finetune_huggingface.html" target="_blank" rel="noopener noreferrer">Finetuning and Pretraining HuggingFace Models</a></td>
      <td>Want to use Hugging Face models with Composer? No problem. Here, we’ll walk through using Composer to fine-tune a pretrained Hugging Face BERT model.</td>
    </tr>
    <tr>
      <td><a href="https://colab.research.google.com/github/mosaicml/composer/blob/dev/examples/custom_speedup_methods.ipynb" target="_blank" rel="noopener noreferrer">Building Speedup Methods</a></td>
      <td>A Colab Notebook showing how to build new training modifications on top of Composer</td>
    </tr>

  </tbody>
  </table>



  Composer can be used on its own, but for the smoothest experience we recommend using it in combination with other components of the MosaicML ecosystem:

  ![We recommend that you train models with Composer, MosaicML StreamingDatasets, and Mosaic AI training.](docs/source/_static/images/ecosystem.png)

  - [**Mosaic AI training**](https://www.databricks.com/product/machine-learning/mosaic-ai-training) (MCLI)- Our proprietary Command Line Interface (CLI) and Python SDK for orchestrating, scaling, and monitoring the GPU nodes and container images executing training and deployment. Used by our customers for training their own Generative AI models.
      - **To get started, [reach out here](https://www.databricks.com/company/contact) and check out our [Training](https://www.databricks.com/product/machine-learning/mosaic-ai-training) product pages**
  - [**MosaicML LLM Foundry**](https://github.com/mosaicml/llm-foundry) - This open source repository contains code for training, finetuning, evaluating, and preparing LLMs for inference with [Composer](https://github.com/mosaicml/composer). Designed to be easy to use, efficient and flexible, this codebase is designed to enable rapid experimentation with the latest techniques.
  - [**MosaicML StreamingDataset**](https://github.com/mosaicml/streaming) - Open-source library for fast, accurate streaming from cloud storage.
  - [**MosaicML Diffusion**](https://github.com/mosaicml/diffusion) - Open-source code to train your own Stable Diffusion model on your own data.  Learn more via our blogs: ([Results](https://www.mosaicml.com/blog/stable-diffusion-2) , [Speedup Details](https://www.mosaicml.com/blog/diffusion))



  Here are some projects and experiments that used Composer. Got something to add? Share in our [Community Slack](https://dub.sh/mcomm)!

  - [**MPT Foundation Series:**](https://www.mosaicml.com/mpt) Commercially usable open source LLMs, optimized for fast training and inference and trained with Composer.
      - [MPT-7B Blog](https://www.mosaicml.com/blog/mpt-7b)
      - [MPT-7B-8k Blog](https://www.mosaicml.com/blog/long-context-mpt-7b-8k)
      - [MPT-30B Blog](https://www.mosaicml.com/blog/mpt-30b)
  - [**Mosaic Diffusion Models**](https://www.mosaicml.com/blog/training-stable-diffusion-from-scratch-costs-160k): see how we trained a stable diffusion model from scratch for <$50k
  - [**replit-code-v1-3b**](https://huggingface.co/replit/replit-code-v1-3b): A 2.7B Causal Language Model focused on **Code Completion,** trained by Replit on Mosaic AI training in 10 days.
  - **BabyLLM:** the first LLM to support both Arabic and English. This 7B model was trained by MetaDialog on the world’s largest Arabic/English dataset to improve customer support workflows ([Blog](https://blogs.nvidia.com/blog/2023/08/31/generative-ai-startups-africa-middle-east/))
  - [**BioMedLM**](https://www.mosaicml.com/blog/introducing-pubmed-gpt): a domain-specific LLM for Bio Medicine built by MosaicML and [Stanford CRFM](https://crfm.stanford.edu/)



  Composer is part of the broader Machine Learning community, and we welcome any contributions, pull requests, or issues!

  To start contributing, see our [Contributing](https://github.com/mosaicml/composer/blob/dev/CONTRIBUTING.md) page.

  P.S.: [We're hiring](https://www.databricks.com/company/careers/open-positions?department=Mosaic%20AI&location=all)!



  - **What is the best tech stack you recommend when training large models?**
      - We recommend that users combine components of the MosaicML ecosystem for the smoothest experience:
          - Composer
          - [StreamingDataset](https://github.com/mosaicml/streaming)
          - [MCLI](https://www.databricks.com/product/machine-learning/mosaic-ai-training) (Databricks Mosaic AI Training)
  - **How can I get community support for using Composer?**
      - You can join our [Community Slack](https://dub.sh/mcomm)!
  - **How does Composer compare to other trainers like NeMo Megatron and PyTorch Lightning?**
      - We built Composer to be optimized for both simplicity and efficiency. Community users have shared that they enjoy Composer for its capabilities and ease of use compared to alternative libraries.
  - **How do I use Composer to train graph neural networks (GNNs), or Generative Adversarial Networks (GANs), or models for reinforcement learning (RL)?**
      - We recommend you use alternative libraries for if you want to train these types of models - a lot of assumptions we made when designing Composer are suboptimal for GNNs, RL, and GANs


  ```
  @misc{mosaicml2022composer,
      author = {The Mosaic ML Team},
      title = {composer},
      year = {2021},
      howpublished = {\url{https://github.com/mosaicml/composer/}},
  }
  ```


pkgId: 40b9a98ee699b1b44173f403bc131d81fad254595f5ec2401cfaee1256e1fae8
name: python-composio-autogen
arch: src
version: 0.5.31
epoch: 0
release: 1
summary: Use Composio to get an array of tools with your Autogen agent.
url: https://github.com/ComposioHQ/composio
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-composio-autogen-0.5.31-1.src.rpm
checksum_type: sha256
description:
  Use Composio to enhance your Autogen workflows with a suite of tools.





  Automatically star a GitHub repository using natural language commands through an Autogen Agent.



  Install Composio Autogen and connect your GitHub account to enable your agents with GitHub functionalities.

  ```bash
  pip install composio-autogen
  composio-cli add github  # Connect your GitHub account
  composio-cli show-apps   # Check all supported apps
  ```





  Setup your environment by importing necessary packages and configuring the Autogen agent.

  ```python
  from autogen import AssistantAgent, UserProxyAgent
  from composio_autogen import ComposioToolSet, App, Action
  import os


  llm_config = {"config_list": [{"model": "gpt-4-turbo", "api_key": os.environ["OPENAI_API_KEY"]}]}


  chatbot = AssistantAgent(
      "chatbot",
      system_message="Reply TERMINATE when the task is done or when user's content is empty",
      llm_config=llm_config,
  )


  user_proxy = UserProxyAgent(
      "user_proxy",
      is_termination_msg=lambda x: x.get("content", "") and "TERMINATE" in x.get("content", ""),
      human_input_mode="NEVER",  # Don't take input from User
      code_execution_config={"use_docker": False}
  )
  ```



  Initialize and register the necessary tools for interacting with GitHub.

  ```python
  from composio_autogen import ComposioToolSet, App, Action


  composio_tools = ComposioToolSet()


  composio_tools.register_tools(tools=[App.GITHUB], caller=chatbot, executor=user_proxy)
  ```



  Perform tasks like starring a repository on GitHub using the configured agent.

  ```python
  task = "Star a repo composiohq/composio on GitHub"


  response = user_proxy.initiate_chat(chatbot, message=task)

  print(response.chat_history)
  ```



  Verify the task completion and response from the agent.

  ```bash
  [{'content': 'I have starred the repository "composio" for you on GitHub under the account "composiohq".', 'role': 'user'},
  {'content': '', 'role': 'assistant'}, {'content': 'TERMINATE', 'role': 'user'}]
  ```



  - **Filter Specific Actions:** Limit the actions an agent can execute to enhance security and focus.

  ```python
  composio_tools.register_tools(actions=[Action.GITHUB_CREATE_ISSUE])
  ```

  - **Filter Specific Apps:** Restrict the agent's access to specific tools for streamlined operations.

  ```python
  composio_tools.register_tools([App.ASANA, App.GITHUB])
  ```


pkgId: ccd428012a4875e2563fa7a448e84eac562afbea8eaf12451e120ea72cf5dc8c
name: python-composio-camel
arch: src
version: 0.5.34
epoch: 0
release: 1
summary: Use Composio to get an array of tools with your Claude LLMs.
url: https://github.com/ComposioHQ/composio
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-composio-camel-0.5.34-1.src.rpm
checksum_type: sha256
description:
  Facilitate the integration of Camel Agents with Composio to empower the agents to directly interact with external applications, broadening their capabilities and application scope.



  - **Automate starring a GitHub repository** using conversational instructions via Camel's Tool Usage ability.



  Ensure you have the necessary packages installed and connect your GitHub account to allow your agents to utilize GitHub functionalities.

  ```bash

  pip install composio-camel


  composio-cli add github


  composio-cli show-apps
  ```





  Prepare your environment by initializing necessary imports from Composio & Camel.

  ```python
  from colorama import Fore

  from camel.agents import ChatAgent
  from camel.configs import ChatGPTConfig
  from camel.messages import BaseMessage
  from camel.models import ModelFactory
  from camel.types import ModelPlatformType, ModelType
  from camel.utils import print_text_animated
  from composio_camel import ComposioToolSet, Action
  ```



  This step involves fetching and integrating GitHub tools provided by Composio, enabling enhanced functionality for Camel operations.
  ```python
  composio_toolset = ComposioToolSet()
  tools = composio_toolset.get_actions(
      actions=[Action.GITHUB_STAR_A_REPOSITORY_FOR_THE_AUTHENTICATED_USER]
  )
  ```



  This step involves configuring and executing the agent to carry out actions, such as starring a GitHub repository.

  ```python

  assistant_model_config = ChatGPTConfig(
      temperature=0.0,
      tools=tools,
  )

  model = ModelFactory.create(
      model_platform=ModelPlatformType.OPENAI,
      model_type=ModelType.GPT_3_5_TURBO,
      model_config_dict=assistant_model_config.__dict__,
  )



  assistant_sys_msg = BaseMessage.make_assistant_message(
      role_name="Developer",
      content=(
          "You are a programmer as well an experienced github user. "
          "When asked given a instruction, "
          "you try to use available tools, and execute it"
      ),
  )

  agent = ChatAgent(
      assistant_sys_msg,
      model,
      tools=tools,
  )
  agent.reset()
  ```



  Execute the following code to execute the agent, ensuring that the intended task has been successfully completed.

  ```python
  prompt = (
      "I have created a new Github Repo,"
      "Please star my github repository: camel-ai/camel"
  )
  user_msg = BaseMessage.make_user_message(role_name="User", content=prompt)
  print(Fore.YELLOW + f"user prompt:\n{prompt}\n")

  response = agent.step(user_msg)
  for msg in response.msgs:
      print_text_animated(Fore.GREEN + f"Agent response:\n{msg.content}\n")

  ```


pkgId: 88c21c09ca5f5a9f3d3640cb240b58c8569452fcf2ea4b532235d31876a5db13
name: python-composio-claude
arch: src
version: 0.5.31
epoch: 0
release: 1
summary: Use Composio to get an array of tools with your Claude LLMs.
url: https://github.com/ComposioHQ/composio
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-composio-claude-0.5.31-1.src.rpm
checksum_type: sha256
description:
  Facilitate the integration of Claude with Composio to empower Claude models to directly interact with external applications, broadening their capabilities and application scope.



  - **Automate starring a GitHub repository** using conversational instructions via Claude Function Calls.



  Ensure you have the necessary packages installed and connect your GitHub account to allow your agents to utilize GitHub functionalities.

  ```bash

  pip install composio-claude


  composio-cli add github


  composio-cli show-apps
  ```





  Prepare your environment by initializing necessary imports from Claude and setting up your client.

  ```python
  import anthropic


  client = anthropic.Anthropic()
  ```



  This step involves fetching and integrating GitHub tools provided by Composio, enabling enhanced functionality for LangChain operations.
  ```python
  from composio_claude import App, ComposioToolSet

  toolset = ComposioToolSet()
  actions = toolset.get_tools(tools=App.GITHUB)
  ```



  This step involves configuring and executing the agent to carry out actions, such as starring a GitHub repository.

  ```python
  my_task = "Star a repo composiohq/composio on GitHub"


  response = client.beta.tools.messages.create(
      model="claude-3-opus-20240229",
      max_tokens=1024,
      tools= actions,
      messages=[{"role": "user", "content": "Star me composiohq/composio repo in github."}],
  )
  pprint(response)
  ```



  Execute the following code to validate the response, ensuring that the intended task has been successfully completed.

  ```python
  result = toolset.handle_tool_calls(response)
  pprint(result)
  ```


pkgId: fc7cfa280c9704efb49c7350fa1c6bef6a538afc2e53751d5ed6320bdab7fe7e
name: python-composio-core
arch: src
version: 0.5.41
epoch: 0
release: 1
summary: Core package to act as a bridge between composio platform and other services.
url: https://github.com/composiohq/composio
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-composio-core-0.5.41-1.src.rpm
checksum_type: sha256
description:
  <p align="center">
    <a href="https://x.com/GanatraSoham/?utm_campaign=github-readme" target="_blank">
      <img src="./python/docs/imgs/follow_x.png" width="100%" alt="Follow me" />
    </a>
    <br /> <br />
  </p>
  <p align="center">
    <a href="https://app.composio.dev/?utm_campaign=github-readme" target="_blank">
      <img src="./python/docs/imgs/try_hosted.png" width="100%" alt="Sign up" />
    </a>

    <br /> <br />
  </p>
  <p>
    <a href="https://github.com/composiohq/composio/blob/master/README.md">EN</a> | <a href="https://github.com/composiohq/composio/blob/master/README-CN.md">CN</a> | <a href="https://github.com/composiohq/composio/blob/master/README-JP.md">JP</a>
  </p>

  <p align="center">
    <a href="https://composio.dev//#gh-dark-mode-only">
      <img src="./python/docs/imgs/composio_white_font.svg" width="318px" alt="Composio logo" />
    </a>
    <a href="https://composio.dev//#gh-light-mode-only">
      <img src="./python/docs/imgs/composio_black_font.svg" width="318px" alt="Composio Logo" />
    </a>
  </p>
  <p align="center">
    <a href="https://github.com/composiodev/composio/actions/workflows/common.yml">
    <img alt="Tests" src="https://img.shields.io/github/actions/workflow/status/composiodev/composio/common.yml?label=Tests&style=plastic&logo=github&color=blue&cacheSeconds=60">
    </a>
    <a href="https://pypi.org/project/composio-core/">
    <img alt="PyPI" src="https://img.shields.io/pypi/v/composio_core?label=Latest&style=plastic&logo=pypi&color=blue&cacheSeconds=60&logoColor=white">
    </a>
    <a href="https://www.npmjs.com/package/composio-core">
    <img alt="NPM" src="https://img.shields.io/npm/v/composio-core?style=plastic&logo=npm&logoColor=white&label=latest&color=blue&cacheSeconds=60">
    </a>
    <a href="https://pypi.org/project/composio-core/">
    <img alt="Downloads" src="https://img.shields.io/pypi/dm/composio-core?label=Downloads&style=plastic&logo=github&color=blue&cacheSeconds=60">
    </a>

  </p>

  <h2 align="center"><i>
    Production Ready Toolset for AI Agents
  </i></h2>

  <h4 align="center">Equip your agent with high-quality tools & integrations without worrying about authentication, accuracy, and reliability in a single line of code!
  </h4>
  <div align="center">
  <p>
  <a href="https://docs.composio.dev" rel="dofollow"><strong>Explore the Docs »</strong></a>
  </p>

  <p>
  <a href="https://app.composio.dev">Try on Dashboard</a> <b>|</b>
  <a href="https://www.composio.dev">Homepage</a> <b>|</b>
  <!-- <a href="https://docs.composio.dev/guides/examples">Examples</a> |
  <a href="https://docs.composio.dev/chat-with-docs">Chat with Docs</a> | -->
  <a href="https://docs.composio.dev/sdk">SDK</a> <b>|</b>
  <a href="https://docs.composio.dev/api-reference/">APIs</a>
  </p>
  </div>

  <hr>
  <div align="center">
  <p >
      <b>✨ Socials >></b>
      <a href="https://dub.composio.dev/JoinHQ">Discord</a> <b>|</b>
      <a href="https://www.youtube.com/@Composio">Youtube</a> <b>|</b>
      <a href="https://twitter.com/composiohq">Twitter</a> <b>|</b>
      <a href="https://www.linkedin.com/company/composio-dev"> Linkedin </a>
  </p>
  <p align="center">
      <b>⛏️ Contribute >></b>
      <a href="https://github.com/composiodev/composio/issues/new?assignees=&labels=type%3A+bug&template=bug_report.yml&title=%F0%9F%90%9B+Bug+Report%3A+">Report Bugs</a> <b>|</b>
      <a href="https://github.com/composiodev/composio/issues/new?assignees=&labels=feature&template=feature_request.yml&title=%F0%9F%9A%80+Feature%3A+">Request Feature</a> <b>|</b>
      <a href="https://github.com/composiodev/composio/blob/master/CONTRIBUTING.md">Contribute</a>
  </p>
  </div>



  - [📋 Table of contents](#-table-of-contents)
  - [🤔 Why Composio?](#-why-composio)
  - [🔥 Key Features](#-key-features)
  - [🚀 Getting Started with Python](#-getting-started-with-python)
    - [1. Installation](#1-installation)
    - [2. Testing Composio in Action](#2-testing-composio-in-action)
  - [🚀 Getting Started with Javascript](#-getting-started-with-javascript)
    - [1. **Install the Composio SDK**:](#1-install-the-composio-sdk)
    - [2. **Setup the OpenAI and Composio Tool Set**:](#2-setup-the-openai-and-composio-tool-set)
    - [3. **Run your script**:](#3-run-your-script)
  - [💡 Examples](#-examples)
    - [Python Examples](#python-examples)
    - [Javascript Examples](#javascript-examples)
  - [Star History](#star-history)
  - [📋 Read Our Code Of Conduct](#-read-our-code-of-conduct)
  - [🤗 Contributions](#-contributions)
  - [🔗 Links](#-links)
  - [🛡️ License](#️-license)
  - [💪 Thanks To All Contributors](#-thanks-to-all-contributors)



  We believe AI Based Agents/Workflows are the future.
  Composio is the best toolset to integrate AI Agents to best Agentic Tools and use them to accomplish tasks.

  <img alt="Illustration" src="./python/docs/imgs/banner.gif" style="border-radius: 5px"/>



  - **100+ Tools**: Support for a range of different categories

    - **Software**: Do anything on GitHub, Notion, Linear, Gmail, Slack, Hubspot, Salesforce, & 90 more.
    - **OS**: Click anywhere, Type anything, Copy to Clipboard, & more.
    - **Browser**: Smart Search, Take a screenshot, MultiOn, Download, Upload, & more.
    - **Search**: Google Search, Perplexity Search, Tavily, Exa & more.
    - **SWE**: Ngrok, Database, Redis, Vercel, Git, etc.
    - **RAG**: Agentic RAG for any type of data on the fly!

  - **Frameworks**: Use tools with agent frameworks like **OpenAI, Groq (OpenAI compatible), Claude, LlamaIndex, Langchain, CrewAI, Autogen, Gemini, Julep, Lyzr**, and more in a single line of code.
  - **Managed Authorisation**: Supports six different auth protocols. _Access Token, Refresh token, OAuth, API Keys, JWT, and more_ abstracted out so you can focus on the building agents.
  - **Accuracy**: Get _up to 40% better agentic accuracy_ in your tool calls due to better tool designs.
  - **Embeddable**: Whitelabel in the backend of your applications managing Auth & Integrations for all your users & agents and maintain a consistent experience.
  - **Pluggable**: Designed to be extended with additional Tools, Frameworks and Authorisation Protocols very easily.





  To get started, type the following command in your Terminal.

  ```bash
  pip install composio-core
  ```

  If you want to install the 'composio' package along with its openai plugin: `pip install composio-openai`.



  Let's use Composio to create an AI Agent that can star a Github Repo.

  ```bash
  composio add github # Connect your Github - Run this in terminal
  ```

  ```python

  from openai import OpenAI
  from composio_openai import ComposioToolSet, App, Action

  openai_client = OpenAI(
      api_key="{{OPENAIKEY}}"
  )



  composio_tool_set = ComposioToolSet()


  actions = composio_tool_set.get_actions(
      actions=[Action.GITHUB_STAR_A_REPOSITORY_FOR_THE_AUTHENTICATED_USER]
  )

  my_task = "Star a repo composiodev/composio on GitHub"


  assistant_instruction = "You are a super intelligent personal assistant"

  assistant = openai_client.beta.assistants.create(
      name="Personal Assistant",
      instructions=assistant_instruction,
      model="gpt-4-turbo",
      tools=actions,
  )


  thread = openai_client.beta.threads.create()

  message = openai_client.beta.threads.messages.create(
      thread_id=thread.id,
      role="user",
      content=my_task
  )


  run = openai_client.beta.threads.runs.create(
      thread_id=thread.id,
      assistant_id=assistant.id
  )



  response_after_tool_calls = composio_tool_set.wait_and_handle_assistant_tool_calls(
      client=openai_client,
      run=run,
      thread=thread,
  )

  print(response_after_tool_calls)
  ```



  To get started with the Composio SDK in Javascript, follow these steps:


     ```bash
     npm install composio-core
     ```


     ```javascript
     import { OpenAI } from "openai";
     import { OpenAIToolSet } from "composio-core";

     const toolset = new OpenAIToolSet({
         apiKey: process.env.COMPOSIO_API_KEY,
     });

     async function setupUserConnectionIfNotExists(entityId) {
         const entity = await toolset.client.getEntity(entityId);
         const connection = await entity.getConnection('github');

         if (!connection) {
             // If this entity/user hasn't already connected the account
             const connection = await entity.initiateConnection(appName);
             console.log("Log in via: ", connection.redirectUrl);
             return connection.waitUntilActive(60);
         }

         return connection;
     }

     async function executeAgent(entityName) {
         const entity = await toolset.client.getEntity(entityName)
         await setupUserConnectionIfNotExists(entity.id);

         const tools = await toolset.get_actions({ actions: ["github_issues_create"] }, entity.id);
         const instruction = "Make an issue with sample title in the repo - himanshu-dixit/custom-repo-breaking"

         const client = new OpenAI({ apiKey: process.env.OPEN_AI_API_KEY })
         const response = await client.chat.completions.create({
             model: "gpt-4-turbo",
             messages: [{
                 role: "user",
                 content: instruction,
             }],
             tools: tools,
             tool_choice: "auto",
         })

         console.log(response.choices[0].message.tool_calls);
         await toolset.handle_tool_call(response, entity.id);
     }

     executeAgent("your-entity-name");
     ```


     ```bash
     node your_script.js
     ```

  This will set up the Composio SDK and execute an agent that creates a GitHub issue using the provided instructions.

  For more details, refer to the [Composio SDK Documentation](https://docs.composio.dev/).












  [![Star History Chart](https://api.star-history.com/svg?repos=composiohq/composio&type=Date)](https://star-history.com/#composiohq/composio&Date)




  As part of our open-source community, we hold ourselves and other contributors to a high standard of communication. As a participant and contributor to this project, you agree to abide by our [Code of Conduct](https://github.com/composiodev/composio/blob/master/CODE_OF_CONDUCT.md).



  Composio is open-source and we welcome contributions. Please fork the repository, create a new branch for your feature, add your feature or improvement, and send a pull request.

  Also go through our [Contribution Guidelines](https://github.com/composiodev/composio/blob/master/CONTRIBUTING.md) and [Code of Conduct](https://github.com/composiodev/composio/blob/master/CODE_OF_CONDUCT.md) before you start.



  - [Home page](https://composio.dev?utm_campaign=github-readme)
  - [Contribution Guidelines](https://github.com/composiodev/composio/blob/master/CONTRIBUTING.md)
  - [Docs](https://docs.composio.dev/?utm_campaign=github-readme)



  Composio is licensed under the Elastic License - see the [LICENSE](https://github.com/composiodev/composio/blob/master/LICENSE) file for details.



  <a href="https://github.com/composiohq/composio/graphs/contributors">
    <img src="https://contributors-img.web.app/image?repo=composiodev/composio" alt="List of Contributors"/>
  </a>


pkgId: 0d13de831b3f6a0309d3cc138835f00877b9bb198bfb2240d80b683e9d8f2363
name: python-composio-crewai
arch: src
version: 0.5.34
epoch: 0
release: 1
summary: Use Composio to get an array of tools with your CrewAI agent.
url: https://github.com/ComposioHQ/composio
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-composio-crewai-0.5.34-1.src.rpm
checksum_type: sha256
description:
  ![CrewAI Logo](https://i.imgur.com/jXeNUda.png)

  **Composio** enables **CrewAI agents** to connect with numerous tools, making it easy for these agents to interact with external applications seamlessly.



  Start by installing Composio CrewAI and connecting your GitHub account to enable GitHub functionalities for your agents.

  ```bash
  pip install composio_crewai
  composio-cli add github  # Connect your GitHub account
  composio-cli show-apps   # Check available applications
  ```



  Automate GitHub interactions, such as starring a repository, using natural language commands through a CrewAI Agent.





  Prepare your environment by importing the necessary packages for CrewAI and setting up your language model.

  ```python
  from crewai import Agent, Task
  from langchain_openai import ChatOpenAI


  llm = ChatOpenAI(openai_api_key="sk-<OPENAI KEY>")
  ```



  Initialize the Composio toolset for interacting with GitHub.

  ```python
  from composio_crewai import ComposioToolSet, Action, App


  tools = ComposioToolSet(apps=[App.GITHUB])
  ```



  Configure and execute the CrewAI agent to perform GitHub actions.

  ```python
  crewai_agent = Agent(
      role='Github Agent',
      goal="You take action on Github using Github APIs",
      backstory="You are an AI agent responsible for taking actions on Github on users' behalf using Github APIs",
      verbose=True,
      tools=tools,
      llm=llm
  )


  task = Task(
      description="Star a repo composiohq/composio on GitHub",
      agent=crewai_agent,
      expected_output="if the star happened"
  )


  task.execute()
  ```



  Verify the agent's actions and responses to ensure the task was completed successfully.

  ```bash
  > Entering new CrewAgentExecutor chain...
  > I need to star the repository "composiohq/composio" on GitHub.
  > {'execution_details': {'executed': True}, 'response_data': ''}
  > Finished chain.
  ```



  - **Filter Specific Actions:** Limit the actions an agent can perform for enhanced security and operational focus.

  ```python
  toolsGithubCreateIssue = ComposioToolSet(actions=[Action.GITHUB_CREATE_ISSUE])
  ```

  - **Filter Specific Apps:** Restrict the agent's access to certain applications for streamlined operations.

  ```python
  toolsAsanaGithub = ComposioToolSet(apps=[App.ASANA, App.GITHUB])
  ```


pkgId: 410963eeca4752f68341dd918cecb1d3cdb6b10aae39e539e2ce43452ac013a9
name: python-composio-google
arch: src
version: 0.5.41
epoch: 0
release: 1
summary: Use Composio to get an array of tools with your Google AI Python Gemini model.
url: https://github.com/ComposioHQ/composio
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-composio-google-0.5.41-1.src.rpm
checksum_type: sha256
description:
  Streamline the integration of Composio with Google AI Python to enhance the capabilities of Gemini models, allowing them to interact directly with external applications and expanding their operational scope.



  - **Automate starring a GitHub repository** using conversational instructions via Google AI Python's Function Calling feature.



  Ensure you have the necessary packages installed and connect your GitHub account to allow your agents to utilize GitHub functionalities.

  ```bash

  pip install composio-google


  composio-cli add github


  composio-cli show-apps
  ```





  Prepare your environment by initializing necessary imports from Google AI Python and setting up your client.

  ```python
  from vertexai.generative_models import GenerativeModel


  model = GenerativeModel("gemini-pro")
  ```



  This step involves fetching and integrating GitHub tools provided by Composio, enabling enhanced functionality for Google AI Python operations.
  ```python
  from composio_google import App, ComposioToolset

  toolset = ComposioToolset()
  actions = toolset.get_tools(apps=[App.GITHUB])
  ```



  This step involves configuring and executing the agent to carry out actions, such as starring a GitHub repository.

  ```python

  task = "Star a repo composiohq/composio on GitHub"


  response = chat.send_message(task)
  ```



  Execute the following code to validate the response, ensuring that the intended task has been successfully completed.

  ```python
  result = composio_toolset.handle_response(response)
  print("Function call result:", result)
  ```


pkgId: f1b536432afd40d6a6de50cf069df2b462dcffde48dda50ae2560a713a4595f8
name: python-composio-griptape
arch: src
version: 0.5.31
epoch: 0
release: 1
summary: Use Composio to get an array of tools with your Griptape wokflow.
url: https://github.com/ComposioHQ/composio
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-composio-griptape-0.5.31-1.src.rpm
checksum_type: sha256
description:
  Streamline the integration of Composio within the Griptape agentic framework to enhance the interaction capabilities of Griptape agents with external applications, significantly extending their operational range and efficiency.



  - **Facilitate the automation of starring a GitHub repository** through the use of conversational commands within the Griptape framework, leveraging Composio's Function Calls.




  Ensure you have the necessary packages installed and connect your GitHub account to allow your agents to utilize GitHub functionalities.

  ```bash

  pip install composio-griptape


  composio-cli add github


  composio-cli show-apps
  ```





  Establish your development environment by importing necessary libraries and setting up the Griptape client.
  ```python
  from griptape.structures import Agent
  from griptape.utils import Chat

  from composio_griptape import App, Action, Tag, ComposioToolSet
  import dotenv


  dotenv.load_dotenv("/Users/sawradip/Desktop/practice_code/practice_composio/composio_sdk/examples/.env")

  ```



  This section guides you through the process of integrating GitHub tools into your Griptape projects using Composio's services.

  ```python
  composio_toolset = ComposioToolSet()
  composio_tools = composio_toolset.get_tools(tools = App.GITHUB)

  agent = Agent(
      tools=composio_tools
  )
  ```



  This step involves configuring and Chatting with the Griptape agent to carry out specific tasks, for example, starring a GitHub repository.

  ```python
  Chat(agent).start()
  ```


pkgId: 8fd27aa4e4443c60d52ed3dc807e092becda071823d80827a15462404d36b183
name: python-composio-julep
arch: src
version: 0.5.31
epoch: 0
release: 1
summary: Use Composio to get an array of tools with your Julep wokflow.
url: https://github.com/ComposioHQ/composio
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-composio-julep-0.5.31-1.src.rpm
checksum_type: sha256
description:
  Streamline the integration of Composio within the Julep agentic framework to enhance the interaction capabilities of Julep agents with external applications, significantly extending their operational range and efficiency.



  - **Facilitate the automation of starring a GitHub repository** through the use of conversational commands within the Julep framework, leveraging Composio's OpenAI Function Calls.



  Ensure you have the necessary packages installed and connect your GitHub account to allow your agents to utilize GitHub functionalities.

  ```bash

  pip install composio-openai


  composio-cli add github


  composio-cli show-apps
  ```





  Set up your development environment by importing essential libraries and configuring the Julep client.
  ```python

  import os
  import textwrap
  from julep import Client
  from dotenv import load_dotenv


  load_dotenv()

  api_key = os.environ["JULEP_API_KEY"]
  base_url = os.environ["JULEP_API_URL"]


  client = Client(api_key=api_key, base_url=base_url)



  name = "Jessica"
  about = "Jessica is a forward-thinking tech entrepreneur with a sharp eye for disruptive technologies. She excels in identifying and nurturing innovative tech startups, with a particular interest in sustainability and AI."
  default_settings = {
      "temperature": 0.7,
      "top_p": 1,
      "min_p": 0.01,
      "presence_penalty": 0,
      "frequency_penalty": 0,
      "length_penalty": 1.0,
      "max_tokens": 150,
  }
  ```



  This section guides you through the process of integrating GitHub tools into your LangChain projects using Composio's services.
  ```python
  from composio_julep import App, ComposioToolSet

  toolset = ComposioToolSet()
  composio_tools = toolset.get_tools(tools=App.GITHUB)


  agent = client.agents.create(
      name=name,
      about=about,
      default_settings=default_settings,
      model="gpt-4-turbo",
      tools=composio_tools,
  )
  ```



  This step involves configuring and executing the agent to carry out specific tasks, for example, starring a GitHub repository.
  ```python
  about = """
  Sawradip, a software developer, is passionate about impactful tech.
  At the tech fair, he seeks investors and collaborators for his project.
  """
  user = client.users.create(
      name="Sawradip",
      about=about,
  )

  situation_prompt = """You are Jessica, a key figure in the tech community, always searching for groundbreaking technologies. At a tech fair filled with innovative projects, your goal is to find and support the next big thing.

  Your journey through the fair is highlighted by encounters with various projects, from groundbreaking to niche. You believe in the power of unexpected innovation.

  Recent Tweets
  1. 'Amazed by the tech fair's creativity. The future is bright. #TechInnovation'
  2. 'Met a developer with a transformative tool for NGOs. This is the
  """

  session = client.sessions.create(
      user_id=user.id, agent_id=agent.id, situation=situation_prompt
  )

  user_msg = "Hi, I am presenting my project, hosted at github repository composiohq/composio. If you like it, adding a star would be helpful "



  response = client.sessions.chat(
      session_id=session.id,
      messages=[
          {
              "role": "user",
              "content": user_msg,
              "name": "Sawradip",
          }
      ],
      recall=True,
      remember=True,
  )

  pprint(response)
  ```



  Execute and validate the response to ensure the task was completed successfully.

  ```python
  execution_output = toolset.handle_tool_calls(response)
  print(execution_output)
  ```


pkgId: 6a15012e74ea4c485efacd4b2b4d1038d67b5d8554e21c138234d27bb0d658b3
name: python-composio-langchain
arch: src
version: 0.5.31
epoch: 0
release: 1
summary: Use Composio to get an array of tools with your LangChain agent.
url: https://github.com/ComposioHQ/composio
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-composio-langchain-0.5.31-1.src.rpm
checksum_type: sha256
description:
  Integrate Composio with LangChain agents to allow them to interact seamlessly with external apps, enhancing their functionality and reach.



  - **Star a repository on GitHub** using natural language commands through a LangChain Agent.



  Ensure you have the necessary packages installed and connect your GitHub account to allow your agents to utilize GitHub functionalities.

  ```bash

  pip install composio-langchain


  composio-cli add github


  composio-cli show-apps
  ```





  Prepare your environment by initializing necessary imports from LangChain and setting up your agent.

  ```python
  from langchain.agents import create_openai_functions_agent, AgentExecutor
  from langchain import hub
  from langchain_openai import ChatOpenAI


  llm = ChatOpenAI()


  prompt = hub.pull("hwchase17/openai-functions-agent")
  ```



  Access GitHub tools provided by Composio for LangChain.

  ```python
  from composio_langchain import ComposioToolSet, Action, App


  tools = ComposioToolSet(apps=[App.GITHUB])
  ```



  Configure and execute the agent to perform tasks such as starring a repository on GitHub.

  ```python
  task = "Star a repo composiohq/composio on GitHub"


  agent = create_openai_functions_agent(llm, tools, prompt)
  agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)


  agent_executor.invoke({"input": task})
  ```



  Validate the execution and response from the agent to ensure the task was completed successfully.

  ```bash
  > Entering new AgentExecutor chain...
  > Invoking: `github_star_repo` with `{'owner': 'composiohq', 'repo': 'composio'}`
  > {'execution_details': {'executed': True}, 'response_data': ''}
  > I have successfully starred the repository composiohq/composio on GitHub.
  ```



  - **Filter Specific Actions:** Limit the actions an agent can execute.

  ```python

  toolsGithubCreateIssue = ComposioToolSet(actions=[Action.GITHUB_CREATE_ISSUE])
  ```

  - **Filter Specific Apps:** Restrict the tools an agent can use.

  ```python

  toolsAsanaGithub = ComposioToolSet(apps=[App.ASANA, App.GITHUB])
  ```


pkgId: 64dd9a5e93873e7358a61696a012d929f4fb462e5efb92d61fc626885a10047e
name: python-composio-langgraph
arch: src
version: 0.5.34
epoch: 0
release: 1
summary: Use Composio to get array of tools with LnagGraph Agent Workflows
url: https://github.com/ComposioHQ/composio
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-composio-langgraph-0.5.34-1.src.rpm
checksum_type: sha256
description:
  Integrate Composio with LangGraph Agentic workflows & enable them to interact seamlessly with external apps, enhancing their functionality and reach.



  - **Star a repository on GitHub** using natural language commands through a LangGraph Agent.



  Ensure you have the necessary packages installed and connect your GitHub account to allow your agents to utilize GitHub functionalities.

  ```bash

  pip install composio-langgraph


  composio-cli add github


  composio-cli show-apps
  ```





  Prepare your environment by initializing necessary imports from LangGraph & LangChain for setting up your agent.

  ```python
  from typing import Literal

  from langchain_openai import ChatOpenAI
  from langgraph.graph import MessagesState, StateGraph
  from langgraph.prebuilt import ToolNode
  ```



  Access GitHub tools provided by Composio for LangGraph, initialize a `ToolNode` with necessary tools obtaned from `ComposioToolSet`.

  ```python
  from composio_langgraph import Action, ComposioToolSet


  composio_toolset = ComposioToolSet()
  tools = composio_toolset.get_actions(
      actions=[
          Action.GITHUB_ACTIVITY_STAR_REPO_FOR_AUTHENTICATED_USER,
          Action.GITHUB_USERS_GET_AUTHENTICATED,
      ])
  tool_node = ToolNode(tools)
  ```



  Initialize the LLM class and bind obtained tools to the model.

  ```python
  model = ChatOpenAI(temperature=0, streaming=True)
  model_with_tools = model.bind_tools(functions)
  ```


  LangGraph expects you to define different nodes of the agentic workflow as separate functions. Here we define a node for calling the LLM model.

  ```python
  def call_model(state: MessagesState):
      messages = state["messages"]
      response = model_with_tools.invoke(messages)
      return {"messages": [response]}
  ```


  To establish the agent's workflow, we begin by initializing the workflow with `agent` and `tools` node, followed by specifying the connecting edges between nodes, finally compiling the workflow. These edges can be straightforward or conditional, depending on the workflow requirements.

  ```python
  def should_continue(state: MessagesState) -> Literal["tools", "__end__"]:
      messages = state["messages"]
      last_message = messages[-1]
      if last_message.tool_calls:
          return "tools"
      return "__end__"


  workflow = StateGraph(MessagesState)


  workflow.add_node("agent", call_model)
  workflow.add_node("tools", tool_node)

  workflow.add_edge("__start__", "agent")
  workflow.add_conditional_edges(
      "agent",
      should_continue,
  )
  workflow.add_edge("tools", "agent")

  app = workflow.compile()
  ```


  After the compilation of workflow, we invoke the LLM with a task, and stream the response.

  ```python
  for chunk in app.stream(
      {
          "messages": [
              (
                  "human",
                  
                  "Get my information.",
              )
          ]
      },
      stream_mode="values",
  ):
      chunk["messages"][-1].pretty_print()
  ```


pkgId: 8f153da87618f2c2ca7cac66e3299c26b0af8fdc403a28649a91bb78f59d837b
name: python-composio-llamaindex
arch: src
version: 0.5.34
epoch: 0
release: 1
summary: Use Composio to get an array of tools with your LlamaIndex agent.
url: https://github.com/ComposioHQ/composio
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-composio-llamaindex-0.5.34-1.src.rpm
checksum_type: sha256
description:
  Integrate Composio with llamaindex agents to allow them to interact seamlessly with external apps & data sources, enhancing their functionality and reach.



  - **Star a repository on GitHub** using natural language commands through a llamaindex Agent.



  Ensure you have the necessary packages installed and connect your GitHub account to allow your agents to utilize GitHub functionalities.

  ```bash

  pip install composio-llamaindex


  composio-cli add github


  composio-cli show-apps
  ```





  Prepare your environment by initializing necessary imports from llamaindex and setting up your agent.

  ```python
  from llama_index.llms.openai import OpenAI
  from llama_index.core.llms import ChatMessage
  from llama_index.core.agent import FunctionCallingAgentWorker

  import dotenv
  from llama_index.core.tools import FunctionTool


  dotenv.load_dotenv()

  llm = OpenAI(model="gpt-4-turbo")
  ```



  Access GitHub tools provided by Composio for llamaindex.

  ```python
  from composio_llamaindex import App, Action, ComposioToolSet


  composio_toolset = ComposioToolSet()
  tools = composio_toolset.get_actions(
      actions=[Action.GITHUB_STAR_A_REPOSITORY_FOR_THE_AUTHENTICATED_USER]
  )
  print(tools)
  ```



  Configure the agent to perform tasks such as starring a repository on GitHub.

  ```python

  prefix_messages = [
      ChatMessage(
          role="system",
          content=(
              "You are now a integration agent, and what  ever you are requested, you will try to execute utilizing your toools."
          ),
      )
  ]

  agent = FunctionCallingAgentWorker(
      tools=tools,
      llm=llm,
      prefix_messages=prefix_messages,
      max_function_calls=10,
      allow_parallel_tool_calls=False,
      verbose=True,
  ).as_agent()
  ```



  Validate the execution and response from the agent to ensure the task was completed successfully.

  ```bash
  response = agent.chat("Hello! I would like to star a repo composiohq/composio on GitHub")
  print("Response:", response)
  ```


pkgId: 8f759ba3fe31a2679b166c250fa74028b0bace43c358c377694cdca88f77932e
name: python-composio-lyzr
arch: src
version: 0.5.31
epoch: 0
release: 1
summary: Use Composio to get an array of tools with your Lyzr workflow.
url: https://github.com/ComposioHQ/composio
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-composio-lyzr-0.5.31-1.src.rpm
checksum_type: sha256
description:
  Integrate Composio with Lyzr agents to allow them to interact seamlessly with external apps, enhancing their functionality and reach.



  - **Star a repository on GitHub** using natural language commands through a LangChain Agent.



  Ensure you have the necessary packages installed and connect your GitHub account to allow your agents to utilize GitHub functionalities.

  ```bash

  pip install composio-lyzr


  composio-cli add github


  composio-cli show-apps
  ```



  ```
  from  lyzr_automata  import  Task, Agent
  from lyzr_automata.ai_models.openai import OpenAIModel
  from lyzr_automata.tasks.task_literals import InputType, OutputType
  from lyzr_automata.pipelines.linear_sync_pipeline import LinearSyncPipeline

  import os
  import dotenv
  dotenv.load_dotenv()
  from composio_lyzr import ComposioToolSet, App, Action

  open_ai_text_completion_model = OpenAIModel(
      api_key=os.environ["OPENAI_API_KEY"],
      parameters={
          "model": "gpt-4-turbo-preview",
          "temperature": 0.2,
          "max_tokens": 1500,
      },
  )


  lyzr_agent = Agent(
          role="Github Agent",
          prompt_persona="You are AI agent that is responsible for taking actions on Github on users behalf. You need to take action on Github using Github APIs"
      )

  composio_toolset = ComposioToolSet()
  composio_tool = composio_toolset.get_lyzr_tool(Action.GITHUB_STAR_REPO)

  task = Task(
          name="Github Starring",
          agent=lyzr_agent,
          tool=composio_tool,
          output_type=OutputType.TEXT,
          input_type=InputType.TEXT,
          model=open_ai_text_completion_model,
          instructions="Star a repo composiohq/composio on GitHub",
          log_output=True,
          enhance_prompt=False,
  )

  lyzr_output = LinearSyncPipeline(
      name="Composio Lyzr",
  		
      completion_message="Task completed",
      tasks=[
  				
          task,
      ],
  ).run()

  print(lyzr_output)
  ```


pkgId: 5fa0a1322c8e17009e4d06675176b43022e163bc700215665ac6700894e3a007
name: python-composio-openai
arch: src
version: 0.5.31
epoch: 0
release: 1
summary: Use Composio to get an array of tools with your OpenAI Function Call.
url: https://github.com/ComposioHQ/composio
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-composio-openai-0.5.31-1.src.rpm
checksum_type: sha256
description:
  Facilitate the integration of OpenAI with Composio to empower OpenAI models to directly interact with external applications, broadening their capabilities and application scope.



  - **Automate starring a GitHub repository** using conversational instructions via OpenAI Function Calls.



  Ensure you have the necessary packages installed and connect your GitHub account to allow your agents to utilize GitHub functionalities.

  ```bash

  pip install composio-openai


  composio-cli add github


  composio-cli show-apps
  ```





  Prepare your environment by initializing necessary imports from OpenAI and setting up your client.

  ```python
  from openai import OpenAI


  openai_client = OpenAI()
  ```



  This step involves fetching and integrating GitHub tools provided by Composio, enabling enhanced functionality for LangChain operations.
  ```python
  from composio_openai import App, ComposioToolSet

  toolset = ComposioToolSet()
  actions = toolset.get_tools(apps=[App.GITHUB])
  ```



  This step involves configuring and executing the agent to carry out actions, such as starring a GitHub repository.

  ```python
  my_task = "Star a repo composiohq/composio on GitHub"


  response = openai_client.chat.completions.create(model="gpt-4-turbo-preview",
      tools=actions, # Passing actions we fetched earlier.
      messages=[
              {"role": "system", "content": "You are a helpful assistant."},
              {"role": "user", "content": my_task}
          ]
      )

  pprint(response)
  ```



  Execute the following code to validate the response, ensuring that the intended task has been successfully completed.

  ```python
  result = toolset.handle_tool_calls(response)
  pprint(result)
  ```


pkgId: 25e4455b24e3fac30af87ad211e8d61244add4cdc3a21d29ae5ecc8b9b0781de
name: python-composio-phidata
arch: src
version: 0.5.35
epoch: 0
release: 1
summary: Use Composio to get an array of tools with your Phidata Plugin.
url: https://github.com/ComposioHQ/composio
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-composio-phidata-0.5.35-1.src.rpm
checksum_type: sha256
description:
  Facilitate the integration of PhiData with Composio to empower LLMs to directly interact with external applications & knowledge base, broadening their capabilities and application scope.



  - **Automate starring a GitHub repository** using conversational instructions via PhiData Tool Calls.



  Ensure you have the necessary packages installed and connect your GitHub account to allow your agents to utilize GitHub functionalities.

  ```bash

  pip install composio-phidata


  composio-cli add github


  composio-cli show-apps
  ```





  Prepare your environment by initializing necessary imports from PhiData.

  ```python
  from phi.assistant import Assistant
  ```



  This step involves fetching and integrating GitHub tools provided by Composio, enabling enhanced functionality for LangChain operations.
  ```python
  from composio_phidata import ComposioToolSet, Action

  toolset = ComposioToolSet()
  composio_tools = toolset.get_actions(actions=[Action.GITHUB_STAR_A_REPOSITORY_FOR_THE_AUTHENTICATED_USER])
  ```



  This step involves configuring and executing the assistant to carry out actions, such as starring a GitHub repository.

  ```python
  my_task = "Star a repo composiohq/composio on GitHub"


  assistant = Assistant(tools=composio_tools, show_tool_calls=True)

  assistant.print_response("Can you star ComposioHQ/composio repo?")
  ```


pkgId: 05db04ee960c78dd0000f224e8839467e1c1ac93772d4e79a2ecaac063af05e7
name: python-composio-praisonai
arch: src
version: 0.5.34
epoch: 0
release: 1
summary: Use Composio Tools to enhance your PraisonAI agents capabilities.
url: https://github.com/ComposioHQ/composio
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-composio-praisonai-0.5.34-1.src.rpm
checksum_type: sha256
description:
  Facilitate the integration of PraisonAI with Composio to empower Praison Agents to directly interact with external applications, broadening their capabilities and application scope.



  - **Automate starring a GitHub repository** using conversational instructions via PraisonAI Agents.



  Ensure you have the necessary packages installed and connect your GitHub account to allow your agents to utilize GitHub functionalities.

  ```bash

  pip install composio-praisonai


  composio-cli add github


  composio-cli show-apps
  ```





  Prepare your environment by initializing necessary imports from Praison and setting up your client.

  ```python
  import os
  import yaml
  from praisonai import PraisonAI
  ```



  This step involves fetching and integrating GitHub tools provided by Composio, and writing them in Praison supported Format, returning the name of tools in a format, that should be added to `agents.yml` file.
  ```python
  from composio_praisonai import Action, ComposioToolSet

  composio_toolset = ComposioToolSet()
  tools = composio_toolset.get_actions(
      actions=[Action.GITHUB_STAR_A_REPOSITORY_FOR_THE_AUTHENTICATED_USER]
  )

  tool_section_str = composio_toolset.get_tools_section(tools)
  print(tool_section_str)
  ```



  This step involves configuring and executing the agent to carry out actions, such as starring a GitHub repository.

  ```python
  agent_yaml = """
  framework: "crewai"
  topic: "Github Management"

  roles:
    developer:
      role: "Developer"
      goal: "An expert programmer"
      backstory: "A developer exploring new codebases and have certain tools available to execute different tasks."
      tasks:
        star_github:
          description: "Star a repo composiohq/composio on GitHub"
          expected_output: "Response whether the task was executed."
  """ + tool_section_str

  print(agent_yaml)
  ```



  Here you initialize PraisonAI class, and execute.
  ```python

  praison_ai = PraisonAI(agent_yaml=agent_yaml)


  result = praison_ai.main()


  print(result)
  ```


pkgId: 84906f5cf1696c8f22d47cc088888f4a96eb1b17429ab5bb3753452a7f6b0e9b
name: python-compoundwidgets
arch: src
version: 0.5.6
epoch: 0
release: 1
summary: Compound TTK Widgets with ttkbootstrap
url: https://github.com/AndreMariano100/CompoundWidgets.git
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-compoundwidgets-0.5.6-1.src.rpm
checksum_type: sha256
description:
  Custom widgets based on tkinter/ttk/ttkbootstrap libraries.

  To be used in conjunction with tkinter, tkinter.ttk and ttkbootstrap.

  Intends to help create widgets on the following formats:

  . label + combobox

  . label + combobox + button

  . label + entry

  . label + entry + button

  . label + entry + combobox (for engineering applications)

  . label + spinbox (including float spinbox)

  . label + text

  . label + spinbox

  Creates a set of buttons with icons for regular methods, such as 'cancel', 'clear', 'help',
  'no', 'yes', 'quit', 'save', and so on.

  Also creates new style of buttons ("led" like buttons). The following are available:

  . CheckLedButton

  . LedButton

  . RadioLedButton

  Has three frames with additional properties:

  . Collapsable frame

  . Scrollable frame

  . Vertical Collapsable frame

  Includes toplevel widgets:

  . ok/cancel message box

  . yes/no message box

  . progress bar message box

  . success message box

  . warning message box

  . tooltip message box

  . timed message box

  Includes the entry validation required methods for "numbers only" and for maximum number of characters.

  Includes autocomplete widgets, such as:

  . Autocomplete Combobox

  . Autocomplete EntryList

  . Autocomplete LabelCombo


pkgId: 223ead0e0c47abb30e42d0bc83aca4d11b50ef3c815becce2e555967e8357c3b
name: python-compreface-sdk
arch: src
version: 0.6.0
epoch: 0
release: 1
summary: CompreFace Python SDK makes face recognition into your application even easier.
url: https://exadel.com/solutions/compreface/
rpm_license: Apache License 2.0
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-compreface-sdk-0.6.0-1.src.rpm
checksum_type: sha256
description:
  CompreFace Python SDK makes face recognition into your application even easier.


  - [Requirements](#requirements)
  - [Installation](#installation)
  - [Usage](#usage)
    - [Initialization](#initialization)
    - [Adding faces into a face collection](#adding-faces-into-a-face-collection)
    - [Recognition](#recognition)
    - [Webcam demo](#webcam-demo)
  - [Reference](#reference)
    - [CompreFace Global Object](#compreFace-global-object)
      - [Methods](#methods)
    - [Options structure](#options-structure)
    - [Face Recognition Service](#face-recognition-service)
      - [Recognize Faces from a Given Image](#recognize-faces-from-a-given-image)
      - [Get Face Collection](#get-face-collection)
        - [Add an Example of a Subject](#add-an-example-of-a-subject)
        - [List of All Saved Examples of the Subject](#list-of-all-saved-examples-of-the-subject)
        - [Delete All Examples of the Subject by Name](#delete-all-examples-of-the-subject-by-name)
        - [Delete an Example of the Subject by ID](#delete-an-example-of-the-subject-by-id)
        - [Verify Faces from a Given Image](#verify-faces-from-a-given-image)
      - [Get Subjects](#get-subjects)
        - [Add a Subject](#add-a-subject)
        - [List Subjects](#list-subjects)
        - [Rename a Subject](#rename-a-subject)
        - [Delete a Subject](#delete-a-subject)
        - [Delete All Subjects](#delete-all-subjects)
      - [Face Detection Service](#face-detection-service)
        - [Detect](#detect)
      - [Face Verification Service](#face-verification-service)
        - [Verify](#verify)
  - [Contributing](#contributing)
      - [Report Bugs](#report-bugs)
      - [Submit Feedback](#submit-feedback)
  - [License info](#license-info)



  Before using our SDK make sure you have installed CompreFace and Python on your machine.

  1. [CompreFace](https://github.com/exadel-inc/CompreFace#getting-started-with-compreface)
  2. [Python](https://www.python.org/downloads/) (Version 3.7+)



  | CompreFace Python SDK version | CompreFace 0.5.x | CompreFace 0.6.x |
  | ------------------------------| ---------------- | ---------------- |
  | 0.1.0                         | ✔                | :yellow_circle:  |
  | 0.6.x                         | :yellow_circle:  | ✔                |

  Explanation:

  * ✔  SDK supports all functionality from CompreFace.
  * :yellow_circle:  SDK works with this CompreFace version.
  In case if CompreFace version is newer - SDK won't support new features of CompreFace. In case if CompreFace version is older - new SDK features will fail.
  * ✘ There are major backward compatibility issues. It is not recommended to use these versions together




  It can be installed through pip:

  ```shell
  pip install compreface-sdk
  ```



  All these examples you can find in repository inside [examples](/examples) folder.



  To start using Python SDK you need to import `CompreFace` object from 'compreface-sdk' dependency.

  Then you need to init it with `url` and `port`. By default, if you run CompreFace on your local machine, it's `http://localhost` and `8000` respectively.
  You can pass optional `options` object when call method to set default parameters, see reference for [more information](#options-structure).

  After you initialized `CompreFace` object you need to init the service object with the `api key` of your face service. You can use this service object to recognize faces.

  However, before recognizing you need first to add faces into the face collection. To do this, get the face collection object from the service object.

  ```python
  from compreface import CompreFace
  from compreface.service import RecognitionService
  from compreface.collections import FaceCollection
  from compreface.collections.face_collections import Subjects

  DOMAIN: str = 'http://localhost'
  PORT: str = '8000'
  API_KEY: str = 'your_face_recognition_key'

  compre_face: CompreFace = CompreFace(DOMAIN, PORT)

  recognition: RecognitionService = compre_face.init_face_recognition(API_KEY)

  face_collection: FaceCollection = recognition.get_face_collection()

  subjects: Subjects = recognition.get_subjects()
  ```



  Here is example that shows how to add an image to your face collection from your file system:

  ```python
  image_path: str = 'examples/common/jonathan-petit-unsplash.jpg'
  subject: str = 'Jonathan Petit'

  face_collection.add(image_path=image_path, subject=subject)
  ```



  This code snippet shows how to recognize unknown face.

  ```python
  image_path: str = 'examples/common/jonathan-petit-unsplash.jpg'

  recognition.recognize(image_path=image_path)
  ```


  Webcam demo shows how to use CompreFace Recognition and Detection services using Python SDK.
  In both cases, age, gender and mask plugins are applied.

  Follow this [link](/webcam_demo) to see the instructions.





  Global CompreFace Object is used for initializing connection to CompreFace and setting default values for options.
  Default values will be used in every service method if applicable.
  If the option’s value is set in the global object and passed as a function argument then the function argument value will be used.

  **Constructor:**

  ```CompreFace(domain, port, options)```

  | Argument | Type   | Required | Notes                                     |
  | ---------| ------ | -------- | ----------------------------------------- |
  | url      | string | required | URL with protocol where CompreFace is located. E.g. `http://localhost` |
  | port     | string | required | CompreFace port. E.g. `8000` |
  | options  | object | optional | Default values for face recognition services. See more [here](#options-structure). `AllOptionsDict` object can be used in this method   |

  Possible options:

  | Option              | Type    | Notes                                     |
  | --------------------| ------  | ----------------------------------------- |
  | det_prob_threshold  | string  | minimum required confidence that a recognized face is actually a face. Value is between 0.0 and 1.0 |
  | limit               | integer | maximum number of faces on the image to be recognized. It recognizes the biggest faces first. Value of 0 represents no limit. Default value: 0       |
  | prediction_count    | integer | maximum number of subject predictions per face. It returns the most similar subjects. Default value: 1    |
  | face_plugins        | string  | comma-separated slugs of face plugins. If empty, no additional information is returned. [Learn more](https://github.com/exadel-inc/CompreFace/tree/master/docs/Face-services-and-plugins.md)    |
  | status              | boolean | if true includes system information like execution_time and plugin_version fields. Default value is false    |

  Example:

  ```python
  from compreface import CompreFace

  DOMAIN: str = 'http://localhost'
  PORT: str = '8000'

  compre_face: CompreFace = CompreFace(domain=DOMAIN, port=PORT, options={
      "limit": 0,
      "det_prob_threshold": 0.8,
      "prediction_count": 1,
      "face_plugins": "calculator,age,gender,landmarks",
      "status": "true"
  })
  ```



  1. ```CompreFace.init_face_recognition(api_key)```

  Inits face recognition service object.

  | Argument | Type   | Required | Notes                                     |
  | ---------| ------ | -------- | ----------------------------------------- |
  | api_key  | string | required | Face Recognition Api Key in UUID format    |

  Example:

  ```python
  from compreface.service import RecognitionService

  API_KEY: str = 'your_face_recognition_key'

  recognition: RecognitionService = compre_face.init_face_recognition(API_KEY)
  ```

  2. ```CompreFace.init_face_detection(api_key)```

  Inits face detection service object.

  | Argument | Type   | Required | Notes                                     |
  | ---------| ------ | -------- | ----------------------------------------- |
  | api_key  | string | required | Face Detection Api Key in UUID format    |

  Example:

  ```python
  from compreface.service import DetectionService

  DETECTION_API_KEY: str = 'your_face_detection_key'

  detection: DetectionService = compre_face.init_face_detection(DETECTION_API_KEY)
  ```

  3. ```CompreFace.init_face_verification(api_key)```

  Inits face verification service object.

  | Argument | Type   | Required | Notes                                     |
  | ---------| ------ | -------- | ----------------------------------------- |
  | api_key  | string | required | Face Verification Api Key in UUID format    |

  Example:

  ```python
  from compreface.service import VerificationService

  VERIFICATION_API_KEY: str = 'your_face_verification_key'

  verify: VerificationService = compre_face.init_face_verification(VERIFICATION_API_KEY)
  ```



  Options is optional field in every request that contains an image.
  If the option’s value is set in the global object and passed as a function argument then the function argument value will be used.

  ```python

  class DetProbOptionsDict(TypedDict):
      det_prob_threshold: float


  class ExpandedOptionsDict(DetProbOptionsDict):
      limit: int
      status: bool
      face_plugins: str


  class AllOptionsDict(ExpandedOptionsDict):
      prediction_count: int

  ```
  | Option              | Type    | Notes                                     |
  | --------------------| ------  | ----------------------------------------- |
  | det_prob_threshold  | string  | minimum required confidence that a recognized face is actually a face. Value is between 0.0 and 1.0 |
  | limit               | integer | maximum number of faces on the image to be recognized. It recognizes the biggest faces first. Value of 0 represents no limit. Default value: 0       |
  | prediction_count    | integer | maximum number of subject predictions per face. It returns the most similar subjects. Default value: 1    |
  | face_plugins        | string  | comma-separated slugs of face plugins. If empty, no additional information is returned. [Learn more](https://github.com/exadel-inc/CompreFace/tree/master/docs/Face-services-and-plugins.md)    |
  | status              | boolean | if true includes system information like execution_time and plugin_version fields. Default value is false    |

  Example of face recognition with object:

  ```python
  recognition.recognize(image_path=image_path, options={
      "limit": 0,
      "det_prob_threshold": 0.8,
      "prediction_count": 1,
      "face_plugins": "calculator,age,gender,landmarks",
      "status": "true"
  })
  ```



  Face recognition service is used for face identification.
  This means that you first need to upload known faces to face collection and then recognize unknown faces among them.
  When you upload an unknown face, the service returns the most similar faces to it.
  Also, face recognition service supports verify endpoint to check if this person from face collection is the correct one.
  For more information, see [CompreFace page](https://github.com/exadel-inc/CompreFace).



  *[Example](examples/recognize_face_from_image.py)*

  Recognizes all faces from the image.
  The first argument is the image location, it can be an url, local path or bytes.

  ```python
  recognition.recognize(image_path, options)
  ```

  | Argument           | Type    | Required | Notes                                                                                                                                          |
  | ------------------ | ------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
  | image_path         | image   | required | Image can pass from url, local path or bytes. Max size is 5Mb                                                         |
  | options            | object  | optional | `AllOptionsDict` object can be used in this method. See more [here](#options-structure).  |

  Response:

  ```json
  {
    "result" : [ {
      "age" : {
        "probability": 0.9308982491493225,
        "high": 32,
        "low": 25
      },
      "gender" : {
        "probability": 0.9898611307144165,
        "value": "female"
      },
      "mask" : {
        "probability": 0.9999470710754395,
        "value": "without_mask"
      },
      "embedding" : [ 9.424854069948196E-4, "...", -0.011415496468544006 ],
      "box" : {
        "probability" : 1.0,
        "x_max" : 1420,
        "y_max" : 1368,
        "x_min" : 548,
        "y_min" : 295
      },
      "landmarks" : [ [ 814, 713 ], [ 1104, 829 ], [ 832, 937 ], [ 704, 1030 ], [ 1017, 1133 ] ],
      "subjects" : [ {
        "similarity" : 0.97858,
        "subject" : "subject1"
      } ],
      "execution_time" : {
        "age" : 28.0,
        "gender" : 26.0,
        "detector" : 117.0,
        "calculator" : 45.0,
        "mask": 36.0
      }
    } ],
    "plugins_versions" : {
      "age" : "agegender.AgeDetector",
      "gender" : "agegender.GenderDetector",
      "detector" : "facenet.FaceDetector",
      "calculator" : "facenet.Calculator",
      "mask": "facemask.MaskDetector"
    }
  }
  ```

  | Element                    | Type    | Description                                                                                                                                                 |
  | -------------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
  | age                        | object  | detected age range. Return only if [age plugin](https://github.com/exadel-inc/CompreFace/blob/master/docs/Face-services-and-plugins.md) is enabled                                                       |
  | gender                     | object  | detected gender. Return only if [gender plugin](https://github.com/exadel-inc/CompreFace/blob/master/docs/Face-services-and-plugins.md) is enabled                                                       |
  | mask                       | object  | detected mask. Return only if [face mask plugin](https://github.com/exadel-inc/CompreFace/blob/master/docs/Face-services-and-plugins.md) is enabled.          |
  | embedding                  | array   | face embeddings. Return only if [calculator plugin](https://github.com/exadel-inc/CompreFace/blob/master/docs/Face-services-and-plugins.md) is enabled                                                   |
  | box                        | object  | list of parameters of the bounding box for this face                                                                                                        |
  | probability                | float   | probability that a found face is actually a face                                                                                                            |
  | x_max, y_max, x_min, y_min | integer | coordinates of the frame containing the face                                                                                                                |
  | landmarks                  | array   | list of the coordinates of the frame containing the face-landmarks.|
  | subjects                   | list    | list of similar subjects with size of <prediction_count> order by similarity                                                                                |
  | similarity                 | float   | similarity that on that image predicted person                                                                                                              |
  | subject                    | string  | name of the subject in Face Collection                                                                                                                      |
  | execution_time             | object  | execution time of all plugins                                                                                                                               |
  | plugins_versions           | object  | contains information about plugin versions                                                                                                                  |



  ```python
  recognition.get_face_collection()
  ```

  Returns Face collection object

  Face collection could be used to manage known faces, e.g. add, list, or delete them.

  Face recognition is performed for the saved known faces in face collection, so before using the `recognize` method you need to save at least one face into the face collection.

  More information about face collection and managing examples [here](https://github.com/exadel-inc/CompreFace/blob/master/docs/Rest-API-description.md#managing-subject-examples)

  **Methods:**



  *[Example](examples/add_example_of_a_subject.py)*

  This creates an example of the subject by saving images. You can add as many images as you want to train the system. Image should
  contain only one face.

  ```python
  face_collection.add(image_path, subject, options)
  ```

  | Argument           | Type   | Required | Notes                                                                                                |
  | ------------------ | ------ | -------- | ---------------------------------------------------------------------------------------------------- |
  | image_path         | image  | required | Image can pass from url, local path or bytes. Max size is 5Mb               |
  | subject            | string | required | is the name you assign to the image you save                                                         |
  | options            | object | optional | `DetProbOptionsDict` object can be used in this method. See more [here](#options-structure).  |

  Response:

  ```json
  {
    "image_id": "6b135f5b-a365-4522-b1f1-4c9ac2dd0728",
    "subject": "subject1"
  }
  ```

  | Element  | Type   | Description                |
  | -------- | ------ | -------------------------- |
  | image_id | UUID   | UUID of uploaded image     |
  | subject  | string | Subject of the saved image |



  To retrieve a list of subjects saved in a Face Collection:

  ```python
  face_collection.list()
  ```

  Response:

  ```
  {
    "faces": [
      {
        "image_id": <image_id>,
        "subject": <subject>
      },
      ...
    ]
  }
  ```

  | Element  | Type   | Description                                                       |
  | -------- | ------ | ----------------------------------------------------------------- |
  | image_id | UUID   | UUID of the face                                                  |
  | subject  | string | <subject> of the person, whose picture was saved for this api key |



  *[Example](examples/delete_all_examples_of_subject.py)*

  To delete all image examples of the <subject>:

  ```python
  face_collection.delete_all(subject)
  ```

  | Argument  | Type   | Required | Notes                                                                                                                                |
  | --------- | ------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------ |
  | subject   | string | optional | is the name you assign to the image you save. If this parameter is absent, all faces in Face Collection will be removed |

  Response:

  ```
  {
      "deleted": <count>
  }
  ```

  | Element  | Type    | Description              |
  | -------- | ------- | ------------------------ |
  | deleted  | integer | Number of deleted faces  |



  *[Example](examples/delete_example_by_id.py)*

  To delete an image by ID:

  ```python
  face_collection.delete(image_id)
  ```
  | Argument  | Type   | Required | Notes                                                        |
  | --------- | ------ | -------- | ------------------------------------------------------------
  | image_id  | UUID   | required | UUID of the removing face                                    |

  Response:

  ```
  {
    "image_id": <image_id>,
    "subject": <subject>
  }
  ```

  | Element  | Type   | Description                                                       |
  | -------- | ------ | ----------------------------------------------------------------- |
  | image_id | UUID   | UUID of the removed face                                          |
  | subject  | string | <subject> of the person, whose picture was saved for this api key |



  *[Example](examples/verification_face_from_image.py)*

  ```python
  face_collection.verify(image_path, image_id, options)
  ```

  Compares similarities of given image with image from your face collection.


  | Argument           | Type    | Required | Notes                                                                                                                                                 |
  | ------------------ | ------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
  | image_path         | image   | required | Image can pass from url, local path or bytes. Max size is 5Mb                                                                |
  | image_id           | UUID    | required | UUID of the verifying face                                                                                                                            |
  | options            | string  | Object   | `ExpandedOptionsDict` object can be used in this method. See more [here](#options-structure).  |

  Response:

  ```json
  {
    "result" : [ {
      "age" : {
        "probability": 0.9308982491493225,
        "high": 32,
        "low": 25
      },
      "gender" : {
        "probability": 0.9898611307144165,
        "value": "female"
      },
      "mask" : {
        "probability": 0.9999470710754395,
        "value": "without_mask"
      },
      "embedding" : [ 9.424854069948196E-4, "...", -0.011415496468544006 ],
      "box" : {
        "probability" : 1.0,
        "x_max" : 1420,
        "y_max" : 1368,
        "x_min" : 548,
        "y_min" : 295
      },
      "landmarks" : [ [ 814, 713 ], [ 1104, 829 ], [ 832, 937 ], [ 704, 1030 ], [ 1017, 1133 ] ],
      "subjects" : [ {
        "similarity" : 0.97858,
        "subject" : "subject1"
      } ],
      "execution_time" : {
        "age" : 28.0,
        "gender" : 26.0,
        "detector" : 117.0,
        "calculator" : 45.0,
        "mask": 36.0
      }
    } ],
    "plugins_versions" : {
      "age" : "agegender.AgeDetector",
      "gender" : "agegender.GenderDetector",
      "detector" : "facenet.FaceDetector",
      "calculator" : "facenet.Calculator",
      "mask": "facemask.MaskDetector"
    }
  }
  ```

  | Element                        | Type    | Description                                                  |
  | ------------------------------ | ------- | ------------------------------------------------------------ |
  | age                            | object  | detected age range. Return only if [age plugin](https://github.com/exadel-inc/CompreFace/tree/master/docs/Face-services-and-plugins.md#face-plugins) is enabled         |
  | gender                         | object  | detected gender. Return only if [gender plugin](https://github.com/exadel-inc/CompreFace/tree/master/docs/Face-services-and-plugins.md#face-plugins) is enabled         |
  | mask                           | object  | detected mask. Return only if [face mask plugin](https://github.com/exadel-inc/CompreFace/blob/master/docs/Face-services-and-plugins.md) is enabled.          |
  | embedding                      | array   | face embeddings. Return only if [calculator plugin](https://github.com/exadel-inc/CompreFace/tree/master/docs/Face-services-and-plugins.md#face-plugins) is enabled      |
  | box                            | object  | list of parameters of the bounding box for this face         |
  | probability                    | float   | probability that a found face is actually a face             |
  | x_max, y_max, x_min, y_min     | integer | coordinates of the frame containing the face                 |
  | landmarks                      | array   | list of the coordinates of the frame containing the face-landmarks. Return only if [landmarks plugin](https://github.com/exadel-inc/CompreFace/tree/master/docs/Face-services-and-plugins.md#face-plugins) is enabled      |
  | similarity                     | float   | similarity that on that image predicted person               |
  | execution_time                 | object  | execution time of all plugins                       |
  | plugins_versions               | object  | contains information about plugin versions                       |



  ```python
  recognition.get_subjects()
  ```

  Returns subjects object

  Subjects object allows working with subjects directly (not via subject examples).

  More information about subjects [here](https://github.com/exadel-inc/CompreFace/blob/master/docs/Rest-API-description.md#managing-subjects)

  **Methods:**



  *[Example](examples/add_subject.py)*

  Create a new subject in Face Collection.
  ```python
  subjects.add(subject)
  ```

  | Argument           | Type   | Required | Notes                                                                   |
  | ------------------ | ------ | -------- | ------------------------------------------------------------------------|
  | subject            | string | required | is the name of the subject. It can be any string                        |

  Response:

  ```json
  {
    "subject": "subject1"
  }
  ```

  | Element  | Type   | Description                |
  | -------- | ------ | -------------------------- |
  | subject  | string | is the name of the subject |



  *[Example](examples/get_list_of_all_subjects.py)*

  Returns all subject related to Face Collection.
  ```python
  subjects.list()
  ```

  Response:

  ```json
  {
    "subjects": [
      "<subject_name1>",
      "<subject_name2>"
    ]
  }
  ```

  | Element  | Type   | Description                |
  | -------- | ------ | -------------------------- |
  | subjects | array  | the list of subjects in Face Collection |



  *[Example](examples/update_existing_subject.py)*

  Rename existing subject. If a new subject name already exists, subjects are merged - all faces from the old subject name are reassigned to the subject with the new name, old subject removed.

  ```python
  subjects.rename(subject, new_name)
  ```

  | Argument            | Type   | Required | Notes                                                                   |
  | ------------------  | ------ | -------- | ------------------------------------------------------------------------|
  | subject             | string | required | is the name of the subject that will be updated                         |
  | new_name            | string | required | is the name of the subject. It can be any string                        |

  Response:

  ```json
  {
    "updated": "true|false"
  }
  ```

  | Element  | Type    | Description                |
  | -------- | ------  | -------------------------- |
  | updated  | boolean | failed or success          |



  *[Example](examples/delete_subject_by_name.py)*

  Delete existing subject and all saved faces.
  ```python
  subjects.delete(subject)
  ```

  | Argument           | Type   | Required | Notes                                                                   |
  | ------------------ | ------ | -------- | ------------------------------------------------------------------------|
  | subject            | string | required | is the name of the subject.                                             |

  Response:

  ```json
  {
    "subject": "subject1"
  }
  ```

  | Element  | Type   | Description                |
  | -------- | ------ | -------------------------- |
  | subject  | string | is the name of the subject |



  *[Example](examples/delete_all_subjects.py)*

  Delete all existing subjects and all saved faces.
  ```python
  subjects.delete_all()
  ```

  Response:

  ```json
  {
    "deleted": "<count>"
  }
  ```

  | Element  | Type    | Description                |
  | -------- | ------  | -------------------------- |
  | deleted  | integer | number of deleted subjects |




  Face detection service is used for detecting faces in the image.

  **Methods:**



  *[Example](examples/detect_face_from_image.py)*

  ```python
  detection.detect(image_path, options)
  ```

  Finds all faces on the image.

  | Argument          | Type    | Required | Notes                                                                                                                                          |
  | ----------------- | ------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
  | image_path        | image   | required | image where to detect faces. Image can pass from url, local path or bytes. Max size is 5Mb                            |
  | options           | string  | Object   | `ExpandedOptionsDict` object can be used in this method. See more [here](#options-structure).  |

  Response:

  ```json
  {
    "result" : [ {
      "age" : {
        "probability": 0.9308982491493225,
        "high": 32,
        "low": 25
      },
      "gender" : {
        "probability": 0.9898611307144165,
        "value": "female"
      },
      "mask" : {
        "probability": 0.9999470710754395,
        "value": "without_mask"
      },
      "embedding" : [ -0.03027934394776821, "...", -0.05117142200469971 ],
      "box" : {
        "probability" : 0.9987509250640869,
        "x_max" : 376,
        "y_max" : 479,
        "x_min" : 68,
        "y_min" : 77
      },
      "landmarks" : [ [ 156, 245 ], [ 277, 253 ], [ 202, 311 ], [ 148, 358 ], [ 274, 365 ] ],
      "execution_time" : {
        "age" : 30.0,
        "gender" : 26.0,
        "detector" : 130.0,
        "calculator" : 49.0,
        "mask": 36.0
      }
    } ],
    "plugins_versions" : {
      "age" : "agegender.AgeDetector",
      "gender" : "agegender.GenderDetector",
      "detector" : "facenet.FaceDetector",
      "calculator" : "facenet.Calculator",
      "mask": "facemask.MaskDetector"
    }
  }
  ```

  | Element                        | Type    | Description                                                  |
  | ------------------------------ | ------- | ------------------------------------------------------------ |
  | age                            | object  | detected age range. Return only if [age plugin](https://github.com/exadel-inc/CompreFace/tree/master/docs/Face-services-and-plugins.md#face-plugins) is enabled         |
  | gender                         | object  | detected gender. Return only if [gender plugin](https://github.com/exadel-inc/CompreFace/tree/master/docs/Face-services-and-plugins.md#face-plugins) is enabled         |
  | mask                           | object  | detected mask. Return only if [face mask plugin](https://github.com/exadel-inc/CompreFace/blob/master/docs/Face-services-and-plugins.md) is enabled.          |
  | embedding                      | array   | face embeddings. Return only if [calculator plugin](https://github.com/exadel-inc/CompreFace/tree/master/docs/Face-services-and-plugins.md#face-plugins) is enabled      |
  | box                            | object  | list of parameters of the bounding box for this face (on processedImage) |
  | probability                    | float   | probability that a found face is actually a face (on processedImage)     |
  | x_max, y_max, x_min, y_min     | integer | coordinates of the frame containing the face (on processedImage)         |
  | landmarks                      | array   | list of the coordinates of the frame containing the face-landmarks. Return only if [landmarks plugin](https://github.com/exadel-inc/CompreFace/tree/master/docs/Face-services-and-plugins.md#face-plugins) is enabled      |
  | execution_time                 | object  | execution time of all plugins                       |
  | plugins_versions               | object  | contains information about plugin versions                       |




  *[Example](examples/verify_face_from_image.py)*

  Face verification service is used for comparing two images.
  A source image should contain only one face which will be compared to all faces on the target image.

  **Methods:**



  ```python
  verify.verify(source_image_path, target_image_path, options)
  ```

  Compares two images provided in arguments. Source image should contain only one face, it will be compared to all faces in the target image.

  | Argument            | Type    | Required | Notes                                                                                                                                                 |
  | ------------------  | ------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
  | image_id            | UUID    | required | UUID of the verifying face                                                                                                                            |
  | source_image_path   | image   | required | file to be verified. Image can pass from url, local path or bytes. Max size is 5Mb                                           |
  | target_image_path   | image   | required | reference file to check the source file. Image can pass from url, local path or bytes. Max size is 5Mb                       |
  | options             | string  | Object   | `ExpandedOptionsDict` object can be used in this method. See more [here](#options-structure).  |

  Response:

  ```json
  {
    "result" : [{
      "source_image_face" : {
        "age" : {
          "probability": 0.9308982491493225,
          "high": 32,
          "low": 25
        },
        "gender" : {
          "probability": 0.9898611307144165,
          "value": "female"
        },
        "mask" : {
          "probability": 0.9999470710754395,
          "value": "without_mask"
        },
        "embedding" : [ -0.0010271212086081505, "...", -0.008746841922402382 ],
        "box" : {
          "probability" : 0.9997453093528748,
          "x_max" : 205,
          "y_max" : 167,
          "x_min" : 48,
          "y_min" : 0
        },
        "landmarks" : [ [ 92, 44 ], [ 130, 68 ], [ 71, 76 ], [ 60, 104 ], [ 95, 125 ] ],
        "execution_time" : {
          "age" : 85.0,
          "gender" : 51.0,
          "detector" : 67.0,
          "calculator" : 116.0,
          "mask": 36.0
        }
      },
      "face_matches": [
        {
          "age" : {
            "probability": 0.9308982491493225,
            "high": 32,
            "low": 25
          },
          "gender" : {
            "probability": 0.9898611307144165,
            "value": "female"
          },
          "mask" : {
            "probability": 0.9999470710754395,
            "value": "without_mask"
          },
          "embedding" : [ -0.049007344990968704, "...", -0.01753818802535534 ],
          "box" : {
            "probability" : 0.99975,
            "x_max" : 308,
            "y_max" : 180,
            "x_min" : 235,
            "y_min" : 98
          },
          "landmarks" : [ [ 260, 129 ], [ 273, 127 ], [ 258, 136 ], [ 257, 150 ], [ 269, 148 ] ],
          "similarity" : 0.97858,
          "execution_time" : {
            "age" : 59.0,
            "gender" : 30.0,
            "detector" : 177.0,
            "calculator" : 70.0,
            "mask": 36.0
          }
        }],
      "plugins_versions" : {
        "age" : "agegender.AgeDetector",
        "gender" : "agegender.GenderDetector",
        "detector" : "facenet.FaceDetector",
        "calculator" : "facenet.Calculator",
        "mask": "facemask.MaskDetector"
      }
    }]
  }
  ```

  | Element                        | Type    | Description                                                  |
  | ------------------------------ | ------- | ------------------------------------------------------------ |
  | source_image_face              | object  | additional info about source image face |
  | face_matches                   | array   | result of face verification |
  | age                            | object  | detected age range. Return only if [age plugin](https://github.com/exadel-inc/CompreFace/tree/master/docs/Face-services-and-plugins.md#face-plugins) is enabled         |
  | gender                         | object  | detected gender. Return only if [gender plugin](https://github.com/exadel-inc/CompreFace/tree/master/docs/Face-services-and-plugins.md#face-plugins) is enabled         |
  | mask                           | object  | detected mask. Return only if [face mask plugin](https://github.com/exadel-inc/CompreFace/blob/master/docs/Face-services-and-plugins.md) is enabled.          |
  | embedding                      | array   | face embeddings. Return only if [calculator plugin](https://github.com/exadel-inc/CompreFace/tree/master/docs/Face-services-and-plugins.md#face-plugins) is enabled      |
  | box                            | object  | list of parameters of the bounding box for this face         |
  | probability                    | float   | probability that a found face is actually a face             |
  | x_max, y_max, x_min, y_min     | integer | coordinates of the frame containing the face                 |
  | landmarks                      | array   | list of the coordinates of the frame containing the face-landmarks. Return only if [landmarks plugin](https://github.com/exadel-inc/CompreFace/tree/master/docs/Face-services-and-plugins.md#face-plugins) is enabled      |
  | similarity                     | float   | similarity between this face and the face on the source image               |
  | execution_time                 | object  | execution time of all plugins                       |
  | plugins_versions               | object  | contains information about plugin versions                       |



  Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are greatly appreciated.

  1. Fork the Project
  2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`)
  3. Commit your Changes (`git commit -m 'Add some AmazingFeature'`)
  4. Push to the Branch (`git push origin feature/AmazingFeature`)
  5. Open a Pull Request

  After creating your first contributing pull request, you will receive a request to sign our Contributor License Agreement by commenting your pull request with a special message.



  Please report any bugs [here](https://github.com/exadel-inc/compreface-python-sdk/issues).

  If you are reporting a bug, please specify:

  - Your operating system name and version
  - Any details about your local setup that might be helpful in troubleshooting
  - Detailed steps to reproduce the bug



  The best way to send us feedback is to file an issue at https://github.com/exadel-inc/compreface-python-sdk/issues.

  If you are proposing a feature, please:

  - Explain in detail how it should work.
  - Keep the scope as narrow as possible to make it easier to implement.



  CompreFace Python SDK is open-source facial recognition SDK released under the [Apache 2.0 license](https://www.apache.org/licenses/LICENSE-2.0.html).


pkgId: ee80af7c833e1088e2ddd50f2f255b404a319ef3d9a7dddf51207db219d4090d
name: python-compreffor
arch: src
version: 0.5.5
epoch: 0
release: 1
summary: A CFF subroutinizer for fontTools.
url: https://pypi.org/project/compreffor/
rpm_license: Apache 2.0
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-compreffor-0.5.5-1.src.rpm
checksum_type: sha256
description:
  |CI Build Status|

  A CFF table subroutinizer for FontTools.

  .. |CI Build Status| image:: https://github.com/googlefonts/compreffor/workflows/Build%20+%20Deploy/badge.svg
     :target: https://github.com/googlefonts/compreffor/actions?query=workflow%3A%22Build+%2B+Deploy%22


pkgId: b8d88b6fc86582d1bb7add8dfac5a55296419915dd99c94a3ce7761bba280a9a
name: python-compress-comics-TheHardew
arch: src
version: 0.4.12
epoch: 0
release: 1
summary: Compress images in cbz/cbr files with jpegxl, fast.
url: https://github.com/TheHardew/compress_comics
rpm_license: The Unlicense (Unlicense)
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-compress-comics-TheHardew-0.4.12-1.src.rpm
checksum_type: sha256
description:
  Find all the cbz/cbr files in the current directory and subdirectories and compress all the jpg/png/gif images inside with jpeg xl.
  Output files preserve the folder structure.
  Repacks cbr into cbz.

  By default, compresses with:
  ```
  -d / --distance 0 (lossless)
  -j / --lossless_jpeg 1
  ```

  Compresses as many images in parallel as there are threads.
  Because of that, it may run out of memory on bigger resolutions.
  Adjust with:
  ```
  -t / --threads
  ```

  Basic usage:
  `compress_comics output_directory`

  To check all program options:
  `compress_comics -h`

  Needs `cjxl` installed and in PATH.


pkgId: 9227b860682554af4dc8c8b92b6d8f37c7b7dbe708f3399c6c5254d6c3138f23
name: python-compress-fasttext
arch: src
version: 0.1.5
epoch: 0
release: 1
summary: A set of tools to compress gensim fasttext models
url: https://github.com/avidale/compress-fasttext
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-compress-fasttext-0.1.5-1.src.rpm
checksum_type: sha256
description:
  This Python 3 package allows to compress fastText word embedding models
  (from the `gensim` package) by orders of magnitude,
  without significantly affecting their quality.

  [Here](https://github.com/avidale/compress-fasttext/releases/tag/gensim-4-draft) are some links to the models that have already been compressed.

  This [blogpost in Russian](https://habr.com/ru/post/489474)
  and [this one in English](https://towardsdatascience.com/eb212e9919ca)
  give more details about the motivation and
  methods for compressing fastText models.


  **Note: gensim==4.0.0 has introduced some backward-incompatible changes:**
  * With gensim<4.0.0, please use compress-fasttext<=0.0.7
  (and optionally Russian models from [our first release](https://github.com/avidale/compress-fasttext/releases/tag/v0.0.1)).
  * With gensim>=4.0.0, please use compress-fasttext>=0.1.0
  (and optionally Russian or English models from [our 0.1.0 release](https://github.com/avidale/compress-fasttext/releases/tag/gensim-4-draft)).
  * Some models are no longer supported in the new version of gensim+compress-fasttext
    (for example, multiple models from [RusVectores](https://rusvectores.org/ru/models/) that use `compatible_hash=False`).
  * For any particular model, compatibility should be determined experimentally.
    If you notice any strange behaviour, please report in the Github issues.


  The package can be installed with `pip`:
  ```commandline
  pip install compress-fasttext[full]
  ```
  If you are not going to perform matrix decomposition or quantization,
   you can install a variety with less dependencies:
  ```commandline
  pip install compress-fasttext
  ```


  You can use this package to compress your own fastText model (or one downloaded e.g. from
  [RusVectores](https://rusvectores.org/ru/models/)):

  Compress a model in Gensim format:
  ```python
  import gensim
  import compress_fasttext
  big_model = gensim.models.fasttext.FastTextKeyedVectors.load('path-to-original-model')
  small_model = compress_fasttext.prune_ft_freq(big_model, pq=True)
  small_model.save('path-to-new-model')
  ```

  Import a model in Facebook original format and compress it:
  ```python
  from gensim.models.fasttext import load_facebook_model
  import compress_fasttext
  big_model = load_facebook_model('path-to-original-model').wv
  small_model = compress_fasttext.prune_ft_freq(big_model, pq=True)
  small_model.save('path-to-new-model')
  ```
  To perform this compression, you will need to `pip install gensim==3.8.3 sklearn` beforehand.

  Different compression methods include:
  - matrix decomposition (`svd_ft`)
  - product quantization (`quantize_ft`)
  - optimization of feature hashing (`prune_ft`)
  - feature selection (`prune_ft_freq`)

  The recommended approach is combination of feature selection and quantization (`prune_ft_freq` with `pq=True`).


  If you just need a tiny fastText model for Russian, you can download
  [this](https://github.com/avidale/compress-fasttext/releases/download/gensim-4-draft/geowac_tokens_sg_300_5_2020-100K-20K-100.bin)
  21-megabyte model. It's a compressed version of
  [geowac_tokens_none_fasttextskipgram_300_5_2020](http://vectors.nlpl.eu/repository/20/214.zip) model
  from [RusVectores](https://rusvectores.org/ru/models/).

  If `compress-fasttext` is already installed, you can download and use this tiny model
  ```python
  import compress_fasttext
  small_model = compress_fasttext.models.CompressedFastTextKeyedVectors.load(
      'https://github.com/avidale/compress-fasttext/releases/download/gensim-4-draft/geowac_tokens_sg_300_5_2020-100K-20K-100.bin'
  )
  print(small_model['спасибо'])

  print(small_model.most_similar('котенок'))

  ```
  The class `CompressedFastTextKeyedVectors` inherits from `gensim.models.fasttext.FastTextKeyedVectors`,
  but makes a few additional optimizations.

  For English, you can use [this](https://github.com/avidale/compress-fasttext/releases/download/v0.0.4/cc.en.300.compressed.bin) tiny model,
  obtained by compressing [the model by Facebook](https://fasttext.cc/docs/en/crawl-vectors.html).

  ```python
  import compress_fasttext
  small_model = compress_fasttext.models.CompressedFastTextKeyedVectors.load(
      'https://github.com/avidale/compress-fasttext/releases/download/v0.0.4/cc.en.300.compressed.bin'
  )
  print(small_model['hello'])

  print(small_model.most_similar('Python'))

  ```

  More compressed models for 101 various languages can be found at https://zenodo.org/record/4905385.



  In practical applications, you usually feed fastText embeddings to some other model.
  The class `FastTextTransformer` uses [the scikit-learn interface](https://scikit-learn.org/stable/data_transforms.html)
  and represents a text as the average of the embedding of its words.
  With it you can, for example, train a classifier on top of fastText
  to tell edible things from inedible ones:

  ```python
  import compress_fasttext
  from sklearn.pipeline import make_pipeline
  from sklearn.linear_model import LogisticRegression
  from compress_fasttext.feature_extraction import FastTextTransformer

  small_model = compress_fasttext.models.CompressedFastTextKeyedVectors.load(
      'https://github.com/avidale/compress-fasttext/releases/download/v0.0.4/cc.en.300.compressed.bin'
  )

  classifier = make_pipeline(
      FastTextTransformer(model=small_model),
      LogisticRegression()
  ).fit(
      ['banana', 'soup', 'burger', 'car', 'tree', 'city'],
      [1, 1, 1, 0, 0, 0]
  )
  classifier.predict(['jet', 'train', 'cake', 'apple'])

  ```


  This code is heavily based on the [navec](https://github.com/natasha/navec) package by Alexander Kukushkin and
  [the blogpost](https://medium.com/@vasnetsov93/shrinking-fasttext-embeddings-so-that-it-fits-google-colab-cd59ab75959e)
  by Andrey Vasnetsov about shrinking fastText embeddings.


pkgId: 018b9bdeb3fe5d5b190fb349bbe22b8405d95364b6f96d81de6b516a63145b05
name: python-compress-json
arch: src
version: 1.1.0
epoch: 0
release: 1
summary: The missing Python utility to read and write large compressed JSONs.
url: https://github.com/LucaCappelletti94/compress_json
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-compress-json-1.1.0-1.src.rpm
checksum_type: sha256
description:
  [![pip](https://badge.fury.io/py/compress-json.svg)](https://pypi.org/project/compress-json/)
  [![python](https://img.shields.io/pypi/pyversions/compress-json.svg)](https://pypi.org/project/compress-json/)
  [![license](https://img.shields.io/pypi/l/compress-json.svg)](https://pypi.org/project/compress-json/)
  [![downloads](https://pepy.tech/badge/compress-json)](https://pepy.tech/project/compress-json)
  [![Github Actions](https://github.com/LucaCappelletti94/ugly_csv_generator/actions/workflows/python.yml/badge.svg)](https://github.com/LucaCappelletti94/ugly_csv_generator/actions/)
  [![Codacy Badge](https://app.codacy.com/project/badge/Grade/6aa4b62b4ed34f7d8e2c37ef09848294)](https://app.codacy.com/gh/LucaCappelletti94/compress_json/dashboard?utm_source=gh&utm_medium=referral&utm_content=&utm_campaign=Badge_grade)

  The missing Python utility to read and write large compressed JSONs.

  The library is loosely based on the [compress_pickle](https://github.com/lucianopaz/compress_pickle) library.



  As usual, just download it using pip:

  ```shell
  pip install compress_json
  ```



  The compression modes, detected automatically by the file name, are **gzip**, **bz2**, and **lzma** (or **xz**), with the notable exception of **zip** which seems difficult to integrate into the JSON pipeline.



  The library is extremely easy to use:

  ```python
  import compress_json

  D = {
      "A": {
          "B": "C"
      }
  }
  compress_json.dump(D, "filepath.json.gz")   # for a gzip file
  compress_json.dump(D, "filepath.json.bz")   # for a bz2 file
  compress_json.dump(D, "filepath.json.lzma") # for a lzma file
  compress_json.dump(D, "filepath.json.xz") # for a lzma file

  D1 = compress_json.load("filepath.json.gz")   # for loading a gzip file
  D2 = compress_json.load("filepath.json.bz")   # for loading a bz2 file
  D3 = compress_json.load("filepath.json.lzma") # for loading a lzma file
  D3 = compress_json.load("filepath.json.xz") # for loading a lzma file
  ```

  If it happens that you have to load or dump a JSON object with a custom extension, you can specify the compression mode by passing the `compression` parameter to the `load` and `dump` methods:

  ```python
  import compress_json

  D = {
      "A": {
          "B": "C"
      }
  }

  compress_json.dump(D, "filepath.custom_extension1", compression="gzip")   # for a gzip file
  compress_json.dump(D, "filepath.custom_extension2", compression="bz2")   # for a bz2 file
  compress_json.dump(D, "filepath.custom_extension3", compression="lzma") # for a lzma file

  D1 = compress_json.load("filepath.custom_extension1", compression="gzip")   # for loading a gzip file
  D2 = compress_json.load("filepath.custom_extension2", compression="bz2")   # for loading a bz2 file
  D3 = compress_json.load("filepath.custom_extension3", compression="lzma") # for loading a lzma file

  assert D == D1 == D2 == D3
  ```



  The library makes available, other than the usual `load` and `dump` from the JSON library, the methods `local_load` and `local_dump`, which let you load and dump files in the same directory as wherever you are calling them, by using the call stack.

  This can be useful, especially when loading files within packages.

  ```python
  import compress_json

  D = {
      "A": {
          "B": "C"
      }
  }
  compress_json.local_dump(D, "filepath.json.gz")   # for a gzip file
  compress_json.local_dump(D, "filepath.json.bz")   # for a bz2 file
  compress_json.local_dump(D, "filepath.json.lzma") # for a lzma file

  D1 = compress_json.local_load("filepath.json.gz")   # for loading a gzip file
  D2 = compress_json.local_load("filepath.json.bz")   # for loading a bz2 file
  D3 = compress_json.local_load("filepath.json.lzma") # for loading a lzma file

  assert D == D1 == D2 == D3
  ```



  Sometimes you need to load a compressed JSON file a LOT of times, and you may want to put this document in a cache or something of the sort. Fortunately, we already provide this option for you:

  ```python
  import compress_json


  D1 = compress_json.load(
      "filepath.json.gz",
      use_cache=True
  )


  D2 = compress_json.local_load(
      "filepath.json.gz",
      use_cache=True
  )

  assert D1 == D2
  ```



  You can pass parameters to either the chosen compression mode or the JSON library.

  With the `json_kwargs` parameter, you can specify any of the kwargs that should be forwarded to the JSON library method, which you can obtain for your Python version by running `help(json.dump)` and `help(json.load)`, depending on whether you are dumping or loading the JSON object.

  Similarly, with the `compression_kwargs` parameter, you can specify any parameter that has to be forwarded to the compression library that you intend to use, whether that is `lzma`, `gzip`, or `bz2`, and as per JSON will depend on which version you have installed.

  Whether you are dumping or loading a compressed JSON object, you can get the list of parameters you have available to forward to the compression method by running `help(lzma.open)`, `help(gzip.open)`, or `help(bz2.open)`, respectively.

  ```python
  import compress_json

  D = {
      "A": {
          "B": "C"
      }
  }
  compress_json.dump(
      D, "filepath.json.gz",
      compression_kwargs={
          "compresslevel": 9 # The kwargs for gzip
      },
      json_kwargs={
          "indent": 4 # The kwargs for json
      }
  )

  D4 = compress_json.load(
      "filepath.json.gz",
      compression_kwargs={
          "compresslevel": 9 # The kwargs for gzip
      },
      json_kwargs={} # The kwargs for json
  )

  assert D == D4
  ```


  The library is released under the MIT license.


pkgId: a936a4cee5d4ed814850069e9f09afe7fe5387599a9e149b635bef2054018a33
name: python-compress-json
arch: src
version: 1.1.1
epoch: 0
release: 1
summary: The missing Python utility to read and write large compressed JSONs.
url: https://github.com/LucaCappelletti94/compress_json
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-compress-json-1.1.1-1.src.rpm
checksum_type: sha256
description:
  [![pip](https://badge.fury.io/py/compress-json.svg)](https://pypi.org/project/compress-json/)
  [![python](https://img.shields.io/pypi/pyversions/compress-json.svg)](https://pypi.org/project/compress-json/)
  [![license](https://img.shields.io/pypi/l/compress-json.svg)](https://pypi.org/project/compress-json/)
  [![downloads](https://pepy.tech/badge/compress-json)](https://pepy.tech/project/compress-json)
  [![Github Actions](https://github.com/LucaCappelletti94/ugly_csv_generator/actions/workflows/python.yml/badge.svg)](https://github.com/LucaCappelletti94/ugly_csv_generator/actions/)
  [![Codacy Badge](https://app.codacy.com/project/badge/Grade/6aa4b62b4ed34f7d8e2c37ef09848294)](https://app.codacy.com/gh/LucaCappelletti94/compress_json/dashboard?utm_source=gh&utm_medium=referral&utm_content=&utm_campaign=Badge_grade)

  The missing Python utility to read and write compressed JSONs.

  The library is loosely based on the [compress_pickle](https://github.com/lucianopaz/compress_pickle) library.



  As usual, just download it using pip:

  ```shell
  pip install compress_json
  ```



  The compression modes, detected automatically by the file name, are **gzip**, **bz2**, and **lzma** (or **xz**), with the notable exception of **zip** which seems difficult to integrate into the JSON pipeline.



  The library is extremely easy to use:

  ```python
  import compress_json

  D = {
      "A": {
          "B": "C"
      }
  }
  compress_json.dump(D, "filepath.json.gz")   # for a gzip file
  compress_json.dump(D, "filepath.json.bz")   # for a bz2 file
  compress_json.dump(D, "filepath.json.lzma") # for a lzma file
  compress_json.dump(D, "filepath.json.xz") # for a lzma file

  D1 = compress_json.load("filepath.json.gz")   # for loading a gzip file
  D2 = compress_json.load("filepath.json.bz")   # for loading a bz2 file
  D3 = compress_json.load("filepath.json.lzma") # for loading a lzma file
  D3 = compress_json.load("filepath.json.xz") # for loading a lzma file
  ```

  If it happens that you have to load or dump a JSON object with a custom extension, you can specify the compression mode by passing the `compression` parameter to the `load` and `dump` methods:

  ```python
  import compress_json

  D = {
      "A": {
          "B": "C"
      }
  }

  compress_json.dump(D, "filepath.custom_extension1", compression="gzip")   # for a gzip file
  compress_json.dump(D, "filepath.custom_extension2", compression="bz2")   # for a bz2 file
  compress_json.dump(D, "filepath.custom_extension3", compression="lzma") # for a lzma file

  D1 = compress_json.load("filepath.custom_extension1", compression="gzip")   # for loading a gzip file
  D2 = compress_json.load("filepath.custom_extension2", compression="bz2")   # for loading a bz2 file
  D3 = compress_json.load("filepath.custom_extension3", compression="lzma") # for loading a lzma file

  assert D == D1 == D2 == D3
  ```



  The library makes available, other than the usual `load` and `dump` from the JSON library, the methods `local_load` and `local_dump`, which let you load and dump files in the same directory as wherever you are calling them, by using the call stack.

  This can be useful, especially when loading files within packages.

  ```python
  import compress_json

  D = {
      "A": {
          "B": "C"
      }
  }
  compress_json.local_dump(D, "filepath.json.gz")   # for a gzip file
  compress_json.local_dump(D, "filepath.json.bz")   # for a bz2 file
  compress_json.local_dump(D, "filepath.json.lzma") # for a lzma file

  D1 = compress_json.local_load("filepath.json.gz")   # for loading a gzip file
  D2 = compress_json.local_load("filepath.json.bz")   # for loading a bz2 file
  D3 = compress_json.local_load("filepath.json.lzma") # for loading a lzma file

  assert D == D1 == D2 == D3
  ```



  Sometimes you need to load a compressed JSON file a LOT of times, and you may want to put this document in a cache or something of the sort. Fortunately, we already provide this option for you:

  ```python
  import compress_json


  D1 = compress_json.load(
      "filepath.json.gz",
      use_cache=True
  )


  D2 = compress_json.local_load(
      "filepath.json.gz",
      use_cache=True
  )

  assert D1 == D2
  ```



  You can pass parameters to either the chosen compression mode or the JSON library.

  With the `json_kwargs` parameter, you can specify any of the kwargs that should be forwarded to the JSON library method, which you can obtain for your Python version by running `help(json.dump)` and `help(json.load)`, depending on whether you are dumping or loading the JSON object.

  Similarly, with the `compression_kwargs` parameter, you can specify any parameter that has to be forwarded to the compression library that you intend to use, whether that is `lzma`, `gzip`, or `bz2`, and as per JSON will depend on which version you have installed.

  Whether you are dumping or loading a compressed JSON object, you can get the list of parameters you have available to forward to the compression method by running `help(lzma.open)`, `help(gzip.open)`, or `help(bz2.open)`, respectively.

  ```python
  import compress_json

  D = {
      "A": {
          "B": "C"
      }
  }
  compress_json.dump(
      D, "filepath.json.gz",
      compression_kwargs={
          "compresslevel": 9 # The kwargs for gzip
      },
      json_kwargs={
          "indent": 4 # The kwargs for json
      }
  )

  D4 = compress_json.load(
      "filepath.json.gz",
      compression_kwargs={
          "compresslevel": 9 # The kwargs for gzip
      },
      json_kwargs={} # The kwargs for json
  )

  assert D == D4
  ```


  The library is released under the MIT license.


pkgId: 68df8bce5f07bf138b908a29c4d9a696a70d7913f2a3ff8bd0dcae5d4e8b6f5e
name: python-compress-json-python
arch: src
version: 3.0.0
epoch: 0
release: 1
summary: convert JSON data to space efficient format
url: https://github.com/beenotung/compress-json
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-compress-json-python-3.0.0-1.src.rpm
checksum_type: sha256
description:
  Store JSON data in space efficient manner.

  [![PyPi Package Version](https://img.shields.io/pypi/v/compress-json-python)](https://pypi.org/project/compress-json-python)

  This library is optimized to compress json object in compact format, which can save network bandwidth and disk space.



  - Supports all JSON types
  - Object key order is preserved
  - Repeated values are stored only once
  - Numbers are encoded in base62 format (0-9A-Za-z)
  - Support custom backend for memory store and cache



  This package is a python implementation of [compress-json](https://github.com/beenotung/compress-json). It is fully compatible with the npm package so the data compressed by either side can be decompressed by another side.



  - Javascript/Typescript: [source](https://github.com/beenotung/compress-json) / [package](https://www.npmjs.com/package/compress-json)
  - PHP: [source](https://github.com/inkrot/php-compress-json) / [package](https://packagist.org/packages/inkrot/php-compress-json)
  - Python: [source](https://github.com/beenotung/compress-json/tree/master/python) / [package](https://pypi.org/project/compress-json-python)
  - C#: (TODO)



  ```bash
  pip install compress-json-python
  ```



  ```python

  from compress_json import compress, decompress

  data = {
    'user': 'Alice',
    
  }

  compressed = compress(data) # the result is a list (array)

  import requests
  requests.post('https://example.com/submit', json=compressed) # used as json value

  import json
  with open("data.json", "w") as fd:
  	fd.write(json.dumps(compressed)) # convert into string if needed

  reversed = decompress(compressed)
  data === reversed # will be true
  ```

  Detail example can refer to the demo [cli.py](./src/compress_json/cli.py) and tests in [core_test.py](./src/compress_json/core_test.py)



  **Sample data**:

  ```python
  long_str = 'A very very long string, that is repeated'
  data = {
    'int': 42,
    'float': 12.34,
    'str': 'Alice',
    'long_str': long_str,
    'longNum': 9876543210.123455,
    'bool': True,
    'bool2': False,
    'arr': [42, long_str],
    'arr2': [42, long_str], # identical values will be deduplidated, including array and object
    'obj': { # nested values are supported
      'id': 123,
      'name': 'Alice',
      'role': [ 'Admin', 'User', 'Guest' ],
      'long_str': 'A very very long string, that is repeated',
      'longNum': 9876543210.123455
    },
    'escape': [ 's|str', 'n|123', 'o|1', 'a|1', 'b|T', 'b|F' ]
  }
  ```

  **Compressed data**:

  ```python

  compressed = [
    [  # encoded value array
      'int', # string
      'float',
      'str',
      'long_str',
      'longNum',
      'bool',
      'bool2',
      'arr',
      'arr2',
      'obj',
      'escape',
      'a|0|1|2|3|4|5|6|7|8|9|A',
      'n|g', # number (integer) (base62-encoded)
      'n|C.h', # number (float) (integer part and decimals are base62-encoded separately)
      'Alice',
      'A very very long string, that is repeated',
      'n|AmOy42.2KCf',
      'b|T', # boolean (True)
      'b|F', # boolean (False)
      'a|C|F', # array
      'id',
      'name',
      'role',
      'a|K|L|M|3|4',
      'n|1z',
      'Admin',
      'User',
      'Guest',
      'a|P|Q|R',
      'o|N|O|E|S|F|G', # object
      's|s|str', # escaped string
      's|n|123', # escaped number
      's|o|1',
      's|a|1',
      's|b|T', # escaped boolean
      's|b|F',
      'a|U|V|W|X|Y|Z',
      'o|B|C|D|E|F|G|H|I|J|J|T|a'
    ],
    'b' # root value index
  ]
  ```



  **Original JSON data**: (749 characters without white-spaces)

  ```json
  {
    "count": 5,
    "names": ["New York", "London", "Paris", "Beijing", "Moscow"],
    "cities": [
      {
        "id": 1,
        "name": "New York",
        "countryName": "USA",
        "location": { "latitude": 40.714606, "longitude": -74.0028 },
        "localityType": "BIG_CITY"
      },
      {
        "id": 2,
        "name": "London",
        "countryName": "UK",
        "location": { "latitude": 51.507351, "longitude": -0.127696 },
        "localityType": "COUNTRY_CAPITAL"
      },
      {
        "id": 3,
        "name": "Paris",
        "countryName": "France",
        "location": { "latitude": 48.856663, "longitude": 2.351556 },
        "localityType": "COUNTRY_CAPITAL"
      },
      {
        "id": 4,
        "name": "Beijing",
        "countryName": "China",
        "location": { "latitude": 39.90185, "longitude": 116.391441 },
        "localityType": "COUNTRY_CAPITAL"
      },
      {
        "id": 5,
        "name": "Moscow",
        "countryName": "Russia",
        "location": { "latitude": 55.755864, "longitude": 37.617698 },
        "localityType": "COUNTRY_CAPITAL"
      }
    ]
  }
  ```

  **Compressed json**: (562 characters without white-spaces)

  ```json
  [["count", "names", "cities", "a|0|1|2", "n|5", "New York", "London", "Paris", "Beijing", "Moscow", "a|5|6|7|8|9", "id", "name", "countryName", "location", "localityType", "a|B|C|D|E|F", "n|1", "USA", "latitude", "longitude", "a|J|K", "n|e.2Xkv", "n|-1C.28G", "o|L|M|N", "BIG_CITY", "o|G|H|5|I|O|P", "n|2", "UK", "n|p.dz7", "n|-0.2vFR", "o|L|T|U", "COUNTRY_CAPITAL", "o|G|R|6|S|V|W", "n|3", "France", "n|m.1XNq", "n|2.2kQz", "o|L|a|b", "o|G|Y|7|Z|c|W", "n|4", "China", "n|d.F7F", "n|1s.bVh", "o|L|g|h", "o|G|e|8|f|i|W", "Russia", "n|t.1xtN", "n|b.3lHA", "o|L|l|m", "o|G|4|9|k|n|W", "a|Q|X|d|j|o", "o|3|4|A|p"], "q"]
  ```

  In this example, compression saves 25% of characters. However, the more complex and repetitive the structure, the more characters can be saved.



  This project is licensed with [BSD-2-Clause](./LICENSE)

  This is free, libre, and open-source software. It comes down to four essential freedoms [[ref]](https://seirdy.one/2021/01/27/whatsapp-and-the-domestication-of-users.html#fnref:2):

  - The freedom to run the program as you wish, for any purpose
  - The freedom to study how the program works, and change it so it does your computing as you wish
  - The freedom to redistribute copies so you can help others
  - The freedom to distribute copies of your modified versions to others

rpm_license:
  BSD 2-Clause License  Copyright (c) [2023], [Beeno Tung (Tung Cheung Leong)] All rights reserved.  Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:  1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.  2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.


pkgId: c323a391bca1e4d8519cc731116e2246cfc6b9b4b587dd16814fd6d1c79271dd
name: python-compress-pickle
arch: src
version: 2.1.0
epoch: 0
release: 1
summary: Standard pickle, wrapped with standard compression libraries
url: https://github.com/lucianopaz/compress_pickle
rpm_license: MIT License
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-compress-pickle-2.1.0-1.src.rpm
checksum_type: sha256
description:
  [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/ambv/black)
  [![Build Status](https://dev.azure.com/lucianopazneuro/lucianopazneuro/_apis/build/status/lucianopaz.compress_pickle?branchName=master)](https://dev.azure.com/lucianopazneuro/lucianopazneuro/_build/latest?definitionId=1&branchName=master)
  [![Coverage Status](https://codecov.io/gh/lucianopaz/compress_pickle/branch/master/graph/badge.svg)](https://codecov.io/gh/lucianopaz/compress_pickle)
  [![PyPI](https://img.shields.io/pypi/v/compress_pickle.svg)](https://pypi.org/project/compress-pickle/)
  [![License: MIT](https://img.shields.io/badge/License-MIT-purple.svg)](https://opensource.org/licenses/MIT)

  The standard [pickle package](https://docs.python.org/3/library/pickle.html) provides an excellent default tool for serializing arbitrary python objects and storing them to disk. Standard python also includes broad set of [data compression packages](https://docs.python.org/3/library/archiving.html). `compress_pickle` provides an interface to the standard `pickle.dump`, `pickle.load`, `pickle.dumps` and `pickle.loads` functions, but wraps them in order to direct the serialized data through one of the standard compression packages. This way you can seemlessly serialize data to disk or to any file-like object in a compressed way.

  `compress_pickle` supports python >= 3.6. If you must support python 3.5, install `compress_pickle==v1.1.1`.

  Supported compression protocols:
  - [gzip](https://docs.python.org/3/library/gzip.html)
  - [bz2](https://docs.python.org/3/library/bz2.html)
  - [lzma](https://docs.python.org/3/library/lzma.html)
  - [zipfile](https://docs.python.org/3/library/zipfile.html)

  Furthermore, `compress_pickle` supports the [`lz4`](https://pypi.org/project/lz4/) compression protocol, that isn't part of the standard python compression packages. This is provided as an optional extra requirement that can be installed as:

  ```bash
  pip install compress_pickle[lz4]
  ```

  Please refer to the [package's documentation](https://lucianopaz.github.io/compress_pickle/html) for more information


pkgId: 175efb653ba9db2a7179635cef0319ca6b3201e96151140690a7cf3e36dbea80
name: python-compressbins
arch: src
version: 1.2.0
epoch: 0
release: 1
url: https://github.com/jtrudan/compressbins
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-compressbins-1.2.0-1.src.rpm
checksum_type: sha256
summary:
  Convert numpy bins to a string format ideal for creating frequency charts in the numpy-pandas ecosystem. Ex. [0, 2, 4, 6, 8, 10] -> ['0-2', '2-4', '4-6', '6-8', '8-10']

description:
  This package provides the compress_bins() function, a quality-of-life function for creating frequency charts in the numpy-pandas ecosystem.

  Usage:
  ```
  import compressbins.compressbins as cb

  compressed_bins = cb.compress_bins([0,2,4,6])
  print(compressed_bins) # ['0-2','2-4','4-6']
  ```

  We can then use these compressed bins in creating frequency charts:
  ```
  import pandas as pd
  import numpy as np

  counts, bins = np.histogram(df, bins=np.arange(10.00, 20.00, 1.00))
  cbins = compress_bins(bins)

  freqChart = pd.DataFrame({"Cost":cbins, "Count":counts})
  print(freqChart)

          Cost  Count
  0  10.0-11.0      1
  1  11.0-12.0      8
  2  12.0-13.0     16
  3  13.0-14.0      9
  4  14.0-15.0      6
  5  15.0-16.0      4
  6  16.0-17.0      3
  7  17.0-18.0      2
  8  18.0-19.0      1
  ```

rpm_license:
  Copyright 2024 Jack Trudan  Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:  The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.  THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.


pkgId: abd5831a9142b2d348069679897825e242b1fd214e80b284d497e1d3c267b84f
name: python-compressed-rtf
arch: src
version: 1.0.6
epoch: 0
release: 1
summary: Compressed Rich Text Format (RTF) compression and decompression package
url: https://github.com/delimitry/compressed_rtf
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-compressed-rtf-1.0.6-1.src.rpm
checksum_type: sha256
description:
  Compressed Rich Text Format (RTF) compression and decompression package

  Based on Rich Text Format (RTF) Compression Algorithm
  https://msdn.microsoft.com/en-us/library/cc463890(v=exchg.80).aspx


pkgId: c7806d2658830b1ec66d8d2a0e5d9363bd7c929ee051a9b999ce4d83c984e5d5
name: python-compressible-flows
arch: src
version: 1.0.3
epoch: 0
release: 1
url: https://pypi.org/project/compressible-flows/
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-compressible-flows-1.0.3-1.src.rpm
checksum_type: sha256
summary:
  Compressible-flows is a library of isentropic equations used to predict shocks, expansions, nozzles, and diffusers


pkgId: 55cee5548dfb57ac763f28bee19f24d1478f6ebc9b6c3d4e5ce973915318c2b7
name: python-comptests
arch: src
version: 1.4.34
epoch: 0
release: 1
summary: Testing utilities for projects that use ConfTools for handling their configuration.
url: http://github.com/AndreaCensi/comptests
rpm_license: UNKNOWN
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-comptests-1.4.34-1.src.rpm
checksum_type: sha256
description:
  Testing utilities built on top of [ConfTools][conftools], [CompMake][compmake]
  and [QuickApp][quickapp].


pkgId: 541a2e9173c6f02d997b6862c9f61ea4fce1e71346962b60fa4f509096605aaf
name: python-compton
arch: src
version: 4.0.6
epoch: 0
release: 1
summary: An abstract data flow framework for quantitative trading
url: https://github.com/kaelzhang/python-compton
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-compton-4.0.6-1.src.rpm
checksum_type: sha256
description:
  [![Build Status](https://travis-ci.org/kaelzhang/python-compton.svg?branch=master)](https://travis-ci.org/kaelzhang/python-compton)
  [![Coverage](https://codecov.io/gh/kaelzhang/python-compton/branch/master/graph/badge.svg)](https://codecov.io/gh/kaelzhang/python-compton)
  <!-- optional appveyor tst
  [![Windows Build Status](https://ci.appveyor.com/api/projects/status/github/kaelzhang/python-compton?branch=master&svg=true)](https://ci.appveyor.com/project/kaelzhang/python-compton)
  -->
  <!-- optional npm version
  [![NPM version](https://badge.fury.io/js/python-compton.svg)](http://badge.fury.io/js/python-compton)
  -->
  <!-- optional npm downloads
  [![npm module downloads per month](http://img.shields.io/npm/dm/python-compton.svg)](https://www.npmjs.org/package/python-compton)
  -->
  <!-- optional dependency status
  [![Dependency Status](https://david-dm.org/kaelzhang/python-compton.svg)](https://david-dm.org/kaelzhang/python-compton)
  -->



  An abstract data-flow framework for quantitative trading, which decouples data initialization, data composition and data processing.



  ```sh
  pip install compton
  ```



  ```py
  from compton import (
    Orchestrator,
    Provider,
    Reducer,
    Consumer
  )
  ```



  We call a tuple of hashable parameters as a vector which is used to identify a certain kind of data.

  ```py
  from enum import Enum

  class DataType(Enum):
      KLINE = 1
      ORDER_BOOK = 2

  class TimeSpan(Enum):
      DAY = 1
      WEEK = 2

  vector = (DataType.KLINE, TimeSpan.DAY)
  ```



  - **reducers** `List[Reducer]` reducers to compose data
  - **loop?** `Optional[EventLoop]` The event loop object to use. In most cases, you should **NOT** pass this argument, unless you exact know what you are doing.

  ```py
  Orchestrator(
      reducers
  ).connect(
      provider
  ).subscribe(
      consumer
  ).add('TSLA')
  ```



  Connects to a data provider



  Subscribes the consumer to orchestrator.



  Adds a new symbol to orchestrator, and start the data flow for `symbol`



  `Provider` is an abstract class which provides initial data and data updates.

  A provider should be implemented to support many symbols

  We must inherit class `Provider` and implement some abstract method before use.

  - `@property vector` returns an `Vector`
  - `async def init()` method returns the initial data
  - There is an protected method `self.dispatch(symbol, payload)` to set the payload updated, which should only be called in a coroutine, or a `RuntimeError` is raised.

  ```py
  class MyProvider(Provider):
      @property
      def vector(self):
          return (DataType.KLINE, TimeSpan.DAY)

      async def init(self, symbol):
          return {}
  ```



  Another abstract class which handles data composition.

  The `reducer.vector` could be a generic vector which applies partial match to other vectors

  ```py
  class MyReducer(Reducer):
      @property
      def vector(self):
          
          
          
          return (DataType.KLINE,)

      def merge(self, old, new):
          
          if old is None:
              
              return clean(new)

          return {**old, **new}
  ```



  A consumer could subscribes to more than one kind of data types

  ```py
  class MyConsumer(Consumer):
      @property
      def vectors(self):
          
          return [
              (DataType.KLINE, TimeSpan.DAY),
              (DataType.KLINE, TimeSpan.WEEK)
          ]

      @property
      def all(self) -> bool:
          """
          `True` indicates that the consumer will only go processing
          if both of the data corresponds with the two vectors have changes

          And by default, `Consumer::all` is False
          """
          return True

      @property
      def concurrency(self) -> int:
          """
          Concurrency limit for method `process()`

          By default, `Consumer::concurrency` is `0` which means no limit
          """
          return 1

      def should_process(self, *payloads) -> bool:
          """
          If this method returns `False`, then the data update will not be processed
          """
          return True

      
      
      async def process(self, symbol, kline_day, kline_week):
          await doSomething(symbol, kline_day, kline_week)
  ```



  [MIT](LICENSE)


pkgId: 6715a3478e8c866e1d361697b176fcbbbe15d825895c81e4001e668fa243e3be
name: python-compudoc
arch: src
version: 0.5.1
epoch: 0
release: 1
summary: Add python calculations to your LaTeX and Markdown documents.
url: https://pypi.org/project/compudoc/
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-compudoc-0.5.1-1.src.rpm
checksum_type: sha256
description:
  Add the power of python to your LaTeX, Markdown, and more...



  Compudoc is a project with similar goals to [pythontex](https://github.com/gpoore/pythontex), [Codebraid](https://codebraid.org/),
  [pweave](https://mpastell.com/pweave/) and [pyptex](https://pypi.org/project/pyptex/). It is most similar to
  pyptex, and if I had found pyptex earlier, I may not have written Compudoc.

  Features include:

  - Like pyptex, compudoc is a preprocessor. All Python code is executed and replaced before LaTeX, Pandoc, mdSlides, etc is ran. You could use it to render Python files if you wanted.
  - As a preprocessor, compudoc can be used with all your existing tooling. Just run compudoc to produce the source file that would normally go into your pipeline.
  - Unlike pyptex, compudoc is not specific to LaTeX. Any text file can be rendered. LaTeX, Markdown, ReStructuredText, etc. can be rendered with Compudoc.
  - Jinja2 is used for injecting values from Python into the source document. That means you can use Jinja2 filters to make common formatting task cleaner.
  - Python code is executed in a separate interactive Python instance and incrementally between chunks of document text. That means you can define a variable `x` in
    one block of Python code, use that value in a Jinja2 template in your document, change the value of `x` in a later code block, and use it again in the document.
    The value inserted into the document will be the value of `x` at the point it is inserted.
  - Python code is embedded in the comments of your source document, so you can still run the unrendered source file through your toolchain.
  - If the source file you are rending does not support comments (there is no standard way to put comments in Markdown), you can define your own comment line
    identifier and have Compudoc strip them during the render process. This means you can use Compudoc to render any plain text source file without the
    final tool knowing anything about it.



  compudoc processes plain text sources files by breaking the file into "chunks" of document text and python code. For example,
  a document with the text

  ```
  Some text
  % {{{
  % import os
  % }}}
  Some more text
  % {{{
  % CWD = os.getcwd()
  % }}}
  The current directory is {{ CWD }}.

  ```
  would be split into 5 chunks. The first chunk is the document text 'Some text\n', the second chunk is python code and so on.




  Python code is embedded in your document's comments. Code blocks within comment blocks
  are marked with a '{{{' and '}}}' line. Here is a LaTeX example.



  ```latex
  % arara: pdflatex

  % start with vim --server latex %
  \documentclass[]{article}

  \usepackage{siunitx}
  \usepackage{physics}
  \usepackage{graphicx}
  \usepackage{fullpage}

  \author{C.D. Clark III}
  \title{On...}
  \begin{document}
  \maketitle

  % {{{ {}
  % import pint
  % ureg = pint.UnitRegistry()
  % Q_ = ureg.Quantity
  % }}}

  Laser exposures are characterized by a power ($\Phi$), energy ($Q$), radiant exposure ($H$),
  or irradiance ($E$). Each of these four radiometric quantities are related to each other
  through the exposure area and duration.

  % {{{ {}
  % power = Q_(100,'mW')
  % duration = Q_(0.25,'s')
  % energy = (power * duration).to("mJ")
  % }}}

  For example, if a laser outputs a power of {{'{:Lx}'.format(power)}} for a
  duration of {{duration | fmt("Lx")}}, then the energy delivered during the
  exposure will be {{energy | fmt("Lx")}}.

  \end{document}

  ```
  Save this to a file named `main.tex` and run
  ```bash
  $ compudoc main.tex
  ```
  This will create a file named `main-rendered.tex` with the following content

  ```latex
  % arara: pdflatex

  % start with vim --server latex %
  \documentclass[]{article}

  \usepackage{siunitx}
  \usepackage{physics}
  \usepackage{graphicx}
  \usepackage{fullpage}

  \author{C.D. Clark III}
  \title{On...}
  \begin{document}
  \maketitle

  % {{{ {}
  % import pint
  % ureg = pint.UnitRegistry()
  % Q_ = ureg.Quantity
  % }}}

  Laser exposures are characterized by a power ($\Phi$), energy ($Q$), radiant exposure ($H$),
  or irradiance ($E$). Each of these four radiometric quantities are related to each other
  through the exposure area and duration.

  % {{{ {}
  % power = Q_(100,'mW')
  % duration = Q_(0.25,'s')
  % energy = (power * duration).to("mJ")
  % }}}

  For example, if a laser outputs a power of \SI[]{100}{\milli\watt} for a
  duration of \SI[]{0.25}{\second}, then the energy delivered during the
  exposure will be \SI[]{25.0}{\milli\joule}.

  \end{document}

  ```


pkgId: 7a4b0ea7fa40b74796538b2f9d99e449004c93d3c7df797843b31a404ca912f6
name: python-computation-graph
arch: src
version: 54
epoch: 0
release: 1
summary: please add a summary manually as the author left a blank one
url: https://pypi.org/project/computation-graph/
rpm_license: UNKNOWN
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-computation-graph-54-1.src.rpm
checksum_type: sha256
description:
  [![Build Status](https://travis-ci.com/hyroai/computation-graph.svg?branch=master)](https://travis-ci.com/hyroai/computation-graph)

  A function composition framework that supports:

  1. State - functions which retain state for their next turn of action.
  2. Prioritized paths - lazily attempt overloaded composition paths according to priorities.
  3. Deep dependency injection - compose a function to a variadic function at the end of an arbitrarily long pipeline.
  4. Non cancerous `asyncio` support.

  `pip install computation-graph`

  To deploy: `python setup.py sdist bdist_wheel; twine upload dist/*; rm -rf dist/;`



  The runner will type check all outputs for nodes with return type annotations. In case of a wrong typing, it will log the node at fault.





  Available computation trace visualizers:

  1. `graphviz.computation_trace`
  1. `mermaid.computation_trace`
  1. `ascii.computation_trace`

  To use, replace `to_callable` with `run.to_callable_with_side_effect` with your selected style as the first argument.



  This debugger will save a file on each graph execution to current working directory.

  You can use this file in a graph viewer like [gephi](https://gephi.org/).
  Nodes colored red are part of the 'winning' computation path.
  Each of these nodes has the attributes 'result' and 'state'.
  'result' is the output of the node, and 'state' is the _new_ state of the node.

  In gephi you can filter for the nodes participating in calculation of final result by filtering on result != null.


pkgId: b152fbd6575dc7d043af19f69854fce20aa4f1188d015a25561d6ce1661808e1
name: python-compute-rhino3d
arch: src
version: 0.12.2
epoch: 0
release: 1
summary: Python client library for compute.rhino3d web service
url: https://github.com/mcneel/compute.rhino3d
rpm_license: MIT License
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-compute-rhino3d-0.12.2-1.src.rpm
checksum_type: sha256
description:
  Python package providing convenience functions to call compute.rhino3d.com
  geometry web services

  Project Hompage: https://github.com/mcneel/compute.rhino3d


  * This is a pure python package and should work on all versions of python




  * start `python`
  ```
  >>> from rhino3dm import *
  >>> import compute_rhino3d.Util
  >>> import compute_rhino3d.Mesh
  >>>
  >>> compute_rhino3d.Util.authToken = AUTH_TOKEN_FROM (rhino3d.com/compute/login)
  >>> center = Point3d(250, 250, 0)
  >>> sphere = Sphere(center, 100)
  >>> brep = sphere.ToBrep()
  >>> meshes = compute_rhino3d.Mesh.CreateFromBrep(brep)
  >>> print("Computed mesh with {} faces".format(len(meshes[0].Faces))
  ```


pkgId: 982edf5a5ed61157ef73c67cf166efacb721a1d2a49d03b076ed41e1ba8b3e9b
name: python-computer-vision-design-patterns
arch: src
version: 2.0.5
epoch: 0
release: 1
summary: please add a summary manually as the author left a blank one
url: https://pypi.org/project/computer-vision-design-patterns/
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-computer-vision-design-patterns-2.0.5-1.src.rpm
checksum_type: sha256
description:
  [![Poetry](https://img.shields.io/endpoint?url=https://python-poetry.org/badge/v0.json)](https://python-poetry.org/)
  [![Coverage Status](/reports/coverage/coverage-badge.svg?dummy=8484744)](./reports/coverage/index.html)


  To create a new release, run the following command:
  ```
  make release
  ```
  By default, the release will be a patch release. To create a minor or major release, run the following command:
  ```
  make release BUMP=minor
  ```
  or
  ```
  make release BUMP=major
  ```


pkgId: 22bdf0d85f494cb47f27ea0039331e2a639ecf6bfe67f4f4aa2c3d8deca3abf9
name: python-computing-toolbox
arch: src
version: 1.6.11
epoch: 0
release: 1
summary: Computing Toolbox for daily computations
url: https://pypi.org/project/computing-toolbox/
rpm_license: MIT License
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-computing-toolbox-1.6.11-1.src.rpm
checksum_type: sha256
description:
  This is a library aiming to help programmers to sort every day tasks easily.
  This library contains:
  1. gcp
     1. secret-manager utils to access secret values
     2. gcp-storage utils to check if a file exists in Google Cloud or get a list of files in a bucket
  2. jsonl class able to read and write json line file very quickly
  3. http-request class able to get html files easily
  4. async-http-request class able to get multiple files easily
  5. utils
     1. deep_get function able to get dynamically values from a dictionary
     2. split function able to split a given interval of size n in m parts



  ``pip install computing-toolbox``


  ```python
  from computing_toolbox.jsonl import Jsonl

  data0 = [x for x in Jsonl("/path/to/file")]

  data1 = list(Jsonl("/path/to/file"))


  data2 = [x for x in Jsonl("/path/to/file", offset=20, limit=100)]


  data3=[{"name":"Newton"}, {"name":"Galileo"}]
  Jsonl("/path/to/file","w").write(data3)
  ```

  If the file is located in the cloud,
  you only need to configure your environment
  `Jsonl` class will handle it for aws, gcp, etc.

  `Jsonl` can handle `.zip` or `.gz` files automatically
  you only need to specify the right extension in the path


  ``pip install .``


  Pedro Mayorga.


pkgId: 4257a46825fa85516c0f7bcb382ad1988751d1cf63ed2e9533095aae42ae5630
name: python-compynator
arch: src
version: 0.1.2
epoch: 0
release: 1
url: https://github.com/TheTerminalTemplar/compynator
rpm_license: Apache-2.0
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-compynator-0.1.2-1.src.rpm
checksum_type: sha256
summary:
  A pure Python implementation of parser combinators with asymptotically best performance and support for context-sensitive or ambiguous grammars.


pkgId: 4013d917337d007c328cf005ce428f286a661f124790319ca9828419cdb38506
name: python-comradewolf
arch: src
version: 0.0.2b2
epoch: 0
release: 1
summary: Core to make OLAP-like engine. Creates SQL queries based on imported meta-data
url: https://github.com/konstantin-suspitsyn/comradewolf
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-comradewolf-0.0.2b2-1.src.rpm
checksum_type: sha256
description:
  Comradewolf — часть бо́льшего проекта, который должен позволить людям осуществлять быстрые выгрузки из таблиц
  с данными, не увеличивая количество агрегированных срезов (отдельных таблиц) в базе

  База данных должна содержать главную таблицу без срезов, и несколько подготовленных таблиц со срезами из главной таблицы.

  В зависимости от необходимых полей, модуль отсеет неподходящие таблицы и предложит простой ```SELECT``` без ```GROUP BY```
  для готовых срезов, либо сформирует запрос с ```GROUP BY```, причем не только из главной таблицы, но и из таблиц
  со срезами, если это возможно


pkgId: b82029d6589016aa2bd7e95a852c424a7227dff009a4a24f5c02e954cadc3307
name: python-comsel
arch: src
version: 1.1.0
epoch: 0
release: 1
summary: A wxPython-based COM port selection tool
url: https://github.com/Molisc/ComSel
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-comsel-1.1.0-1.src.rpm
checksum_type: sha256
description:
  `comsel` — это простая библиотека на Python с использованием `wxPython` для выбора COM порта и скорости (baudrate) в графическом интерфейсе.



  Убедитесь, что у вас установлен Python версии 3.9 или выше. Затем установите библиотеку через `pip`:

  ```bash
  pip install comsel
  ```


  Если библиотека установлена из исходников, убедитесь, что у вас установлены зависимости:

  ```bash
  pip install -r requirements.txt
  ```



  Библиотека предоставляет удобную функцию для открытия окна выбора COM порта и скорости:

  ```python
  from comsel.portSelect import select_com_port_and_baudrate


  port, baudrate = select_com_port_and_baudrate()


  if port and baudrate:
      print(f"Выбран порт: {port}, скорость: {baudrate}")
  else:
      print("Порт или скорость не были выбраны.")
  ```


  1. Импортируйте функцию `select_com_port_and_baudrate` из модуля `comsel.portSelect`.
  2. Вызовите функцию и обработайте результат.



  - Поддержка автоматического обнаружения доступных COM портов.
  - Возможность ручного ввода порта и скорости.
  - Проверка на корректность введённых данных:
    - Формат порта должен соответствовать `COM1`, `COM2` и т.д.
    - Скорость должна быть в пределах от `1` до `1,000,000`.



  - Python 3.9+
  - `wxPython`
  - `pyserial`



  ![Пример работы приложения](docs/screenshot.png)



  Если вы хотите внести изменения в проект:

  1. Клонируйте репозиторий:
     ```bash
     git clone https://github.com/ваш_проект/comsel.git
     ```
  2. Установите зависимости:
     ```bash
     pip install -r requirements.txt
     ```
  3. Внесите изменения и протестируйте их.



  Этот проект распространяется под лицензией [MIT](LICENSE). Вы можете свободно использовать, изменять и распространять код.



  Если у вас возникли вопросы или предложения, свяжитесь с автором:

  - **Автор**: Molisc
  - **Email**: ivanrudko445@gmail.com


pkgId: eb245dc9b90ca135fcca5d0b90ff93d02693219db59c03320a07b4b04ca50fd3
name: python-comtacts
arch: src
version: 0.1.0
epoch: 0
release: 1
summary: A tool to remember and detect communication ports using hardware identifiers
url: https://pypi.org/project/comtacts/
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-comtacts-0.1.0-1.src.rpm
checksum_type: sha256
description:
  ![COMtacts](./assets/logo.png)

  This is a tool to remember and detect communication ports using hardware identifiers.





  Lets assume you've plugged in a USB device and you know it is currently at port "COM3". We will use COMtacts to remember this hardware so next time you plug it in, we can find it even if its not at "COM3".

  ```bash
  comtacts add --name "My USB Device" --port "COM3"
  ```

  To overwrite an existing contact:

  ```bash
  comtacts add --name "My USB Device" --port "COM3" --overwrite
  ```

  or if you're in python already:

  ```python
  from comtacts import add_port
  add_port(name="My USB Device", port="COM3", overwrite=False) # or overwrite=True, up to you
  ```



  ```bash
  comtacts get --name "My USB Device"
  ```

  or if you're in python already:

  ```python
  from comtacts import get_port
  get_port(name="My USB Device")
  ```

  Typical usage would be to let your Python script automatically find the port of a device.

  ```python
  from serial import Serial
  from comtacts import get_port

  ser = Serial(
      port = get_port(name="My USB Device"),
      baudrate = 9600,
      timeout = 1
  )
  ```



  ```bash
  comtacts all
  ```

  or if you're in python already:

  ```python
  from comtacts import all_contacts
  all_contacts()
  ```



  ```bash
  comtacts where
  ```



  Under the hood, COMtacts uses `serial.tools.list_ports.comports()` to find serial ports. `list_ports.comports()` returns a list of `serial.tools.list_ports.ListPortInfo` objects, which have attributes that can be used to identify a device. If you know the values of these attributes for your device, you can use them to find the port. See the [documentation](<[function](https://pyserial.readthedocs.io/en/latest/tools.html)>) on these functions to see which attributes are available.

  For example, if you know the manufacturer, vid, and pid of your device, you can use the `manufacturer`, `vid`, and `pid` attributes to find the port:

  ```python
  from comtacts import get_port_with_attributes
  get_port_with_attributes(manufacturer="My Manufacturer", vid="0403", pid="6001")
  ```



  By default, COMtacts saves contacts _inside its package directory_. This was done such that contacts are specific to a python environment -- most of my use cases are specific to a certain application. If you would like to make contacts available to all python environments, you can set the `COMTACTS_DIR` environment variable to the directory where you would like to save contacts.


pkgId: 6d9c949ce87aaff70749f202dfeb283f19885219f748bb254fad7a28d0dfefa3
name: python-comtrade
arch: src
version: 0.1.2
epoch: 0
release: 1
summary: A module designed to read Common Format for Transient Data Exchange (COMTRADE) file format.
url: https://github.com/dparrini/python-comtrade
rpm_license: MIT License
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-comtrade-0.1.2-1.src.rpm
checksum_type: sha256
description:
  __Python Comtrade__ is a module for Python 3 designed to read _Common Format for Transient Data Exchange_ (COMTRADE) files. These consists of oscillography data recorded during power system outages, control systems tests, validation and tests of field equipment, protective relaying logs, etc. The COMTRADE format is defined by IEEE Standards, summarized in the table below. Some equipment vendors put additional information in proprietary versions of it. This module aims IEEE definitions but may support those proprietary versions.


  | Standard                                  | Revision |
  |:------------------------------------------|:--------:|
  | IEEE C37.111(TM)-1991                     |   1991   |
  | IEEE C37.111(TM)-1999                     |   1999   |
  | IEC 60255-24:2001                         |   2001   |
  | IEEE C37.111(TM)-2013 / IEC 60255-24:2013 |   2013   |




  ```
  pip install comtrade
  ```

  Or just copy `comtrade.py` from this repository.



  - Python 3.6 or newer
  - (optional) numpy for speed improvement
  - (optional) pandas for data frame support




  The examples below shows how to open both CFG and DAT files or the new CFF file to plot (using `pyplot`) analog channel oscillography.




  Comtrade files separated in CFG and DAT formats can also be read with `load` function. A `CFG` file path must be passed as an argument and, optionally, a `DAT` file path too (if the file name is not equal of the CFG file).

  ```python
  import matplotlib.pyplot as plt
  import comtrade

  rec = comtrade.load("sample_files/sample_ascii.cfg", "sample_files/sample_ascii.dat")
  print("Trigger time = {}s".format(rec.trigger_time))

  plt.figure()
  plt.plot(rec.time, rec.analog[0])
  plt.plot(rec.time, rec.analog[1])
  plt.legend([rec.analog_channel_ids[0], rec.analog_channel_ids[1]])
  plt.show()
  ```

  It will read the contents of additional header (`*.hdr`) and information (`*.inf`) files.
  Their contents are available through `Comtrade.hdr` and `Comtrade.inf` properties.




  ```python
  import matplotlib.pyplot as plt
  import comtrade

  rec = comtrade.load("sample_files/sample_ascii.cff")
  print("Trigger time = {}s".format(rec.trigger_time))

  plt.figure()
  plt.plot(rec.time, rec.analog[0])
  plt.plot(rec.time, rec.analog[1])
  plt.legend([rec.analog_channel_ids[0], rec.analog_channel_ids[1]])
  plt.show()
  ```

  The module function `load` must be called with the `CFF` file path specified.

  `Comtrade.analog` and `Comtrade.status` lists stores analog and status channel sample lists respectively. These can be accessed through zero-based indexes, i.e., `Comtrade.analog[0]`. The list `Comtrade.time` stores each sample time in seconds.

  More information can be accessed through `Comtrade.cfg` object, which stores data such as detailed channel information.

  Data of additional sections, such as HDR and INF, can be accessed through `hdr` and `inf` properties, respectively.




  The module supports the use of `pandas.DataFrame` as a data structure. Use the conveninent function `load_as_dataframe` or call
  `Comtrade.to_dataframe()` function after loading the data.

  ```python
  import comtrade
  df = comtrade.load_as_dataframe("sample_files/sample_ascii.cfg")
  print(df.head())


  rec = comtrade.load("sample_files/sample_ascii.cfg")
  df = rec.to_dataframe()
  print(df.head())
  ```




  This module implements some of the functionality described in each of the Standard revisions. The tables below lists some features and file formats and which revision supports it. It also shows whether this module support the feature or the format.

  Feel free to pull requests implementing one of these unsupported features or fixing bugs.

  | Formats                                               | 1991 | 1999  | 2001 | 2013 | Module Support  |
  |:------------------------------------------------------|:----:|:-----:|:----:|:----:|:---------------:|
  | CFG file format                                       | x    | x     | x    | x    | x               |
  | DAT file format                                       | x    | x     | x    | x    | x               |
  | HDR file format                                       | x    | x     | x    | x    | no              |
  | INF file format                                       |      | x     | x    | x    | no              |
  | CFF file format                                       |      |       |      | x    | x               |
  | ASCII data file format                                | x    | x     | x    | x    | x               |
  | Binary data file format                               | x    | x     | x    | x    | x               |
  | Binary32 data file format                             |      |       |      | x    | x               |
  | Float32 data file format                              |      |       |      | x    | x               |
  | Schema for phasor data                                |      |       |      | x    | no              |


  | Features                                              | 1991 | 1999 | 2001 | 2013 | Module Support  |
  |:------------------------------------------------------|:----:|:----:|------|:----:|:---------------:|
  | COMTRADE standard revision                            |      |  x   | x    | x    | x               |
  | Timestamp multiplication factor                       |      |  x   | x    | x    | x               |
  | Time code and local code                              |      |      |      | x    | x               |
  | Time quality of the samples                           |      |      |      | x    | x               |
  | Analog channel time skew                              |      |  x   | x    | x    | Partial         |
  | Analog channel primary and secondary VT or CT ratio   |      |  x   | x    | x    | x               |
  | Status channel phase and monitored circuit            |      |  x   | x    | x    | x               |
  | Multiple sample rates                                 | x    |  x   | x    | x    | Partial         |
  | Nanoseconds scale                                     |      |      | x    | x    | x               |




  * Nanoseconds time base within Python's `datetime` objects (such as `start_timestamp` and `trigger_timestamp` properties). It warns the user but doesn't use it, truncating the numbers.
  * Use of multiple sample rates in time calculations for binary data.






  The use of `numpy.array` as a data structure to hold time, analog and status data can be enforced
  in `Comtrade` object constructor:

  ```python
  obj = Comtrade(use_numpy_arrays=True)
  ```

  It may improve performance for computations after loading data.




  Missing analog values are replaced with `float('nan')`. The value used to represent missing data varies with the
  Comtrade dat format and revision year. These are listed in the table next.

  |  revision  |  format  | Missing Data Value            |
  |:----------:|:--------:|:------------------------------|
  |    1991    |  ascii   | `""` (empty string)           |
  | 1999, 2013 |  ascii   | `99999`                       |
  |    1991    |  binary  | `0xFFFF`                      |
  | 1999, 2013 |  binary  | `0x8000` or `-32768`          |
  |    2013    | binary32 | `0x80000000` or `-2147483648` |
  |    2013    | float32  | minimum negative float value  |




  Specify the `encoding` as a keyword argument on all load methods as you'd specify for common file loading:

  ```python
  import comtrade
  comtrade.load("sample_files/sample_ascii.cff", encoding="iso-8859-1")
  ```




  https://github.com/dparrini/python-comtrade



  Feel free to report any bugs you find. You are welcome to fork and submit pull requests.



  To run tests, use Python's `unittest`. From a clone of the GitHub repository, run the command:

  ```sh
  python3 -m unittest ./tests/tests.py
  ```



  The module is available at [GitHub](https://github.com/dparrini/python-comtrade) under the MIT license.


pkgId: 7e44a6dbad27ed84b9a73d0fbf623b7cb200eb633bfe788ef47686b9688b8cbc
name: python-comtrade-pysharp
arch: src
version: 0.0.3
epoch: 0
release: 1
summary: Simple reader for COMTRADE files, using C# underneath to be fast
url: https://pypi.org/project/comtrade-pysharp/
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-comtrade-pysharp-0.0.3-1.src.rpm
checksum_type: sha256
description:
  A Python wrapper to the Gemstone COMTRADE reader (C#, hence pysharp). This should make it much faster than pure python.
  Additionally, it tries to keep the RAM usage fairly low, though the entire file is loaded at once.

  Depending on the size of the files to be read, the time to read can be reduced by 50x.

  **This package does not follow the COMTRADE standard**. In terms of data representation, we consider ``signal names``
  to be the relevant part, not ``signal id``. Similarly, typically the data and names are simply put in independent lists,
  and users have to re-map the arrays by themselves.
  We provide the data as 2 dictionaries (one for analog, one for digital) and a list of timestamps.

  In theory, all COMTRADE formats should be supported, but it's not tested. Same for different OS, architectures,
  python versions.





  Internals/FileReader contains two items:
  - ReaderDll, which provides a simplification layer to the [GSF.COMTRADE](https://github.com/GridProtectionAlliance/gsf)
  - FileReader, which just uses the one above

  The ReaderDll builds to a single dll, this is done with [ILRepack](https://github.com/gluck/il-repack).



  We use the ReaderDll and access it there.
  The data is then converted to [numpy](https://numpy.org/) arrays and the memory of the C# object is freed.



  Contains two COMTRADE files:
  - One example of a big file
  - One example taken from from [comtrade](https://github.com/drewsilcock/comtrade/tree/main/tests/comtrade_files)



  To use this, once installed. Call

  ```
  from comtrade_pysharp import read_comtrade
  file = "C:/my_comtrade.cfg"
  data = read_comtrade(file)
  ```

  You can also pass the following arguments ``analog_channels`` or ``digital_channels`` to keep only these channels, this
  reduces the RAM usage. As well as ``subsampling`` which will do a basic subsampling of the file to further reduce RAM usage.




  No idea, everything is open source and redistributable as far as I can tell, but there's some nuget packages in there that have other licenses, so do your due diligence. Everything is put "as is" on here...
  Some seem to be BSD (Antlr3 and another one)





  I have no idea what I'm doing :).

  ![](cat.gif)



  I only tested this against one of the files I'm using, in the one version of Python I'm using (3.9.13-32 bits). There's no reason it wouldn't work for other versions, but I haven't even tried.



  I did this "for fun" to evaluate how much faster I could make a reader using a multi-language approach. You can report issues but this is not something I want to maintain significantly, unless I see it gain a lot of traction.





  * 	Seems the example throws an exception at the end. This appeared once I merged all dlls, not sure why.




  * 	Figure out the proper way to create the package (copy the dll and make it available on pypi)


pkgId: dfd32466afe8902a912677c0cfb643280783e0427c574d510082904d2bdbac2c
name: python-comtradeapicall
arch: src
version: 1.2.1
epoch: 0
release: 1
summary: A package to call UN Comtrade APIs
url: https://github.com/uncomtrade/comtradeapicall
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-comtradeapicall-1.2.1-1.src.rpm
checksum_type: sha256
description:
  This package simplifies calling [APIs of UN Comtrade](https://comtradedeveloper.un.org) to extract and download data
   (and much more).


  [UN Comtrade](https://comtrade.un.org) provides free and premium APIs to extract and download data/metadata, however
   it is quite a learning curve to understand all of APIs end-points and parameters. This package simplifies it by
    calling a single python function with the appropriate parameters. Learn more about UN Comtrade at the [UN Comtrade Docs](https://uncomtrade.org/docs).

  This project is intended to be deployed at [The Python Package Index](https://pypi.org/project/comtradeapicall/), therefore the structure of
   folders follows the suggested layout from [Packaging Python Project](https://packaging.python.org/en/latest/tutorials/packaging-projects/). The main scripts are located at **/src/comtradeapicall/**. And the folder **tests** contains the example scripts how to install and use the package.

   
  This package assumes using Python 3.7 and the expected package dependencies are listed in the "requirements.txt" file
   for PIP, you need to run the following command to get dependencies:
  ```
  pip install -r requirements.txt
  ```


  The package has been deployed to the PyPi and it can be install using pip command below:
  ```
  pip install comtradeapicall
  ```


  - **Get/Preview:** Model class to extract the data into pandas data frame
    - previewFinalData(**SelectionCriteria**, **query_option**) : return data frame containing final trade data (limited to 500 records)
    - previewTarifflineData(**SelectionCriteria**, **query_option**) : return data frame containing tariff line data (limited to 500
     records)
    - getFinalData(**subscription_key**, **SelectionCriteria**, **query_option**) : return data frame containing final
     trade data (limited to 250K records)
    - getTarifflineData(**subscription_key**, **SelectionCriteria**, **query_option**)  : return data frame containing
     tariff line data (limited to 250K records)
    - Alternative functions of _previewFinalData, _previewTarifflineData, _getFinalData, _getTarifflineData returns the
     same data frame, respectively,  with query optimization by calling multiple APIs based on the periods (instead of
      single API call)

  - **DataAvailability:** Model class to extract data availability
    - _getFinalDataAvailability(**SelectionCriteria**) : return data frame containing final data
     availability - no subscription key
    - getFinalDataAvailability(**subscription_key**, **SelectionCriteria**) : return data frame containing final data
     availability
    - _getTarifflineDataAvailability(**SelectionCriteria**) : return data frame containing tariff
     line
     data
     availability - no subscription key
    - getTarifflineDataAvailability(**subscription_key**, **SelectionCriteria**) : return data frame containing tariff
     line
     data
     availability
    - getFinalDataBulkAvailability(**subscription_key**, **SelectionCriteria**, **[publishedDateFrom]**, **[publishedDateTo]**) : return data frame containing final bulk files data
     availability
    - getTarifflineDataBulkAvailability(**subscription_key**, **SelectionCriteria**, **[publishedDateFrom]**, **[publishedDateTo]**) : return data frame containing tariff
     line bulk files
     data
     availability
    - getLiveUpdate(**subscription_key**) : return data frame recent data releases

  - **BulkDownload:** Model class to download the data files
    - bulkDownloadFinalData(**subscription_key**, **directory**,  **SelectionCriteria**, **decompress**, **[publishedDateFrom]**, **[publishedDateTo]**) : download/save
     final data files to specified folder
    - bulkDownloadFinalClassicData(**subscription_key**, **directory**,  **SelectionCriteria**, **decompress**, **[publishedDateFrom]**, **[publishedDateTo]**) : download/save
     final classic data files to specified folder
    - bulkDownloadTarifflineData(**subscription_key**, **directory**,  **SelectionCriteria**, **decompress**, **[publishedDateFrom]**, **[publishedDateTo]**) : download
    /save tariff line data files to specified folder

  - **Async:** Model class to extract the data asynchronously (limited to 2.5M records) with email notification
    - submitAsyncFinalDataRequest(**subscription_key**, **SelectionCriteria**, **query_option**) : submit a final data job
    - submitAsyncTarifflineDataRequest(**subscription_key**, **SelectionCriteria**, **query_option**) : submit a tariff line data job
    - checkAsyncDataRequest(**subscription_key**, **[batchId]**) : check status of submitted job
    - downloadAsyncFinalDataRequest(**subscription_key**, **directory**, **SelectionCriteria**, **query_option**) : submit, wait and download the resulting final file
    - downloadAsyncTarifflineDataRequest(**subscription_key**, **directory**, **SelectionCriteria**, **query_option**) : submit, wait and download the resulting  tariff line file

  - **Metadata:** Model class to extract metadata and publication notes
    - _getMetadata(**SelectionCriteria**, **showHistory**) : return data frame with metadata and publication notes - no subscription key
    - getMetadata(**subscription_key**, **SelectionCriteria**, **showHistory**) : return data frame with metadata and publication notes
    - listReference(**[category]**) : return data frame containing list of references
    - getReference(**category**) : return data frame with the contents of specific references

  - **SUV:** Model class to extract data on Standard Unit Values (SUV) and their ranges
    - getSUV(**subscription_key**, **SelectionCriteria**, **[qtyUnitCode]**) : return data frame with SUV data

  - **AIS:** Model class to extract experimental trade data generated from AIS (ships tracking movement). See [Cerdeiro, Komaromi, Liu and Saeed (2020)](https://www.imf.org/en/Publications/WP/Issues/2020/05/14/World-Seaborne-Trade-in-Real-Time-A-Proof-of-Concept-for-Building-AIS-based-Nowcasts-from-49393). *When consuming the data, users should understand its limitation.*
    - getAIS(**subscription_key**, **AISSelectionCriteria**, **[vesselTypeCode]**) : return data frame with AIS trade data

  See differences between final and tariff line data at the [Docs](https://uncomtrade.org/docs/what-is-tariffline-data/)


  - typeCode(str) : Product type. Goods (C) or Services (S)
  - freqCode(str) : The time interval at which observations occur. Annual (A) or Monthly (M)
  - clCode(str) : Indicates the product classification used and which version (HS, SITC)
  - period(str) :  Combination of year and month (for monthly), year for (annual)
  - reporterCode(str) : The country or geographic area to which the measured statistical phenomenon relates
  - cmdCode(str) : Product code in conjunction with classification code
  - flowCode(str) : Trade flow or sub-flow (exports, re-exports, imports, re-imports, etc.)
  - partnerCode(str) : The primary partner country or geographic area for the respective trade flow
  - partner2Code(str) : A secondary partner country or geographic area for the respective trade flow
  - customsCode(str) : Customs or statistical procedure
  - motCode(str) : The mode of transport used when goods enter or leave the economic territory of a country


  - maxRecords(int) : Limit number of returned records
  - format_output(str) : The output format. CSV or JSON
  - aggregateBy(str) : Option for aggregating the query
  - breakdownMode(str) : Option to select the classic (trade by partner/product) or plus (extended breakdown) mode
  - countOnly(bool) : Return the actual number of records if set to True
  - includeDesc(bool) : Option to include the description or not


  - typeCode(str) : Product type. Only Goods (C)
  - freqCode(str) : The time interval at which observations occur. Daily (D)
  - datefrom(str) and dateto(str) :  Date(s) of observation - ASCII format
  - countryareaCode(str) : The country or geographic area to which the measured statistical phenomenon relates. Use *getReference('ais:countriesareas')* for the complete list.
  - vesselTypeCode(str) : The high level categorization of vessels transporting the goods. Use *getReference('ais:vesseltypes')* for the complete list.
  - flowCode(str) : Trade flow (exports, imports)


  - proxy_url(str) : All functions that call the API support the proxy server. Use the parameter proxy_url.



  - Extract Australia imports of commodity code 91 in classic mode in May 2022
  ``` python
  mydf = comtradeapicall.previewFinalData(typeCode='C', freqCode='M', clCode='HS', period='202205',
                                          reporterCode='36', cmdCode='91', flowCode='M', partnerCode=None,
                                          partner2Code=None,
                                          customsCode=None, motCode=None, maxRecords=500, format_output='JSON',
                                          aggregateBy=None, breakdownMode='classic', countOnly=None, includeDesc=True)
  ```
  - Extract Australia tariff line imports of commodity code started with 90 and 91 from Indonesia in May 2022
  ``` python
  mydf = comtradeapicall.previewTarifflineData(typeCode='C', freqCode='M', clCode='HS', period='202205',
                                               reporterCode='36', cmdCode='91,90', flowCode='M', partnerCode=36,
                                               partner2Code=None,
                                               customsCode=None, motCode=None, maxRecords=500, format_output='JSON',
                                               countOnly=None, includeDesc=True)
  ```
  - Extract Australia imports of commodity codes 90 and 91 from all partners in classic mode in May 2022
  ``` python
  mydf = comtradeapicall.getFinalData(subscription_key, typeCode='C', freqCode='M', clCode='HS', period='202205',
                                      reporterCode='36', cmdCode='91,90', flowCode='M', partnerCode=None,
                                      partner2Code=None,
                                      customsCode=None, motCode=None, maxRecords=2500, format_output='JSON',
                                      aggregateBy=None, breakdownMode='classic', countOnly=None, includeDesc=True)
  ```
  - Extract Australia tariff line imports of commodity code started with 90 and 91 from Indonesia in May 2022
  ``` python
  mydf = comtradeapicall.getTarifflineData(subscription_key, typeCode='C', freqCode='M', clCode='HS', period='202205',
                                           reporterCode='36', cmdCode='91,90', flowCode='M', partnerCode=36,
                                           partner2Code=None,
                                           customsCode=None, motCode=None, maxRecords=2500, format_output='JSON',
                                           countOnly=None, includeDesc=True)
  ```
  - Download monthly France final data of Jan-2000
  ``` python
  comtradeapicall.bulkDownloadFinalFile(subscription_key, directory, typeCode='C', freqCode='M', clCode='HS',
                                        period='200001', reporterCode=251, decompress=True)
  ```
  - Download monthly France tariff line data of Jan-March 2000
  ``` python
  comtradeapicall.bulkDownloadTarifflineFile(subscription_key, directory, typeCode='C', freqCode='M', clCode='HS',
                                             period='200001,200002,200003', reporterCode=504, decompress=True)
  ```
  - Download annual Morocco tariff line  data of 2010
  ``` python
  comtradeapicall.bulkDownloadTarifflineFile(subscription_key, directory, typeCode='C', freqCode='A', clCode='HS',
                                             period='2010', reporterCode=504, decompress=True)
  ```
  - Download all final annual data  in HS classification released yesterday
  ``` python
  yesterday = date.today() - timedelta(days=1)
  comtradeapicall.bulkDownloadTarifflineFile(subscription_key, directory, typeCode='C', freqCode='A', clCode='HS',
                                                period=None, reporterCode=None, decompress=True,
                                                publishedDateFrom=yesterday, publishedDateTo=None)
  ```
  - Show the recent releases
  ``` python
  mydf = comtradeapicall.getLiveUpdate(subscription_key)
  ```
  - Extract final data availability in 2021
  ``` python
  mydf = comtradeapicall.getFinalDataAvailability(subscription_key, typeCode='C', freqCode='A', clCode='HS',
                                                           period='2021', reporterCode=None)
  ```
  - Extract tariff line data availability in June 2022
  ``` python
  mydf = comtradeapicall.getTarifflineDataAvailability(subscription_key, typeCode='C', freqCode='M', clCode='HS',
                                                          period='202206', reporterCode=None)
  ```
  - Extract final bulk files data availability in 2021 for the SITC Rev.1 classification
  ``` python
  mydf = comtradeapicall.getFinalDataBulkAvailability(subscription_key, typeCode='C', freqCode='A', clCode='S1',
                                                           period='2021', reporterCode=None)
  ```
  - Extract tariff line bulk files data availability in June 2022
  ``` python
  mydf = comtradeapicall.getTarifflineDataBulkAvailability(subscription_key, typeCode='C', freqCode='M', clCode='HS',
                                                          period='202206', reporterCode=None)
  ```
  - List data availabity from last week for reference year 2021
  ``` python
  mydf = comtradeapicall.getFinalDataAvailability(subscription_key, typeCode='C', freqCode='A', clCode='HS',period='2021', reporterCode=None, publishedDateFrom=lastweek, publishedDateTo=None)
  ```
  - List tariffline data availabity from last week for reference period June 2022
  ``` python
  mydf = comtradeapicall.getTarifflineDataAvailability(subscription_key, typeCode='C', freqCode='M',
                                                          clCode='HS',
                                                          period='202206', reporterCode=None, publishedDateFrom=lastweek, publishedDateTo=None)
  ```
  - List bulk data availability for SITC Rev.1 for reference year 2021 released since last week
  ``` python
  mydf = comtradeapicall.getFinalDataBulkAvailability(subscription_key, typeCode='C', freqCode='A',
                                                      clCode='S1',
                                                      period='2021', reporterCode=None, publishedDateFrom=lastweek, publishedDateTo=None)
  ```
  - List bulk tariffline data availability from last week for reference period June 2022
  ``` python
  mydf = comtradeapicall.getTarifflineDataBulkAvailability(subscription_key, typeCode='C', freqCode='M',
                                                              clCode='HS',
                                                              period='202206', reporterCode=None, publishedDateFrom=lastweek, publishedDateTo=None)

  ```
  - Obtain all metadata and publication notes for May 2022
  ``` python
  mydf = comtradeapicall.getMetadata(subscription_key, typeCode='C', freqCode='M', clCode='HS', period='202205',
                                                   reporterCode=None, showHistory=True)
  ```
  - Submit asynchronous final data request
  ``` python
  myJson = comtradeapicall.submitAsyncFinalDataRequest(subscription_key, typeCode='C', freqCode='M', clCode='HS',
                                      period='202205',
                                      reporterCode='36', cmdCode='91,90', flowCode='M', partnerCode=None,
                                      partner2Code=None,
                                      customsCode=None, motCode=None, aggregateBy=None, breakdownMode='classic')
  print("requestID: ",myJson['requestId'])
  ```
  - Submit asynchronous tariff line data request
  ``` python
  myJson = comtradeapicall.submitAsyncTarifflineDataRequest(subscription_key, typeCode='C', freqCode='M',
                                                         clCode='HS',
                                            period='202205',
                                           reporterCode=None, cmdCode='91,90', flowCode='M', partnerCode=None,
                                           partner2Code=None,
                                           customsCode=None, motCode=None)
  print("requestID: ",myJson['requestId'])
  ```
  - Check status of asynchronous job
  ``` python
  mydf = comtradeapicall.checkAsyncDataRequest(subscription_key,
                                            batchId ='2f92dd59-9763-474c-b27c-4af9ce16d454' )
  ```
  - Submit final data  asynchronous job and download the resulting file
  ``` python
  comtradeapicall.downloadAsyncFinalDataRequest(subscription_key, directory,  typeCode='C', freqCode='M',
                                          clCode='HS', period='202209', reporterCode=None, cmdCode='91,90',
                                          flowCode='M', partnerCode=None, partner2Code=None,
                                          customsCode=None, motCode=None)
  ```
  - Submit tariffline data  asynchronous job and download the resulting file
  ``` python
  comtradeapicall.downloadAsyncTarifflineDataRequest(subscription_key, directory,  typeCode='C', freqCode='M',
                                          clCode='HS', period='202209', reporterCode=None, cmdCode='91,90',
                                          flowCode='M', partnerCode=None, partner2Code=None,
                                          customsCode=None, motCode=None)
  ```
  - View list of reference tables
  ``` python
  mydf = comtradeapicall.listReference()
  mydf = comtradeapicall.listReference('cmd:B5')
  ```
  - Download specific reference
  ``` python
  mydf = comtradeapicall.getReference('reporter')
  mydf = comtradeapicall.getReference('partner')
  ```
  - Convert country/area ISO3 to Comtrade code
  ``` python
  country_code = comtradeapicall.convertCountryIso3ToCode('USA,FRA,CHE,ITA')
  ```
  - Get the Standard unit value (qtyUnitCode 8 [kg]) for commodity 010391 in 2022
  ``` python
  mydf = comtradeapicall.getSUV(subscription_key, period='2022', cmdCode='010391', flowCode=None, qtyUnitCode=8)
  ```
  - Get number of port calls and trade volume estimates derrived from AIS data for Australia between 1 and 8 February 2023 with vessel types bulk and container.
  ``` python
  mydf = comtradeapicall.getAIS(subscription_key, countryareaCode=36, vesselTypeCode='1,2', dateFrom='2023-02-01', dateTo='2023-02-08')
  ```
  - Tests folder contain more examples including calculation of unit value


  The naming convention follows the following : "COMTRADE-\<DATA>-\<TYPE>\<FREQ>\<COUNTRY CODE>\<YEAR\[
  -MONTH\]>\<CLASSIFICATION CODE>\[\<RELEASE DATE\>\]"

  As examples:
  - Final merchandise trade data from Morocco (code 504) in March 2000 released on 3 Jan 2023 coded using H1
   classification:
    - *COMTRADE-FINAL-CM504200003H1[2023-01-03]*
  - Tariffline merchandise trade from Morocco (code 504) in March 2000 released on 3 Jan 2023 coded using H1 classification:
    - *COMTRADE-TARIFFLINE-CM504200003H1[2023-01-03]*

  Note: Async download retains the original batch id


pkgId: 3af93423ed0c850d4f751254139886affc802d3a2f6f5d9c346ae5c4d5a3f9c0
name: python-comtraderecord
arch: src
version: 1.0.2
epoch: 0
release: 1
summary: Read/Write comtrade recording, commonly used algorithm for recording.
url: https://pypi.org/project/comtraderecord/
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-comtraderecord-1.0.2-1.src.rpm
checksum_type: sha256
description:
  ```shell
  pip3 uninstall comtraderecord
  pip3 install comtraderecord
  ```




  - pyComtrade
    - read comtrade file

  - pyRelayAlg
    - commonly used algorithm for comtrade recording

  - ComtradeWriter
    - generate comtrade file





  ```python

  import matplotlib.pyplot as plt
  from comtraderecord import pyComtrade, pyRelayAlg

  recFile = pyComtrade.ComtradeFile(r"D:\CustomerCare\FRA00001.CFG")


  VA = recFile.getAnalogData("MPV3p1:V A")
  IB = recFile.getAnalogData(2)

  pickup = recFile.getDigitalData(1)
  operate = recFile.getDigitalData("VI3p1:Group indicat.:Operate:general")


  timestamp = recFile.getTime()



  plt.subplot(5,1,1)
  plt.plot(timestamp,VA)
  plt.title("sampling VA")
  plt.subplot(5,1,2)
  plt.plot(timestamp,IB)
  plt.title("sampling IB")

  plt.subplot(5,1,3)
  mag_Va = abs(pyRelayAlg.fourier(VA,recFile.samplePoint))
  plt.plot(timestamp,mag_Va)
  plt.title("magnitude VA")

  plt.subplot(5,1,4)
  plt.plot(timestamp,pickup)
  plt.title("pickup")
  plt.subplot(5,1,5)
  plt.plot(timestamp,operate)
  plt.title("operate")

  plt.show()

  print("samplePoint:",recFile.samplePoint)
  print("sampleRate:",recFile.sampleRate)
  print("numberOfSamples:",recFile.numberOfSamples)
  print("analog_id:",recFile.analog_id)
  print("analog_An:",recFile.analog_An)
  print("digital_id:",recFile.digital_id)
  print("digital_Dn:",recFile.digital_Dn)
  print("freq:",recFile.freq)
  print("fileType:",recFile.fileType)
  print("startTime:",recFile.startTime)
  print("triggerTime:",recFile.triggerTime)

  print("\n\nrecording cfg data:")
  for k,v in recFile.cfg_data.items():
    print(f"{k}:{v}")

  ```


  ```python

  recFile = pyComtrade.ComtradeFile(r"D:\CustomerCare\FRA00001.CFG")


  VA = recFile.getAnalogData("MPV3p1:V A")

  Harm_5th = pyRelayAlg.fourier(VA,samplePoint,2,5)

  TrueRMS = pyRelayAlg.TrueRMS(VA,samplePoint)

  ```


  ```python
  from comtraderecord import comtradeWriter
  import numpy as np
  import math

  time_s = 0.2
  sample = 1000
  ratedFreq = 50.0
  ts = np.arange(0,time_s,1/sample)
  fi = 2*np.pi*ratedFreq

  recData = [0]*3
  recData[0] = 5.0*math.sqrt(2)*np.sin(fi*ts)
  recData[1] = 2.0*math.sqrt(2)*np.sin(fi*ts+np.pi*4/3)
  recData[2] = 5.0*math.sqrt(2)*np.sin(fi*ts+np.pi*2/3)


  f = comtradeWriter("testRec",recData,sample,ratedFreq)
  channelID = [
      'Line VA',
      'Line VB',
      'Line VC'
      ]
  f.setChannelID(channelID)
  f.setUnit(['V','V','V'])
  f.createFile()

  ```


pkgId: 6a7c434e26b62db6aa0dbc58ae2c395212c6660ee5bb29e5806135a86858e3d3
name: python-comtypes
arch: src
version: 1.4.7
epoch: 0
release: 1
summary: Pure Python COM package
url: https://github.com/enthought/comtypes
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-comtypes-1.4.7-1.src.rpm
checksum_type: sha256
description:
  ![Works on Windows only](https://img.shields.io/badge/-Windows-0078D6.svg?logo=windows&style=flat)
  [![PyPI version](https://badge.fury.io/py/comtypes.svg)](https://pypi.org/project/comtypes/) [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/comtypes)](https://pypi.org/project/comtypes/) [![PyPI - License](https://img.shields.io/pypi/l/comtypes)](https://pypi.org/project/comtypes/) [![PyPI - Downloads](https://img.shields.io/pypi/dm/comtypes)](https://pypi.org/project/comtypes/)
  [![GitHub Repo stars](https://img.shields.io/github/stars/enthought/comtypes?style=social)](https://github.com/enthought/comtypes/stargazers) [![GitHub forks](https://img.shields.io/github/forks/enthought/comtypes?style=social)](https://github.com/enthought/comtypes/network/members)
  [![Tidelift Subscription](https://tidelift.com/badges/package/pypi/comtypes)](https://tidelift.com/subscription/pkg/pypi-comtypes?utm_source=pypi-comtypes&utm_medium=readme)


  `comtypes` is a lightweight pure Python [COM](https://learn.microsoft.com/en-us/windows/win32/com/component-object-model--com--portal) package based on the [`ctypes`](https://docs.python.org/library/ctypes.html) foreign function interface library.

  `comtypes` allows you to define, call, and implement custom and dispatch-based COM interfaces in pure Python.

  `comtypes` requires Windows and Python 3.8 or later.
  - Version [1.4.6](https://pypi.org/project/comtypes/1.4.6/) is the last version to support Python 3.7.
  - Version [1.2.1](https://pypi.org/project/comtypes/1.2.1/) is the last version to support Python 2.7 and 3.3–3.6.
  - `comtypes` does not work with Python 3.8.1 as reported in [GH-202](https://github.com/enthought/comtypes/issues/202). This bug has been fixed in Python >= 3.8.2.
  - Certain `comtypes` functions may not work correctly in Python 3.8 and 3.9 as reported in [GH-212](https://github.com/enthought/comtypes/issues/212). This bug has been fixed in Python >= 3.10.10 and >= 3.11.2.



  `comtypes` is available on [PyPI](https://pypi.org/project/comtypes) and can be installed with `pip`:
  ```sh

  pip install comtypes
  ```

  The source code is currently hosted [here](https://github.com/enthought/comtypes) on GitHub.



  `comtypes` is a pure Python package — it has no additional required dependencies.

  Optional functionalities can be enabled by installing:
  - `numpy`, in order to process arrays as `numpy`'s `ndarray`.
  - `mypy` or other static type checkers, in order to interpret type hints.

  None of these packages, however, are required in order to run `comtypes`.



  The [GitHub repository](https://github.com/enthought/comtypes) is used for tracking issues, reporting bugs, and contributing to the codebase and documentation.



  Available as part of the Tidelift Subscription.

  This project and the maintainers of thousands of other packages are working with Tidelift to deliver one enterprise subscription that covers all of the open source you use.

  [Learn more](https://tidelift.com/subscription/pkg/pypi-comtypes?utm_source=pypi-comtypes&utm_medium=referral&utm_campaign=github).




  **A:** The [Microsoft Component Object Model (COM)](https://learn.microsoft.com/en-us/windows/win32/com/com-technical-overview) is a technology that is unique to Windows and is not supported on other platforms.

  [The phrase _"COM is a platform-independent"_ in the MS documentation](https://learn.microsoft.com/en-us/windows/win32/com/component-object-model--com--portal) means that COM maintains compatibility across different versions of Windows, and does NOT imply that it is supported on Linux or Mac.

  For as long as COM is not supported outside of Windows, there is no plan to port `comtypes` to other platforms.


  **A:** The [`_ctypes`](https://github.com/python/cpython/blob/main/Modules/_ctypes/_ctypes.c) is part of the internal implementation of the [`ctypes`](https://github.com/python/cpython/blob/main/Lib/ctypes/) standard library that exists for Python on all platforms.
  However, `COMError` and COM-related features are only implemented in Python for Windows.

  In cross-platform software development, care must be taken to ensure that codebases that depend on `comtypes` do not execute in environments other than Windows.



  **A:** Executing `py -m comtypes.clear_cache` and then running the script again might resolve the problem.

  When `comtypes.client.GetModule` is called (either directly or indirectly), `comtypes` generates Python module files.
  If Python is forced to terminate or crashes in the middle of file generation, the codebase written to the file becomes partial.
  When Python tries to import this unexecutable partial codebase module, an error occurs.

  Executing `py -m comtypes.clear_cache` identifies the directories where the "cache module files" are stored and deletes them.
  After deleting these partial modules and running the script again, `comtypes.client.GetModule` is called and executable modules are generated anew.

  However, if the script implementation does not use `comtypes.client.GetModule` or processes generated files, it may not be a solution.



  The documentation is currently hosted [here](https://pythonhosted.org/comtypes) on PythonHosted.


pkgId: 82c4d2003844ce7cbda5e6b57c65eb757dca8f66d58e27c64b434acfea9752d1
name: python-con-duct
arch: src
version: 0.8.0
epoch: 0
release: 1
summary: Runs a not-so-simple command and collects resource usage metrics
url: https://github.com/con/duct/
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-con-duct-0.8.0-1.src.rpm
checksum_type: sha256
description:
  [![codecov](https://codecov.io/gh/con/duct/graph/badge.svg?token=JrPazw0Vn4)](https://codecov.io/gh/con/duct)
  [![PyPI version](https://badge.fury.io/py/con-duct.svg)](https://badge.fury.io/py/con-duct)



      pip install con-duct



  Try it out!

      duct --sample-interval 0.5 --report-interval 1 test/data/test_script.py --duration 3 --memory-size=1000

  `duct` is most useful when the report-interval is less than the duration of the script.



  A process wrapper script that monitors the execution of a command.

  <!-- BEGIN HELP -->
  ```shell
  >duct --help

  usage: duct [-h] [--version] [-p OUTPUT_PREFIX]
              [--summary-format SUMMARY_FORMAT] [--colors] [--clobber]
              [-l {NONE,CRITICAL,ERROR,WARNING,INFO,DEBUG}] [-q]
              [--sample-interval SAMPLE_INTERVAL]
              [--report-interval REPORT_INTERVAL] [-c {all,none,stdout,stderr}]
              [-o {all,none,stdout,stderr}]
              [-t {all,system-summary,processes-samples}]
              command [command_args ...] ...

  duct is a lightweight wrapper that collects execution data for an arbitrary
  command.  Execution data includes execution time, system information, and
  resource usage statistics of the command and all its child processes. It is
  intended to simplify the problem of recording the resources necessary to
  execute a command, particularly in an HPC environment.

  Resource usage is determined by polling (at a sample-interval).
  During execution, duct produces a JSON lines (see https://jsonlines.org) file
  with one data point recorded for each report (at a report-interval).

  environment variables:
    Many duct options can be configured by environment variables (which are
    overridden by command line options).

    DUCT_LOG_LEVEL: see --log-level
    DUCT_OUTPUT_PREFIX: see --output-prefix
    DUCT_SUMMARY_FORMAT: see --summary-format
    DUCT_SAMPLE_INTERVAL: see --sample-interval
    DUCT_REPORT_INTERVAL: see --report-interval
    DUCT_CAPTURE_OUTPUTS: see --capture-outputs

  positional arguments:
    command [command_args ...]
                          The command to execute, along with its arguments.
    command_args          Arguments for the command.

  options:
    -h, --help            show this help message and exit
    --version             show program's version number and exit
    -p OUTPUT_PREFIX, --output-prefix OUTPUT_PREFIX
                          File string format to be used as a prefix for the
                          files -- the captured stdout and stderr and the
                          resource usage logs. The understood variables are
                          {datetime}, {datetime_filesafe}, and {pid}. Leading
                          directories will be created if they do not exist. You
                          can also provide value via DUCT_OUTPUT_PREFIX env
                          variable. (default:
                          .duct/logs/{datetime_filesafe}-{pid}_)
    --summary-format SUMMARY_FORMAT
                          Output template to use when printing the summary
                          following execution. Accepts custom conversion flags:
                          !S: Converts filesizes to human readable units, green
                          if measured, red if None. !E: Colors exit code, green
                          if falsey, red if truthy, and red if None. !X: Colors
                          green if truthy, red if falsey. !N: Colors green if
                          not None, red if None (default: Summary: Exit Code:
                          {exit_code!E} Command: {command} Log files location:
                          {logs_prefix} Wall Clock Time: {wall_clock_time:.3f}
                          sec Memory Peak Usage (RSS): {peak_rss!S} Memory
                          Average Usage (RSS): {average_rss!S} Virtual Memory
                          Peak Usage (VSZ): {peak_vsz!S} Virtual Memory Average
                          Usage (VSZ): {average_vsz!S} Memory Peak Percentage:
                          {peak_pmem!N}% Memory Average Percentage:
                          {average_pmem!N}% CPU Peak Usage: {peak_pcpu!N}%
                          Average CPU Usage: {average_pcpu!N}% )
    --colors              Use colors in duct output. (default: False)
    --clobber             Replace log files if they already exist. (default:
                          False)
    -l {NONE,CRITICAL,ERROR,WARNING,INFO,DEBUG}, --log_level {NONE,CRITICAL,ERROR,WARNING,INFO,DEBUG}
                          Level of log output to stderr, use NONE to entirely
                          disable. (default: INFO)
    -q, --quiet           [deprecated, use log level NONE] Disable duct logging
                          output (to stderr) (default: False)
    --sample-interval SAMPLE_INTERVAL, --s-i SAMPLE_INTERVAL
                          Interval in seconds between status checks of the
                          running process. Sample interval must be less than or
                          equal to report interval, and it achieves the best
                          results when sample is significantly less than the
                          runtime of the process. (default: 1.0)
    --report-interval REPORT_INTERVAL, --r-i REPORT_INTERVAL
                          Interval in seconds at which to report aggregated
                          data. (default: 60.0)
    -c {all,none,stdout,stderr}, --capture-outputs {all,none,stdout,stderr}
                          Record stdout, stderr, all, or none to log files. You
                          can also provide value via DUCT_CAPTURE_OUTPUTS env
                          variable. (default: all)
    -o {all,none,stdout,stderr}, --outputs {all,none,stdout,stderr}
                          Print stdout, stderr, all, or none to stdout/stderr
                          respectively. (default: all)
    -t {all,system-summary,processes-samples}, --record-types {all,system-summary,processes-samples}
                          Record system-summary, processes-samples, or all
                          (default: all)

  ```
  <!-- END HELP -->



  In addition to `duct`, this project also includes a set of optional helpers under the `con-duct` command.
  These helpers may use 3rd party python libraries.



      pip install con-duct[all]



  <!-- BEGIN EXTRAS HELP -->
  ```shell
  >con-duct --help

  usage: con-duct <command> [options]

  A command-line tool for managing various tasks.

  positional arguments:
    {pp,plot}   Available subcommands
      pp        Pretty print a JSON log
      plot      Plot resource usage for an execution.

  options:
    -h, --help  show this help message and exit

  ```
  <!-- END EXTRAS HELP -->


pkgId: e7b51ae3b9e00d07caf2029ec7c4f4e26d46a20cf9fde393c86a2337f9122ccc
name: python-con-duct
arch: src
version: 0.9.0
epoch: 0
release: 1
summary: Runs a not-so-simple command and collects resource usage metrics
url: https://github.com/con/duct/
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-con-duct-0.9.0-1.src.rpm
checksum_type: sha256
description:
  [![codecov](https://codecov.io/gh/con/duct/graph/badge.svg?token=JrPazw0Vn4)](https://codecov.io/gh/con/duct)
  [![PyPI version](https://badge.fury.io/py/con-duct.svg)](https://badge.fury.io/py/con-duct)



      pip install con-duct



  Try it out!

      duct --sample-interval 0.5 --report-interval 1 test/data/test_script.py --duration 3 --memory-size=1000

  `duct` is most useful when the report-interval is less than the duration of the script.



  A process wrapper script that monitors the execution of a command.

  <!-- BEGIN HELP -->
  ```shell
  >duct --help

  usage: duct [-h] [--version] [-p OUTPUT_PREFIX]
              [--summary-format SUMMARY_FORMAT] [--colors] [--clobber]
              [-l {NONE,CRITICAL,ERROR,WARNING,INFO,DEBUG}] [-q]
              [--sample-interval SAMPLE_INTERVAL]
              [--report-interval REPORT_INTERVAL] [--fail-time FAIL_TIME]
              [-c {all,none,stdout,stderr}] [-o {all,none,stdout,stderr}]
              [-t {all,system-summary,processes-samples}]
              command [command_args ...] ...

  duct is a lightweight wrapper that collects execution data for an arbitrary
  command.  Execution data includes execution time, system information, and
  resource usage statistics of the command and all its child processes. It is
  intended to simplify the problem of recording the resources necessary to
  execute a command, particularly in an HPC environment.

  Resource usage is determined by polling (at a sample-interval).
  During execution, duct produces a JSON lines (see https://jsonlines.org) file
  with one data point recorded for each report (at a report-interval).

  environment variables:
    Many duct options can be configured by environment variables (which are
    overridden by command line options).

    DUCT_LOG_LEVEL: see --log-level
    DUCT_OUTPUT_PREFIX: see --output-prefix
    DUCT_SUMMARY_FORMAT: see --summary-format
    DUCT_SAMPLE_INTERVAL: see --sample-interval
    DUCT_REPORT_INTERVAL: see --report-interval
    DUCT_CAPTURE_OUTPUTS: see --capture-outputs

  positional arguments:
    command [command_args ...]
                          The command to execute, along with its arguments.
    command_args          Arguments for the command.

  options:
    -h, --help            show this help message and exit
    --version             show program's version number and exit
    -p OUTPUT_PREFIX, --output-prefix OUTPUT_PREFIX
                          File string format to be used as a prefix for the
                          files -- the captured stdout and stderr and the
                          resource usage logs. The understood variables are
                          {datetime}, {datetime_filesafe}, and {pid}. Leading
                          directories will be created if they do not exist. You
                          can also provide value via DUCT_OUTPUT_PREFIX env
                          variable. (default:
                          .duct/logs/{datetime_filesafe}-{pid}_)
    --summary-format SUMMARY_FORMAT
                          Output template to use when printing the summary
                          following execution. Accepts custom conversion flags:
                          !S: Converts filesizes to human readable units, green
                          if measured, red if None. !E: Colors exit code, green
                          if falsey, red if truthy, and red if None. !X: Colors
                          green if truthy, red if falsey. !N: Colors green if
                          not None, red if None (default: Summary: Exit Code:
                          {exit_code!E} Command: {command} Log files location:
                          {logs_prefix} Wall Clock Time: {wall_clock_time:.3f}
                          sec Memory Peak Usage (RSS): {peak_rss!S} Memory
                          Average Usage (RSS): {average_rss!S} Virtual Memory
                          Peak Usage (VSZ): {peak_vsz!S} Virtual Memory Average
                          Usage (VSZ): {average_vsz!S} Memory Peak Percentage:
                          {peak_pmem:.2f!N}% Memory Average Percentage:
                          {average_pmem:.2f!N}% CPU Peak Usage:
                          {peak_pcpu:.2f!N}% Average CPU Usage:
                          {average_pcpu:.2f!N}% )
    --colors              Use colors in duct output. (default: False)
    --clobber             Replace log files if they already exist. (default:
                          False)
    -l {NONE,CRITICAL,ERROR,WARNING,INFO,DEBUG}, --log_level {NONE,CRITICAL,ERROR,WARNING,INFO,DEBUG}
                          Level of log output to stderr, use NONE to entirely
                          disable. (default: INFO)
    -q, --quiet           [deprecated, use log level NONE] Disable duct logging
                          output (to stderr) (default: False)
    --sample-interval SAMPLE_INTERVAL, --s-i SAMPLE_INTERVAL
                          Interval in seconds between status checks of the
                          running process. Sample interval must be less than or
                          equal to report interval, and it achieves the best
                          results when sample is significantly less than the
                          runtime of the process. (default: 1.0)
    --report-interval REPORT_INTERVAL, --r-i REPORT_INTERVAL
                          Interval in seconds at which to report aggregated
                          data. (default: 60.0)
    --fail-time FAIL_TIME, --f-t FAIL_TIME
                          If command fails in less than this specified time,
                          duct would remove logs. Set to 0 if you would like to
                          keep logs for a failing command regardless of its run
                          time. Set to negative (e.g. -1) if you would like to
                          not keep logs for any failing command. (default: 3.0)
    -c {all,none,stdout,stderr}, --capture-outputs {all,none,stdout,stderr}
                          Record stdout, stderr, all, or none to log files. You
                          can also provide value via DUCT_CAPTURE_OUTPUTS env
                          variable. (default: all)
    -o {all,none,stdout,stderr}, --outputs {all,none,stdout,stderr}
                          Print stdout, stderr, all, or none to stdout/stderr
                          respectively. (default: all)
    -t {all,system-summary,processes-samples}, --record-types {all,system-summary,processes-samples}
                          Record system-summary, processes-samples, or all
                          (default: all)

  ```
  <!-- END HELP -->



  In addition to `duct`, this project also includes a set of optional helpers under the `con-duct` command.
  These helpers may use 3rd party python libraries.



      pip install con-duct[all]



  <!-- BEGIN EXTRAS HELP -->
  ```shell
  >con-duct --help

  usage: con-duct <command> [options]

  A suite of commands to manage or manipulate con-duct logs.

  positional arguments:
    {pp,plot}   Available subcommands
      pp        Pretty print a JSON log.
      plot      Plot resource usage for an execution.

  options:
    -h, --help  show this help message and exit

  ```
  <!-- END EXTRAS HELP -->





  By default, [git-annex](https://git-annex.branchable.com/) treats all dotfiles, and files under directories starting with a `.` as "small" regardless of `annex.largefiles` setting [[ref: an issue describing the logic](https://git-annex.branchable.com/bugs/add__58___inconsistently_treats_files_in_dotdirs_as_dotfiles/?updated#comment-efc1f2aa8f46e88a8be9837a56cfa6f7)].
  It is necessary to set `annex.dotfiles` variable to `true` to make git-annex treat them as regular files and thus subject to `annex.largefiles` setting [[ref: git-annex config](https://git-annex.branchable.com/git-annex-config/)].
  Could be done the repository (not just specific clone, but any instance since records in `git-annex` branch) wide using `git annex config --set annex.dotfiles true`.


pkgId: ba3165d2294c290d2e7f314554913f5e38a09581c2f50a4ee6731ab491a54d7e
name: python-conan
arch: src
version: 2.10.0
epoch: 0
release: 1
summary: Conan C/C++ package manager
url: https://conan.io
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-conan-2.10.0-1.src.rpm
checksum_type: sha256
description:
  Ran 146 tests in 50.993s
  OK
  ```
  To run specific tests, you can specify the test name too, something like:
  ```bash
  $ python -m pytest test/functional/command/export_test.py::TestRevisionModeSCM::test_revision_mode_scm -s
  ```
  The `-s` argument can be useful to see some output that otherwise is captured by *pytest*.
  Also, you can run tests against an instance of Artifactory. Those tests should add the attribute
  `artifactory_ready`.
  ```bash
  $ python -m pytest . -m artifactory_ready
  ```
  Some environment variables have to be defined to run them. For example, for an
  Artifactory instance that is running on the localhost with default user and password configured, the
  variables could take the values:
  ```bash
  $ export CONAN_TEST_WITH_ARTIFACTORY=1
  $ export ARTIFACTORY_DEFAULT_URL=http://localhost:8081/artifactory
  $ export ARTIFACTORY_DEFAULT_USER=admin
  $ export ARTIFACTORY_DEFAULT_PASSWORD=password
  ```
  `ARTIFACTORY_DEFAULT_URL` is the base URL for the Artifactory repo, not one for a specific
  repository. Running the tests with a real Artifactory instance will create repos on the fly so please
  use a separate server for testing purposes.

  [MIT LICENSE](LICENSE.md)


pkgId: 9d62a66bcfb0acfa53aea12e7101b595a929930d792abd1f050ef8240503b572
name: python-conan-app-launcher
arch: src
version: 2.2.0
epoch: 0
release: 1
url: https://pypi.org/project/conan-app-launcher/
rpm_license: LGPL-3.0-only
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-conan-app-launcher-2.2.0-1.src.rpm
checksum_type: sha256
summary:
  GUI for Conan users providing an easy way to install and manage local packages and launch application from them.

description:
  ![https://pypi.org/project/conan-explorer/](https://img.shields.io/pypi/v/conan-explorer?logo=pypi)
  ![PyPI Python versions](https://img.shields.io/pypi/pyversions/conan-explorer?logo=python)
  ![Windows](https://img.shields.io/badge/Windows-blue?logo=windows10&logoColor=white)
  ![Linux](
  https://img.shields.io/badge/Linux-purple?logo=linux&logoColor=white)
  ![MilestoneProgress](https://img.shields.io/github/milestones/progress-percent/goszpeti/conan_explorer/21?logo=conan)
  [![Python Tests](https://github.com/goszpeti/conan_explorer/actions/workflows/test.yml/badge.svg)](https://github.com/goszpeti/conan_explorer/actions/workflows/test.yml)
  ![SonarStatus](https://sonarcloud.io/api/project_badges/measure?project=goszpeti_conan_explorer&metric=alert_status)
  ![Downloads](https://img.shields.io/pypi/dm/conan_explorer)




  This package has been renamed from version 2.2.0 onwards. Use `pip install conan-explorer` instead.

  New package: https://pypi.org/project/conan-explorer/



  The goal of this project is to provide a standalone Graphical User Interface (GUI) to
  * Start executables contained in packages of the package manager [Conan](https://conan.io/)
  * Browse the local package cache
  * Search Packages in remotes
  * Configure Remotes and Profiles

  It is end-user oriented and focuses on using packages, rather then developing them. It can be used on Windows and Linux x64 platforms.













  **Main Features**
  - compatible with a wide range of conan versions (from 1.24 onwards)
  - integrated console for information an packages and config file
  - installable with pip

  **📑 Quicklaunch**

  - configurable layout (tabs and applications) in the GUI
  - list and grid view
  - can also open files with their associated default program
  - installs all referenced packages automatically
  - automatic conan settings resolution for your platform
  - uses the default icons of files or can be configured to use custom ones
  - quick controls to switch between versions and channels

  **📦 Local Package Explorer**

  - tabbed view for browsing through installed packages
  - understand package settings at a glance from a shortened representation, e.g. Linux_x64_ggc7
  - supports rename/copy/cut/paste on file, open in file explorer, copy reference, etc. functions which are often needed in daily Conan workflow

  **🔍 Conan Search**

  - search for references in selected remotes
  - show all existing packages for a reference and their infos
  - install package directly via right-click menu

  **📝 Conan Config**

  - view and edit your profiles
  - view and edit your remotes - with multi-login to the same artifactory server for multiple remotes
  - see the most important paths and config at one glance

  **🔌 Plugin Mechanism**

  - All views are now plugins, which can be extended by the user





  Currently testing and compatibility is only endured for Debian based distros, specifically Ubuntu 20.04 and 22.04. If you want to run on Ubuntu 18 you can only use the 1.4.3 release, due to the Qt dependency.

  1. Pip must be updated to at least pip 20.3, so using a venv like this is recommended:

  ```bash
  sudo apt-get install python3-venv
  python3 -m venv .venv
  source .venv/bin/activate
  python3 -m pip install --upgrade pip
  pip install conan_explorer
  ```

  2. An x-terminal emulator must be available for "Open Files in cmd" and console based programs for the App Grid. Type `x-terminal-emulator` to get a list of available terminals.

  3. To open files with its associated program xdg-open is used: `sudo apt install xdg-utils`

  4. Not all Qt6 versions support the Wayland lib of the operating system.
    * For Ubuntu 20.04 please execute "pip install PySide6-Essentials==6.4.3" to get the correct Qt6 version and update pip to latest version!
    * For Ubuntu 22.04 please ensure that the system Qt6 packages are available. Simply execute `sudo apt-get install qt6-wayland` on a wayland system, or `sudo apt-get install qt6-base-dev` for an X11 based system.




      pip install conan-explorer



  After checkout use the command:

      pip install .



  Execute `conan-explorer` if the Python "scripts" or "bin" folder is on your system path, or look it up manually in the site-packages folder.
  You can also assign its icon to it from the site packages folder in **conan_explorer/assets/icons/icon.ico**.



  * Pyside6 >= 6.4.0
  * 1.48.0 <= conan < 2.1

  > ⚠ **Warning** - **Deprecation of Python 3.X**
  > From version 2.0.0 Python 3.6 support will be dropped, having reached end-of-life.
  > From version 2.2.0 Python 3.7 support will be dropped, having reached end-of-life.



  This project uses Python with Qt as a frontend using the PySide6 integration.
  An IDE configuration is available for VsCode.

  The .ui files are created with Qt Designer and automatically converted to .py code when starting the application. The Designer can be installed via the PySide6-Addons pip package.

  See https://sonarcloud.io/project/overview?id=goszpeti_conan_explorer for Static Code Analysis.



  > **Warning** - **Change of License to LGPL**
  > From version 2.0.0 the project will use the [LGPL 3.0](https://www.gnu.org/licenses/lgpl-3.0.en.html) license to cleanly comply with PySide6.


  * Conan Package Manager Icon by Conan.io developers under [MIT License](http://opensource.org/licenses/mit-license.php), via Wikimedia Commons
  * [Noto Sans and Noto Sans Mono fonts](https://fonts.google.com/) by Google under [SIL Open Font License](https://scripts.sil.org/cms/scripts/page.php?site_id=nrsi&id=OFL)
  * [Material icons](https://fonts.google.com/) by Google  under [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0)
  * [Linux icon](https://www.svgrepo.com/svg/340563/linux-alt) by Carbon Design  under [Apache License](https://opensource.org/licenses/Apache-1.1)
  * [Apple icon](https://www.svgrepo.com/svg/488495/apple) by Klever Space  under [MIT License](http://opensource.org/licenses/mit-license.php)
  * [Windows icon](https://www.svgrepo.com/svg/488736/windows) by Klever Space under [MIT License](http://opensource.org/licenses/mit-license.php)
  * Modified [Package icon](https://www.svgrepo.com/svg/487645/package) by Neuicons [MIT License](http://opensource.org/licenses/mit-license.php)
  * Modified [Open Box icon](https://www.svgrepo.com/svg/383786/open-box-parcel) by wishforge.gamesunder [CC Attribution License](https://creativecommons.org/licenses/by/4.0/legalcode)</li>


  * PySide6 by Qt, [LGPL V3](https://www.gnu.org/licenses/lgpl-3.0.en.html)
  * Conan by JFrog LTD under [MIT License](<http://opensource.org/licenses/mit-license.php>)
  * jsonschema by Julian Berman under [MIT License](<http://opensource.org/licenses/mit-license.php>)
  * dictdiffer by Invenio Collaboration under [MIT License](<http://opensource.org/licenses/mit-license.php>)
  * Using a modified version of Toggle Widget from QtWidgets (https://github.com/pythonguis/python-qtwidgets) under [MIT License](<http://opensource.org/licenses/mit-license.php>)


  * contextlib-chdir by Álvaro Mondéjar Rubio under [BSD License (BSD-3-Clause) ](https://opensource.org/license/BSD-3-clause/)
  * typing-extensions by Guido van Rossum, Jukka Lehtosalo, Łukasz Langa, Michael Lee under [Python Software Foundation License(PSF)](https://docs.python.org/3/license.html)


pkgId: 118f8a793dd312e7230990de82526dad4f657d49fd2046bc5fe396866414bc01
name: python-conan-zbuild
arch: src
version: 1.65.2
epoch: 0
release: 1
summary: Conan C/C++ package manager
url: https://conan.io
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-conan-zbuild-1.65.2-1.src.rpm
checksum_type: sha256
description:
  Decentralized, open-source (MIT), C/C++ package manager.
  - Homepage: https://conan.io/
  - Github: https://github.com/conan-io/conan
  - Docs: https://docs.conan.io/en/latest/
  - Slack: https://cppalliance.org/slack/ (#conan channel)
  - Twitter: https://twitter.com/conan_io
  Conan is a package manager for C and C++ developers:
  - It is fully decentralized. Users can host their packages on their servers, privately. Integrates with Artifactory and Bintray.
  - Portable. Works across all platforms, including Linux, OSX, Windows (with native and first-class support, WSL, MinGW),
    Solaris, FreeBSD, embedded and cross-compiling, docker, WSL
  - Manage binaries. It can create, upload and download binaries for any configuration and platform,
    even cross-compiling, saving lots of time in development and continuous integration. The binary compatibility can be configured
    and customized. Manage all your artifacts in the same way on all platforms.
  - Integrates with any build system, including any proprietary and custom one. Provides tested support for major build systems
    (CMake, MSBuild, Makefiles, Meson, etc).
  - Extensible: Its python based recipes, together with extensions points allows for great power and flexibility.
  - Large and active community, especially in Github (https://github.com/conan-io/conan) and Slack (https://cpplang-inviter.cppalliance.org/ #conan channel).
    This community also creates and maintains packages in ConanCenter and Bincrafters repositories in Bintray.
  - Stable. Used in production by many companies, since 1.0 there is a commitment not to break package recipes and documented behavior.
  +-------------------------+-------------------------+
  | **develop**             |    **Code Climate**     |
  +=========================+=========================+
  | |Build Status Develop|  |   |Develop climate|     |
  +-------------------------+-------------------------+


pkgId: 2112cc7045356f078a878b6920d87c8006107b35807048e92383c51174d19871
name: python-conanex
arch: src
version: 2.1.1
epoch: 0
release: 1
summary: Conan Extended C/C++ package manager
url: https://github.com/redradist/conanex
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-conanex-2.1.1-1.src.rpm
checksum_type: sha256
description:
  What does it for ?

  ConanEx is using [`conan`](https://github.com/conan-io/conan) as underlying tool.
  ConanEx is a command line wrapper around `conan` with additional features.

  Consider the following workflow:
  ```console
  wget https://github.com/google/flatbuffers/archive/refs/tags/v22.10.26.zip
  unzip v22.10.26.zip -d flatbuffers_22_10_26
  conan create flatbuffers/22.10.26 flatbuffers_22_10_26/
  conan create ctre/3.6 ../../../../compile-time-regular-expressions
  ```

  Such workflow has a following drawbacks:
  1. Not all dependencies are specified in `conanfile.txt`
  2. It adds a boilerplate commands to execute each time when environment should be deployed

  Lets also describe `conancenter` drawbacks:
  1. Centralized repository (do not allow decentralized behaviour)
  2. Hard to add `conanfile.py` receipt to [`conan-center-index`](https://github.com/conan-io/conan-center-index). It takes too long to pass review with all unwritten `conan-center-index` rules for `conanfile.py`

  Lets also describe a current `conanfile.txt` drawbacks:
  1. `conanfile.txt` does not allow specifying dependencies to other package sources like `git`, remote `zip` archive and etc. It makes it less decentralized as claimed

  Let's consider what ConanEx brings to `conanfile.txt` syntax.
  Consider the following `conanfile.txt`:

  ```console
  [requires]
  poco/1.9.4
  flatbuffers/22.10.26 {
      zip = 'https://github.com/google/flatbuffers/archive/refs/tags/v22.10.26.zip',
      sha256 = 'B97C7C017B05F20B17939FEBD413C75201F5B704C8DE49ADB0B35A70D50478CD'
  }
  ctre/3.6 { remote = "conancenter" }










  [options]
  flatbuffers/*:shared=True
  poco/*:shared=True
  ```
  As you can see in this file we have 5 additional ways to install package

  Lets describe them:
  1) `git` allow to download package using Git and run `conanfile.py` located in root directory
  2) `zip` (_url/file_path_) allow installing package from archive, unpack it and run _conanfile.py_ located in root directory.
     There are the following formats that supported: _zip_, _tar.gz_, _tar.bz2_
  3) `conan` (_url/file_path_) if you receipt is completely independent, then you could specify url/path to it to create package.
     Independent means that receipt could download source files by itself.
  4) `path` allow to install package from folder
  5) `remote` specify separate remote for this particular package

  _url/file_path_ supports the hash calculation with options: `md5`, `sha256` and `sha512`

  To install `conanex`:
  ```console
  python3 -m pip install conanex
  ```

  To use `conanex` use it the same way you use `conan`:
  ```console
  conanex install <path_to_conanfile.txt> -pr=<path_to_profile>
  ```

  If you are using `cmake-conan`:
  ```cmake
  if(NOT EXISTS "${CMAKE_BINARY_DIR}/conan.cmake")
      message(STATUS "Downloading conan.cmake from https://github.com/conan-io/cmake-conan")
      file(DOWNLOAD "https://raw.githubusercontent.com/conan-io/cmake-conan/0.18.1/conan.cmake"
           "${CMAKE_BINARY_DIR}/conan.cmake"
           TLS_VERIFY ON)
  endif()

  include(${CMAKE_BINARY_DIR}/conan.cmake)

  set(CONAN_COMMAND conanex)
  conan_cmake_autodetect(settings)
  conan_cmake_install(PATH_OR_REFERENCE ${CMAKE_CURRENT_LIST_DIR}
                      BUILD missing
                      REMOTE conancenter
                      SETTINGS ${settings})
  ```
  Only thing you need is to specify `set(CONAN_COMMAND conanex)` before any conan command


pkgId: 703124398c69dca06da224f20453cfb549140abbc73b030f441e10f6bc4ee6c5
name: python-conbench
arch: src
version: 2023.4.10
epoch: 0
release: 1
summary: Continuous Benchmarking (CB) Framework
url: https://github.com/conbench/conbench
rpm_license: MIT License
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-conbench-2023.4.10-1.src.rpm
checksum_type: sha256
description:
  <p align="right">
  <a href="https://github.com/conbench/conbench/blob/main/.github/workflows/actions.yml"><img alt="Build Status" src="https://github.com/conbench/conbench/actions/workflows/actions.yml/badge.svg?branch=main"></a>
  </p>



  Check out the docs at https://conbench.github.io/conbench.

  <img src="https://raw.githubusercontent.com/conbench/conbench/main/conbench.png" alt="Language-independent Continuous Benchmarking (CB) Framework">


  Conbench allows you to write benchmarks in any language, publish the
  results as JSON via an API, and persist them for comparison while
  iterating on performance improvements or to guard against regressions.

  Conbench includes a runner which can be used as a stand-alone library
  for traditional macro benchmark authoring. The runner will time a unit of
  work (or measure throughput), collect machine information that may be relevant
  for hardware specific optimizations, and return JSON formatted results.

  You can optionally host a Conbench server (API & dashboard) to share
  benchmark results more widely, explore the changes over time, and
  compare results across varying benchmark machines, languages, and cases.

  There is also a Conbench command line interface, useful for Continuous
  Benchmarking (CB) orchestration alongside your development pipeline.

  <p align="center">
      <img src="https://arrow.apache.org/img/arrow.png" alt="Apache Arrow" height="100">
  </p>


  The [Apache Arrow](https://arrow.apache.org/) project is using Conbench
  for Continuous Benchmarking. They have both native Python Conbench
  benchmarks, and Conbench benchmarks written in Python that know how to
  execute their external C++/R/Java/JavaScript benchmarks and record those results
  too. Those benchmarks can be found in the
  [ursacomputing/benchmarks](https://github.com/ursacomputing/benchmarks)
  repository, and the results are hosted on the
  [Arrow Conbench Server](https://conbench.ursa.dev/).


  - May 2021: https://ursalabs.org/blog/announcing-conbench/


  <br>




  All packages in this repo can be installed from PyPI. Each package uses
  [CalVer](https://calver.org/) for versioning. No stability is guaranteed between PyPI
  versions, so consider pinning packages to a specific version in your code.

  ```bash
  pip install benchadapt
  pip install benchalerts
  pip install benchclients
  pip install benchconnect
  pip install benchrun
  pip install conbench  # legacy CLI
  ```

  We typically publish to PyPI often, when new features or bugfixes are needed by users,
  but not on every merge to `main`. To install the latest development version, install
  from git like so:

  ```bash
  pip install 'benchadapt@git+https://github.com/conbench/conbench.git@main#subdirectory=benchadapt/python'
  pip install 'benchalerts@git+https://github.com/conbench/conbench.git@main#subdirectory=benchalerts'
  pip install 'benchclients@git+https://github.com/conbench/conbench.git@main#subdirectory=benchclients/python'
  pip install 'benchconnect@git+https://github.com/conbench/conbench.git@main#subdirectory=benchconnect'
  pip install 'benchrun@git+https://github.com/conbench/conbench.git@main#subdirectory=benchrun/python'
  pip install 'conbench@git+https://github.com/conbench/conbench.git@main'
  ```






  - [`make`](https://www.gnu.org/software/make/), [`docker compose`](https://docs.docker.com/compose/install/): common developer tasks depend on these tools. They need to be set up on your system.
  - `GITHUB_API_TOKEN` environment variable: set up a GitHub API token using [GitHub's instructions](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token). It's recommended to only give the token read-only permissions to public repositories (which is the default for fine-grained personal access tokens). Run `export GITHUB_API_TOKEN="{token}"` in your current shell.



  The following Makefile targets implement common developer tasks. They assume to be run in the root folder of the repository.

  * `make run-app`: This command lets you experiment with Conbench locally.
  It runs the stack in a containerized fashion.
  It rebuilds container images from the current checkout, spawns
  multiple containers (including one for the database), and then exposes Conbench's
  HTTP server on the host at http://127.0.0.1:5000.
  The command will stay in the foreground of your terminal, showing log output of all containers.
  Once you see access log lines like `GET /api/ping/ HTTP/1.1" 200` you can point your browser to http://127.0.0.1:5000.
  You can use `Ctrl+C` to terminate the containerized stack (this only stops containers, and the next invocation of `make run-app` will use previous database state -- invoke `make teardown-app` to stop and remove containers).
  If you wish to clear all database tables during local development you can hit http://127.0.0.1:5000/api/wipe-db with the browser or with e.g. curl.

  * `make run-app-dev`: Similar to `make run-app`, but also mounts the repository's root directory into the container.
  Code changes are (should be) detected automatically and result in automatic code reload.

  * `make tests`: The nobrainer command to run the test suite just like CI does.
  For more fine-grained control see further below.

  * `make lint`: Performs invasive code linting in your local checkout.
  May modify files. Analogue to what CI requires.
  It requires for some commands to be available in your current shell.
  Dependencies can be installed with `pip install -r requirements-dev.txt`.

  * `make alembic-new-migration`: Attempts to generate a new migration Python module in `migrations/versions/`.
  Requires setting the environment variable `ALEMBIC_MIGRATION_NAME` before invocation.
  Example: `export ALEMBIC_MIGRATION_NAME='repo_url_lenth'`.
  After the file was created you may want to change its permissions and re-format it with `black`.

  * `make conbench-on-minikube`: requires [minikube](https://minikube.sigs.k8s.io/docs/start/).
  Deploys the Conbench API server to a local minikube-powered Kubernetes cluster.
  This also deploys a kube-prometheus-based observability stack.
  Use this target for local development in this area.

  * `make docs-build`: Builds HTML docs locally so you may check that they render
    correctly with no linting problems. Dependencies can be installed with
    `pip install -r requirements-dev.txt`. Also, if you're working on the docstrings of
    any of this repo's python packages, ensure the package is installed locally before
    using this command.

    In CI, we use `make build-docs SPHINXOPTS='-W --keep-going'` to fail the build if
    there are Sphinx warnings. When using this command locally, you can just do
    `make build-docs`, but keep an eye on the warnings.



  Point your browser to http://127.0.0.1:5000/api/docs/.



  CI and common developer commands use containerized workflows where dependencies are defined and easy to reason about via `Dockerfile`s.

  Note that the CPython version that Conbench is tested with in CI and that it is recommended to be deployed with is currently the latest 3.11.x release, as also defined in `Dockerfile` at the root of this repository.

  Some developer tasks may involve running Conbench tooling straight on the host.
  Here is how to install the Python dependencies for the Conbench web application:

  ```bash
  pip install -r requirements-webapp.txt
  ```

  Dependencies for running code analysis and tests straight on the host can be installed with

  ```bash
  pip install -r requirements-dev.txt
  ```

  Dependencies for the (legacy) `conbench` CLI can be installed with

  ```bash
  pip install -r requirements-cli.txt
  ```



  If `make test` is too coarse-grained, then this is how to take control of the containerized `pytest` test runner:

  ```bash
  docker compose down && docker compose build app && \
      docker compose run app \
      pytest -vv conbench/tests
  ```

  This command attempts to stop and remove previously spawned test runner containers, and it rebuilds the `app` container image prior to running tests to pick up code changes in the local checkout.

  Useful command line arguments for local development (can be combined as desired):

  * `... pytest -k test_login`: run only string-matching tests
  * `... pytest -x`: exit upon first error
  * `... pytest -s`: do not swallow log output during run
  * `... run -e CONBENCH_LOG_LEVEL_STDERR=DEBUG app ...`



  The following commands are not guaranteed to work as documented, but provide valuable inspiration:



      (conbench) $ brew services start postgres
      (conbench) $ dropdb conbench_prod
      (conbench) $ createdb conbench_prod
      (conbench) $ git checkout main && git pull
      (conbench) $ alembic upgrade head
      (conbench) $ git checkout your-branch
      (conbench) $ alembic revision --autogenerate -m "new"




  1. Start conbench app in Terminal window 1:

          (conbench) $ dropdb conbench_prod && createdb conbench_prod && alembic upgrade head && flask run

  2. Run `conbench.tests.populate_local_conbench` in Terminal window 2 while conbench app is running:

          (conbench) $ python -m conbench.tests.populate_local_conbench




  Kick off a new run of the "Build and upload a package to PyPI" workflow on the [Actions
  page](https://github.com/conbench/conbench/actions).



  To add a new page to our GitHub Pages-hosted documentation:

  1. Add a Markdown file to `docs/pages/`.
  2. In the toctree in the `docs/index.rst` file, add `pages/your_new_page`, where
     `your_new_page` is your new filename without the `.md` file suffix.

  To test that your new pages pass our documentation linter, run the `make docs-build`
  command, as described above.



  The conbench web application can be configured with various environment variables as
  defined in
  [config.py](https://github.com/conbench/conbench/blob/main/conbench/config.py).
  Instructions are in that file.



  By default, conbench has open read access, so a user account is not required to
  view results or read from the API. An account is required only if the conbench
  instance is private or to write data to conbench.

  If you do need an account, follow the login screen's "Sign Up" link, and use the
  registration key specified in the server configuration above. If you are a user
  of conbench, you may need to talk to your user administrator to get the
  registration key. SSO can be configured to avoid requiring the registration key.

  If you have an account and need to create an additional account (say for a machine
  user of the API) either repeat the process if you have the registration key, or if
  you don't have the registration key (say if your account uses SSO), when logged in,
  go to the gear menu / Users and use the "Add User" button to create a new account
  without the registration key.



  There are three main types of benchmarks: "simple benchmarks" that time the
  execution of a unit of work, "external benchmarks" that just record benchmark
  results that were obtained from some other benchmarking tool, and "case
  benchmarks" which benchmark a unit of work under different scenarios (cases).

  Included in this repository are contrived, minimal examples of these different
  kinds of benchmarks to be used as templates for benchmark authoring. These
  example benchmarks and their tests can be found here:


  * [_example_benchmarks.py](https://github.com/conbench/conbench/blob/main/conbench/tests/benchmark/_example_benchmarks.py)
  * [test_cli.py](https://github.com/conbench/conbench/blob/main/conbench/tests/benchmark/test_cli.py)
  * [test_runner.py](https://github.com/conbench/conbench/blob/main/conbench/tests/benchmark/test_runner.py)




  A "simple benchmark" runs and records the execution time of a unit of work.

  Implementation details: Note that this benchmark extends
  `conbench.runner.Benchmark`, implements the minimum required `run()`
  method, and registers itself with the `@conbench.runner.register_benchmark`
  decorator.

  ```python
  import conbench.runner


  @conbench.runner.register_benchmark
  class SimpleBenchmark(conbench.runner.Benchmark):
      name = "addition"

      def run(self, **kwargs):
          yield self.conbench.benchmark(
              self._get_benchmark_function(), self.name, options=kwargs
          )

      def _get_benchmark_function(self):
          return lambda: 1 + 1
  ```

  Successfully registered benchmarks appear in the `conbench --help` list.
  For this benchmark to appear, the file must match the following paterns (documented in the [`utils.py:register_benchmarks` function](https://github.com/conbench/conbench/blob/35fde9954e2b365e36472f16b46fe3067e9dc455/conbench/util.py#L104):
  * `benchmark*.py`
  * `*benchmark.py`
  * `*benchmarks.py`

  ```
  (conbench) $ cd ~/workspace/conbench/conbench/tests/benchmark/
  (conbench) $ conbench --help
  Usage: conbench [OPTIONS] COMMAND [ARGS]...

    Conbench: Language-independent Continuous Benchmarking (CB) Framework

  Options:
    --help  Show this message and exit.

  Commands:
    addition            Run addition benchmark.
    external            Run external benchmark.
    external-r          Run external-r benchmark.
    external-r-options  Run external-r-options benchmark.
    list                List of benchmarks (for orchestration).
    matrix              Run matrix benchmark(s).
    version             Display Conbench version.
  ```

  Benchmarks can be run from command line within the directory where the
  benchmarks are defined.

  Benchmark classes can also be imported and executed via the `run` method which
  accepts the same arguments that appear in the command line help.

  ```
  (conbench) $ cd ~/workspace/conbench/conbench/tests/benchmark/
  (conbench) $ conbench addition --help

  Usage: conbench addition [OPTIONS]

    Run addition benchmark.

  Options:
    --iterations INTEGER   [default: 1]
    --drop-caches BOOLEAN  [default: false]
    --gc-collect BOOLEAN   [default: true]
    --gc-disable BOOLEAN   [default: true]
    --show-result BOOLEAN  [default: true]
    --show-output BOOLEAN  [default: false]
    --run-id TEXT          Group executions together with a run id.
    --run-name TEXT        Free-text name of run (commit ABC, pull request 123,
                           etc).
    --run-reason TEXT      Low-cardinality reason for run (commit, pull request,
                           manual, etc).
    --help                 Show this message and exit.
  ```

  Example command line execution:

  ```
  (conbench) $ cd ~/workspace/conbench/conbench/tests/benchmark/
  (conbench) $ conbench addition

  Benchmark result:
  {
      "batch_id": "c9db942c27db4359923eb08aa553beb7",
      "run_id": "f6c7d0b3b3f146f9b1ad297fc6e5776b",
      "timestamp": "2021-11-09T22:09:17.790397+00:00",
      "context": {
          "benchmark_language": "Python"
      },
      "github": {
          "commit": "61dec915b9dd230ca5029f5e586f8bd95c3e0c05",
          "repository": "https://github.com/conbench/conbench"
      },
      "info": {
          "benchmark_language_version": "Python 3.9.7"
      },
      "machine_info": {
          "architecture_name": "arm64",
          "cpu_core_count": "8",
          "cpu_frequency_max_hz": "0",
          "cpu_l1d_cache_bytes": "65536",
          "cpu_l1i_cache_bytes": "131072",
          "cpu_l2_cache_bytes": "4194304",
          "cpu_l3_cache_bytes": "0",
          "cpu_model_name": "Apple M1",
          "cpu_thread_count": "8",
          "gpu_count": "0",
          "gpu_product_names": [],
          "kernel_name": "20.6.0",
          "memory_bytes": "17179869184",
          "name": "diana",
          "os_name": "macOS",
          "os_version": "11.5.2"
      },
      "stats": {
          "data": [
              "0.000001"
          ],
          "iqr": "0.000000",
          "iterations": 1,
          "max": "0.000001",
          "mean": "0.000001",
          "median": "0.000001",
          "min": "0.000001",
          "q1": "0.000001",
          "q3": "0.000001",
          "stdev": 0,
          "time_unit": "s",
          "times": [],
          "unit": "s"
      },
      "tags": {
          "name": "addition"
      }
  }
  ```

  Example Python execution:

  ```
  (conbench) $ python
  >>> import json
  >>> from conbench.tests.benchmark import _example_benchmarks
  >>> benchmark = _example_benchmarks.SimpleBenchmark()
  >>> [(result, output)] = benchmark.run(iterations=10)
  >>> output
  2
  >>> print(json.dumps(result, indent=2))
  {
    "run_id": "dfe3a816ca9e451a9da7d940a974cb95",
    "batch_id": "0e869934b391424a8199c485dfbbc066",
    "timestamp": "2021-11-09T22:11:25.262330+00:00",
    "stats": {
      "data": [
        "0.000002",
        "0.000001",
        "0.000000",
        "0.000001",
        "0.000001",
        "0.000001",
        "0.000001",
        "0.000000",
        "0.000001",
        "0.000001"
      ],
      "times": [],
      "unit": "s",
      "time_unit": "s",
      "iterations": 10,
      "mean": "0.000001",
      "median": "0.000001",
      "min": "0.000000",
      "max": "0.000002",
      "stdev": "0.000001",
      "q1": "0.000001",
      "q3": "0.000001",
      "iqr": "0.000000"
    },
    "machine_info": {
      "name": "diana",
      "os_name": "macOS",
      "os_version": "11.5.2",
      "architecture_name": "arm64",
      "kernel_name": "20.6.0",
      "memory_bytes": "17179869184",
      "cpu_model_name": "Apple M1",
      "cpu_core_count": "8",
      "cpu_thread_count": "8",
      "cpu_l1d_cache_bytes": "65536",
      "cpu_l1i_cache_bytes": "131072",
      "cpu_l2_cache_bytes": "4194304",
      "cpu_l3_cache_bytes": "0",
      "cpu_frequency_max_hz": "0",
      "gpu_count": "0",
      "gpu_product_names": []
    },
    "context": {
      "benchmark_language": "Python"
    },
    "info": {
      "benchmark_language_version": "Python 3.9.7"
    },
    "tags": {
      "name": "addition"
    },
    "github": {
      "commit": "61dec915b9dd230ca5029f5e586f8bd95c3e0c05",
      "repository": "https://github.com/conbench/conbench"
    }
  }
  ```

  By default, Conbench will try to publish your results to a Conbench server. If
  you don't have one running or are missing a `.conbench` credentials file, you'll
  see error messages like the following when you execute benchmarks.

  ```
  POST http://localhost:5000/api/login/ failed
  {
    "code": 400,
    "description": {
      "_errors": [
        "Invalid email or password."
      ]
    },
    "name": "Bad Request"
  }

  POST http://localhost:5000/api/benchmarks/ failed
  {
    "code": 401,
    "name": "Unauthorized"
  }
  ```

  To publish your results to a Conbench server, place a `.conbench` file in the
  same directory as your benchmarks. The `cat` command below shows the contents
  of an example `.conbench` config file.

  ```
  (conbench) $ cd ~/workspace/conbench/conbench/tests/benchmark/
  (conbench) $ cat .conbench
  url: http://localhost:5000
  email: you@example.com
  password: conbench
  ```

  If you don't yet have a Conbench server user account, you'll need to create one
  to publish results (registration key defaults to `conbench`).

  - http://localhost:5000/register/




  An "external benchmark" records results that were obtained from some other
  benchmarking tool (like executing an R benchmark from command line, parsing
  the resulting JSON, and recording those results).

  Implementation details: Note that the following benchmark sets
  `external = True`, and calls `self.conbench.record()` rather than
  `self.conbench.benchmark()` as the example above does.

  ```python
  import conbench.runner


  @conbench.runner.register_benchmark
  class ExternalBenchmark(conbench.runner.Benchmark):
      """Example benchmark that just records external results."""

      external = True
      name = "external"

      def run(self, **kwargs):
          
          result = {
              "data": [100, 200, 300],
              "unit": "i/s",
              "times": [0.100, 0.200, 0.300],
              "time_unit": "s",
          }

          context = {"benchmark_language": "C++"}
          yield self.conbench.record(
              result, self.name, context=context, options=kwargs, output=result
          )
  ```


  ```
  (conbench) $ cd ~/workspace/conbench/conbench/tests/benchmark/
  (conbench) $ conbench external --help

  Usage: conbench external [OPTIONS]

    Run external benchmark.

  Options:
    --show-result BOOLEAN  [default: true]
    --show-output BOOLEAN  [default: false]
    --run-id TEXT          Group executions together with a run id.
    --run-name TEXT        Free-text name of run (commit ABC, pull request 123,
                           etc).
    --run-reason TEXT      Low-cardinality reason for run (commit, pull request,
                           manual, etc).
    --help                 Show this message and exit.
  ```


  Note that the use of `--iterations=3` results in 3 runs of the benchmark, and
  the `mean`, `stdev`, etc calculated.


  ```
  (conbench) $ cd ~/workspace/conbench/conbench/tests/benchmark/
  (conbench) $ conbench external --iterations=3

  Benchmark result:
  {
      "run_id": "8058dde1491b49e5bd514646797c2a20",
      "batch_id": "8058dde1491b49e5bd514646797c2a20",
      "timestamp": "2021-06-21T22:16:54.786499+00:00",
      "context": {
          "benchmark_language": "C++"
      },
      "github": {
          "commit": "58fb35dc593dca82c917cf18c1c65c059b9fb12c",
          "repository": "https://github.com/conbench/conbench"
      },
      "info": {},
      "machine_info": {
          "architecture_name": "x86_64",
          "cpu_core_count": "2",
          "cpu_frequency_max_hz": "3500000000",
          "cpu_l1d_cache_bytes": "32768",
          "cpu_l1i_cache_bytes": "32768",
          "cpu_l2_cache_bytes": "262144",
          "cpu_l3_cache_bytes": "4194304",
          "cpu_model_name": "Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz",
          "cpu_thread_count": "4",
          "kernel_name": "20.5.0",
          "memory_bytes": "17179869184",
          "name": "machine-abc",
          "os_name": "macOS",
          "os_version": "10.16"
      },
      "stats": {
          "data": [
              "100.000000",
              "200.000000",
              "300.000000"
          ],
          "iqr": "100.000000",
          "iterations": 3,
          "max": "300.000000",
          "mean": "200.000000",
          "median": "200.000000",
          "min": "100.000000",
          "q1": "150.000000",
          "q3": "250.000000",
          "stdev": "100.000000",
          "time_unit": "s",
          "times": [
              "0.100000",
              "0.200000",
              "0.300000"
          ],
          "unit": "i/s"
      },
      "tags": {
          "name": "external"
      }
  }
  ```



  If your benchmark is executed on a machine cluster instead of one machine, you
  can capture cluster info in the following manner.
  Note that a benchmark will have a continuous history on a specific cluster as long as cluster's `name` and `info` do not change.
  There is also an `optional_info` field for information that should not impact the cluster's hash (and thus disrupt the distribution history), but should still be recorded.
  ```python
  import conbench.runner


  @conbench.runner.register_benchmark
  class SimpleBenchmarkWithClusterInfo(conbench.runner.Benchmark):
      name = "product"

      def run(self, **kwargs):
          cluster_info = {
              "name": "cluster 1",
              "info": {"gpu": 1},
              "optional_info": {"workers": 2},
          }
          yield self.conbench.benchmark(
              self._get_benchmark_function(),
              self.name,
              cluster_info=cluster_info,
              options=kwargs,
          )

      def _get_benchmark_function(self):
          return lambda: 1 * 2
  ```



  A "case benchmark" is a either a "simple benchmark" or an "external benchmark"
  executed under various predefined scenarios (cases).

  Implementation details: Note that the following benchmark declares the valid
  combinations in `valid_cases`, which reads like a CSV (the first row contains
  the cases names).


  ```python
  import conbench.runner


  @conbench.runner.register_benchmark
  class CasesBenchmark(conbench.runner.Benchmark):
      """Example benchmark with cases."""

      name = "matrix"
      valid_cases = (
          ("rows", "columns"),
          ("10", "10"),
          ("2", "10"),
          ("10", "2"),
      )

      def run(self, case=None, **kwargs):
          for case in self.get_cases(case, kwargs):
              rows, columns = case
              tags = {"rows": rows, "columns": columns}
              func = self._get_benchmark_function(rows, columns)
              benchmark, output = self.conbench.benchmark(
                  func,
                  self.name,
                  tags=tags,
                  options=kwargs,
              )
              yield benchmark, output

      def _get_benchmark_function(self, rows, columns):
          return lambda: int(rows) * [int(columns) * [0]]
  ```


  ```
  (conbench) $ cd ~/workspace/conbench/conbench/tests/benchmark/
  (conbench) $ conbench matrix --help

  Usage: conbench matrix [OPTIONS]

    Run matrix benchmark(s).

    For each benchmark option, the first option value is the default.

    Valid benchmark combinations:
    --rows=10 --columns=10
    --rows=2 --columns=10
    --rows=10 --columns=2

    To run all combinations:
    $ conbench matrix --all=true

  Options:
    --rows [10|2]
    --columns [10|2]
    --all BOOLEAN          [default: false]
    --iterations INTEGER   [default: 1]
    --drop-caches BOOLEAN  [default: false]
    --gc-collect BOOLEAN   [default: true]
    --gc-disable BOOLEAN   [default: true]
    --show-result BOOLEAN  [default: true]
    --show-output BOOLEAN  [default: false]
    --run-id TEXT          Group executions together with a run id.
    --run-name TEXT        Free-text name of run (commit ABC, pull request 123,
                           etc).
    --run-reason TEXT      Low-cardinality reason for run (commit, pull request,
                           manual, etc).
    --help                 Show this message and exit.
      """
  ```


  Note that the use of `--all=true` results in 3 benchmark results, one for each
  case (`10 x 10`, `2, x 10`, and `10, x 2`).


  ```
  (conbench) $ cd ~/workspace/conbench/conbench/tests/benchmark/
  (conbench) $ conbench matrix --all=true

  Benchmark result:
  {
      "batch_id": "13b87cc6d9a84f2188df279d8c513933",
      "run_id": "48acd853b8294df9a1f5457f192456f3",
      "timestamp": "2021-11-09T22:15:23.501923+00:00",
      "context": {
          "benchmark_language": "Python"
      },
      "github": {
          "commit": "61dec915b9dd230ca5029f5e586f8bd95c3e0c05",
          "repository": "https://github.com/conbench/conbench"
      },
      "info": {
          "benchmark_language_version": "Python 3.9.7"
      },
      "machine_info": {
          "architecture_name": "arm64",
          "cpu_core_count": "8",
          "cpu_frequency_max_hz": "0",
          "cpu_l1d_cache_bytes": "65536",
          "cpu_l1i_cache_bytes": "131072",
          "cpu_l2_cache_bytes": "4194304",
          "cpu_l3_cache_bytes": "0",
          "cpu_model_name": "Apple M1",
          "cpu_thread_count": "8",
          "gpu_count": "0",
          "gpu_product_names": [],
          "kernel_name": "20.6.0",
          "memory_bytes": "17179869184",
          "name": "diana",
          "os_name": "macOS",
          "os_version": "11.5.2"
      },
      "run_id": "48acd853b8294df9a1f5457f192456f3",
      "stats": {
          "data": [
              "0.000004"
          ],
          "iqr": "0.000000",
          "iterations": 1,
          "max": "0.000004",
          "mean": "0.000004",
          "median": "0.000004",
          "min": "0.000004",
          "q1": "0.000004",
          "q3": "0.000004",
          "stdev": 0,
          "time_unit": "s",
          "times": [],
          "unit": "s"
      },
      "tags": {
          "columns": "10",
          "name": "matrix",
          "rows": "10"
      },
      "timestamp": "2021-11-09T22:15:23.397819+00:00"
  }

  Benchmark result:
  {
      "batch_id": "13b87cc6d9a84f2188df279d8c513933",
      "context": {
          "benchmark_language": "Python"
      },
      "github": {
          "commit": "61dec915b9dd230ca5029f5e586f8bd95c3e0c05",
          "repository": "https://github.com/conbench/conbench"
      },
      "info": {
          "benchmark_language_version": "Python 3.9.7"
      },
      "machine_info": {
          "architecture_name": "arm64",
          "cpu_core_count": "8",
          "cpu_frequency_max_hz": "0",
          "cpu_l1d_cache_bytes": "65536",
          "cpu_l1i_cache_bytes": "131072",
          "cpu_l2_cache_bytes": "4194304",
          "cpu_l3_cache_bytes": "0",
          "cpu_model_name": "Apple M1",
          "cpu_thread_count": "8",
          "gpu_count": "0",
          "gpu_product_names": [],
          "kernel_name": "20.6.0",
          "memory_bytes": "17179869184",
          "name": "diana",
          "os_name": "macOS",
          "os_version": "11.5.2"
      },
      "stats": {
          "data": [
              "0.000004"
          ],
          "iqr": "0.000000",
          "iterations": 1,
          "max": "0.000004",
          "mean": "0.000004",
          "median": "0.000004",
          "min": "0.000004",
          "q1": "0.000004",
          "q3": "0.000004",
          "stdev": 0,
          "time_unit": "s",
          "times": [],
          "unit": "s"
      },
      "tags": {
          "columns": "10",
          "name": "matrix",
          "rows": "2"
      }
  }

  Benchmark result:
  {
      "batch_id": "13b87cc6d9a84f2188df279d8c513933",
      "run_id": "48acd853b8294df9a1f5457f192456f3",
      "timestamp": "2021-11-09T22:15:23.509211+00:00",
      "context": {
          "benchmark_language": "Python"
      },
      "github": {
          "commit": "61dec915b9dd230ca5029f5e586f8bd95c3e0c05",
          "repository": "https://github.com/conbench/conbench"
      },
      "info": {
          "benchmark_language_version": "Python 3.9.7"
      },
      "machine_info": {
          "architecture_name": "arm64",
          "cpu_core_count": "8",
          "cpu_frequency_max_hz": "0",
          "cpu_l1d_cache_bytes": "65536",
          "cpu_l1i_cache_bytes": "131072",
          "cpu_l2_cache_bytes": "4194304",
          "cpu_l3_cache_bytes": "0",
          "cpu_model_name": "Apple M1",
          "cpu_thread_count": "8",
          "gpu_count": "0",
          "gpu_product_names": [],
          "kernel_name": "20.6.0",
          "memory_bytes": "17179869184",
          "name": "diana",
          "os_name": "macOS",
          "os_version": "11.5.2"
      },
      "stats": {
          "data": [
              "0.000002"
          ],
          "iqr": "0.000000",
          "iterations": 1,
          "max": "0.000002",
          "mean": "0.000002",
          "median": "0.000002",
          "min": "0.000002",
          "q1": "0.000002",
          "q3": "0.000002",
          "stdev": 0,
          "time_unit": "s",
          "times": [],
          "unit": "s"
      },
      "tags": {
          "columns": "2",
          "name": "matrix",
          "rows": "10"
      }
  }
  ```



  Here are a few examples illustrating how to integrate R benchmarks with
  Conbench.

  The first one just times `1 + 1` in R, and the second one executes an R
  benchmark from a library of R benchmarks (in this case
  [arrowbench](https://github.com/ursacomputing/arrowbench)).

  If you find yourself wrapping a lot of R benchmarks in Python to integrate them
  with Conbench (to get uniform JSON benchmark results which you can persist and
  publish on a Conbench server), you'll probably want to extract much of the
  boilerplate out into a base class.


  ```python
  import conbench.runner


  @conbench.runner.register_benchmark
  class ExternalBenchmarkR(conbench.runner.Benchmark):
      """Example benchmark that records an R benchmark result."""

      external = True
      name = "external-r"

      def run(self, **kwargs):
          result, output = self._run_r_command()
          info, context = self.conbench.get_r_info_and_context()

          yield self.conbench.record(
              {"data": [result], "unit": "s"},
              self.name,
              info=info,
              context=context,
              options=kwargs,
              output=output,
          )

      def _run_r_command(self):
          output, _ = self.conbench.execute_r_command(self._get_r_command())
          result = float(output.split("\n")[-1].split("[1] ")[1])
          return result, output

      def _get_r_command(self):
          return (
              f"addition <- function() { 1 + 1 }; "
              "start_time <- Sys.time();"
              "addition(); "
              "end_time <- Sys.time(); "
              "result <- end_time - start_time; "
              "as.numeric(result); "
          )
  ```


  ```
  (conbench) $ cd ~/workspace/conbench/conbench/tests/benchmark/
  (conbench) $ conbench external-r --help

  Usage: conbench external-r [OPTIONS]

    Run external-r benchmark.

  Options:
    --show-result BOOLEAN  [default: true]
    --show-output BOOLEAN  [default: false]
    --run-id TEXT          Group executions together with a run id.
    --run-name TEXT        Free-text name of run (commit ABC, pull request 123,
                           etc).
    --run-reason TEXT      Low-cardinality reason for run (commit, pull request,
                           manual, etc).
    --help                 Show this message and exit.
  ```


  ```python
  import json

  import conbench.runner


  @conbench.runner.register_benchmark
  class ExternalBenchmarkOptionsR(conbench.runner.Benchmark):
      """Example benchmark that records an R benchmark result (with options)."""

      external = True
      name = "external-r-options"
      options = {
          "iterations": {"default": 1, "type": int},
          "drop_caches": {"type": bool, "default": "false"},
      }

      def run(self, **kwargs):
          data, iterations = [], kwargs.get("iterations", 1)
          info, context = self.conbench.get_r_info_and_context()

          for _ in range(iterations):
              if kwargs.get("drop_caches", False):
                  self.conbench.sync_and_drop_caches()
              result, output = self._run_r_command()
              data.append(result["result"][0]["real"])

          yield self.conbench.record(
              {"data": data, "unit": "s"},
              self.name,
              info=info,
              context=context,
              options=kwargs,
              output=output,
          )

      def _run_r_command(self):
          r_command = self._get_r_command()
          self.conbench.execute_r_command(r_command)
          with open("placebo.json") as json_file:
              data = json.load(json_file)
          return data, json.dumps(data, indent=2)

      def _get_r_command(self):
          return (
              "library(arrowbench); "
              "out <- run_one(arrowbench:::placebo); "
              "cat(jsonlite::toJSON(out), file='placebo.json'); "
          )
  ```

  ```
  (conbench) $ cd ~/workspace/conbench/conbench/tests/benchmark/
  (conbench) $ conbench external-r --help

  Usage: conbench external-r-options [OPTIONS]

    Run external-r-options benchmark.

  Options:
    --iterations INTEGER   [default: 1]
    --drop-caches BOOLEAN  [default: false]
    --show-result BOOLEAN  [default: true]
    --show-output BOOLEAN  [default: false]
    --run-id TEXT          Group executions together with a run id.
    --run-name TEXT        Free-text name of run (commit ABC, pull request 123,
                           etc).
    --run-reason TEXT      Low-cardinality reason for run (commit, pull request,
                           manual, etc).
    --help                 Show this message and exit.
  ```


pkgId: 32888f393d9cb028046ca9a9f184fc109e26d36ee2d4c76868d64645e4f7b7c6
name: python-concepts
arch: src
version: 0.9.2
epoch: 0
release: 1
summary: Formal Concept Analysis with Python
url: https://github.com/xflr6/concepts
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-concepts-0.9.2-1.src.rpm
checksum_type: sha256
description:
  |PyPI version| |License| |Supported Python| |Format|
  |Travis| |Codecov| |Readthedocs-stable| |Readthedocs-latest|
  Concepts is a simple Python implementation of **Formal Concept Analysis**
  (FCA_).
  FCA provides a mathematical model for describing a set of **objects** (e.g. King
  Arthur, Sir Robin, and the holy grail) with a set of **properties** (e.g. human,
  knight, king, and mysterious) which each of the objects either has or not. A
  table called *formal context* defines which objects have a given property and
  vice versa which properties a given object has.


pkgId: 071684e5eeff8101d5231e6b41e2620e169c36b9c2ce2a8dd9f78d5dd05b6732
name: python-concise-concepts
arch: src
version: 0.8.1
epoch: 0
release: 1
url: https://github.com/pandora-intelligence/concise-concepts
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-concise-concepts-0.8.1-1.src.rpm
checksum_type: sha256
summary:
  This repository contains an easy and intuitive approach to few-shot NER using most similar expansion over spaCy embeddings. Now with entity confidence scores!

description:
  When wanting to apply NER to concise concepts, it is really easy to come up with examples, but pretty difficult to train an entire pipeline. Concise Concepts uses few-shot NER based on word embedding similarity to get you going
  with easy! Now with entity scoring!


  [![Python package](https://github.com/Pandora-Intelligence/concise-concepts/actions/workflows/python-package.yml/badge.svg?branch=main)](https://github.com/Pandora-Intelligence/concise-concepts/actions/workflows/python-package.yml)
  [![Current Release Version](https://img.shields.io/github/release/pandora-intelligence/concise-concepts.svg?style=flat-square&logo=github)](https://github.com/pandora-intelligence/concise-concepts/releases)
  [![pypi Version](https://img.shields.io/pypi/v/concise-concepts.svg?style=flat-square&logo=pypi&logoColor=white)](https://pypi.org/project/concise-concepts/)
  [![PyPi downloads](https://static.pepy.tech/personalized-badge/concise-concepts?period=total&units=international_system&left_color=grey&right_color=orange&left_text=pip%20downloads)](https://pypi.org/project/concise-concepts/)
  [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg?style=flat-square)](https://github.com/ambv/black)



  This library defines matching patterns based on the most similar words found in each group, which are used to fill a [spaCy EntityRuler](https://spacy.io/api/entityruler). To better understand the rule definition, I recommend playing around with the [spaCy Rule-based Matcher Explorer](https://demos.explosion.ai/matcher).


  - [TechVizTheDataScienceGuy](https://www.youtube.com/c/TechVizTheDataScienceGuy) created a [nice tutorial](https://prakhar-mishra.medium.com/few-shot-named-entity-recognition-in-natural-language-processing-92d31f0d1143) on how to use it.

  - [I](https://www.linkedin.com/in/david-berenstein-1bab11105/) created a [tutorial](https://www.rubrix.ml/blog/concise-concepts-rubrix/) in collaboration with Rubrix.

  The section [Matching Pattern Rules](#matching-pattern-rules) expands on the construction, analysis and customization of these matching patterns.




  ```
  pip install concise-concepts
  ```



  Take a look at the [configuration section](#configuration) for more info.



  Note that, [custom embedding models](#custom-embedding-models) are passed via `model_path`.

  ```python
  import spacy
  from spacy import displacy

  import concise_concepts

  data = {
      "fruit": ["apple", "pear", "orange"],
      "vegetable": ["broccoli", "spinach", "tomato"],
      "meat": ['beef', 'pork', 'turkey', 'duck']
  }

  text = """
      Heat the oil in a large pan and add the Onion, celery and carrots.
      Then, cook over a medium–low heat for 10 minutes, or until softened.
      Add the courgette, garlic, red peppers and oregano and cook for 2–3 minutes.
      Later, add some oranges and chickens. """

  nlp = spacy.load("en_core_web_md", disable=["ner"])

  nlp.add_pipe(
      "concise_concepts",
      config={
          "data": data,
          "ent_score": True,  # Entity Scoring section
          "verbose": True,
          "exclude_pos": ["VERB", "AUX"],
          "exclude_dep": ["DOBJ", "PCOMP"],
          "include_compound_words": False,
          "json_path": "./fruitful_patterns.json",
          "topn": (100,500,300)
      },
  )
  doc = nlp(text)

  options = {
      "colors": {"fruit": "darkorange", "vegetable": "limegreen", "meat": "salmon"},
      "ents": ["fruit", "vegetable", "meat"],
  }

  ents = doc.ents
  for ent in ents:
      new_label = f"{ent.label_} ({ent._.ent_score:.0%})"
      options["colors"][new_label] = options["colors"].get(ent.label_.lower(), None)
      options["ents"].append(new_label)
      ent.label_ = new_label
  doc.ents = ents

  displacy.render(doc, style="ent", options=options)
  ```
  ![](https://raw.githubusercontent.com/Pandora-Intelligence/concise-concepts/master/img/example.png)



  This might be useful when iterating over few_shot training data when not wanting to reload larger models continuously.
  Note that, [custom embedding models](#custom-embedding-models) are passed via `model`.

  ```python
  import gensim
  import spacy

  from concise_concepts import Conceptualizer

  model = gensim.downloader.load("fasttext-wiki-news-subwords-300")
  nlp = spacy.load("en_core_web_sm")
  data = {
      "disease": ["cancer", "diabetes", "heart disease", "influenza", "pneumonia"],
      "symptom": ["headache", "fever", "cough", "nausea", "vomiting", "diarrhea"],
  }
  conceptualizer = Conceptualizer(nlp, data, model)
  conceptualizer.nlp("I have a headache and a fever.").ents

  data = {
      "disease": ["cancer", "diabetes"],
      "symptom": ["headache", "fever"],
  }
  conceptualizer = Conceptualizer(nlp, data, model)
  conceptualizer.nlp("I have a headache and a fever.").ents
  ```



  A general introduction about the usage of matching patterns in the [usage section](#usage).

  Even though the baseline parameters provide a decent result, the construction of these matching rules can be customized via the config passed to the spaCy pipeline.

   - `exclude_pos`: A list of POS tags to be excluded from the rule-based match.
   - `exclude_dep`: A list of dependencies to be excluded from the rule-based match.
   - `include_compound_words`:  If True, it will include compound words in the entity. For example, if the entity is "New York", it will also include "New York City" as an entity.
   - `case_sensitive`: Whether to match the case of the words in the text.



  To motivate actually looking at the data and support interpretability, the matching patterns that have been generated are stored as `./main_patterns.json`. This behavior can be changed by using the `json_path` variable via the config passed to the spaCy pipeline.




   - `fuzzy`: A boolean value that determines whether to use fuzzy matching

  ```python
  data = {
      "fruit": ["apple", "pear", "orange"],
      "vegetable": ["broccoli", "spinach", "tomato"],
      "meat": ["beef", "pork", "fish", "lamb"]
  }

  nlp.add_pipe("concise_concepts", config={"data": data, "fuzzy": True})
  ```



  - `topn`: Use a specific number of words to expand over.

  ```python
  data = {
      "fruit": ["apple", "pear", "orange"],
      "vegetable": ["broccoli", "spinach", "tomato"],
      "meat": ["beef", "pork", "fish", "lamb"]
  }

  topn = [50, 50, 150]

  assert len(topn) == len

  nlp.add_pipe("concise_concepts", config={"data": data, "topn": topn})
  ```



  - `ent_score`: Use embedding based word similarity to score entities against their groups

  ```python
  import spacy
  import concise_concepts

  data = {
      "ORG": ["Google", "Apple", "Amazon"],
      "GPE": ["Netherlands", "France", "China"],
  }

  text = """Sony was founded in Japan."""

  nlp = spacy.load("en_core_web_lg")
  nlp.add_pipe("concise_concepts", config={"data": data, "ent_score": True, "case_sensitive": True})
  doc = nlp(text)

  print([(ent.text, ent.label_, ent._.ent_score) for ent in doc.ents])



  ```



  - `model_path`: Use custom `sense2vec.Sense2Vec`, `gensim.Word2vec` `gensim.FastText`, or `gensim.KeyedVectors`, or a pretrained model from [gensim](https://radimrehurek.com/gensim/downloader.html) library or a custom model path. For using a `sense2vec.Sense2Vec` take a look [here](https://github.com/explosion/sense2vec#pretrained-vectors).
  - `model`: within [standalone usage](#standalone), it is possible to pass these models directly.

  ```python
  data = {
      "fruit": ["apple", "pear", "orange"],
      "vegetable": ["broccoli", "spinach", "tomato"],
      "meat": ["beef", "pork", "fish", "lamb"]
  }


  model_path = "glove-wiki-gigaword-300"

  nlp.add_pipe("concise_concepts", config={"data": data, "model_path": model_path})
  ````


pkgId: 6872e43ad46f458637f37492e4bccedc83f0f4a8b7334f022c40714dc852e780
name: python-concog-dreem-lib
arch: src
version: 0.1.1
epoch: 0
release: 1
summary: Library for LZ complexity, wSMI calculations, and PSD analysis with EEG data.
url: https://pypi.org/project/concog-dreem-lib/
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-concog-dreem-lib-0.1.1-1.src.rpm
checksum_type: sha256
description:
  A Python library for EEG data analysis, including Lempel-Ziv (LZ) complexity, weighted Symbolic Mutual Information (wSMI), Power Spectral Density (PSD) analysis, and Permutation Entropy calculations. This library is designed to work with EEG data and integrates MATLAB functions via the MATLAB Engine API for Python.



  - [Features](#features)
  - [Installation](#installation)
  - [Dependencies](#dependencies)
    - [MATLAB and MATLAB Engine API](#matlab-and-matlab-engine-api)
  - [Usage](#usage)
    - [LZ Complexity](#lz-complexity)
    - [Weighted Symbolic Mutual Information (wSMI)](#weighted-symbolic-mutual-information-wsmi)
    - [Power Spectral Density (PSD) Analysis](#power-spectral-density-psd-analysis)
    - [Permutation Entropy](#permutation-entropy)
  - [Contributing](#contributing)
  - [License](#license)
  - [Contact](#contact)



  - **LZ Complexity**: Compute the Lempel-Ziv complexity of EEG signals.
  - **wSMI Calculations**: Compute weighted Symbolic Mutual Information from EEG data.
  - **PSD Analysis**: Perform Power Spectral Density analysis with options for FOOOF fitting.
  - **Permutation Entropy**: Calculate permutation entropy measures for time-series data.



  Install the package using pip:

  ```bash
  pip install concog_dreem_lib


pkgId: 71d1c0bcf53152c76c61a653b6159283b337c937ef5cbd0bbea8be985bc244ed
name: python-concord232
arch: src
version: 0.15.1
epoch: 0
release: 1
summary: GE Concord 4 RS232 Serial Interface Library and Server
url: http://github.com/JasonCarter80/concord232
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-concord232-0.15.1-1.src.rpm
checksum_type: sha256
description:
  This is a tool to let you interact with your GE Concord 4 alarm panel via
  the RS232 Automation module.
  The goal of this project was to utilize my GE Concord 4 alarm panel with [Home Assistant](https://home-assistant.io/)
  Following the framework of [kk7ds](https://github.com/kk7ds/pynx584) to integrate the nx584 into Home Assistant, and [douglasdecouto](https://github.com/douglasdecouto/py-concord)'s work into building the base communication class as part of their integration into the Indigo platform, we now have a working Interlogix/GE Concord 4 Automation Module interface
  To install::
  ```
  sudo pip3 install concord232
  ```
  The server must be run on a machine with connectivity to the panel, to get started, you must only supply the serial port.  In this case I use a USB to Serial adapter
  ```
  concord232_server --serial /dev/ttyUSB0
  ```
  Once that is running, you should be able to do something like this::
  ```
   $ concord232_client summary
   +------+-----------------+--------+--------+
   | Zone |       Name      | Bypass | Status |
   +------+-----------------+--------+--------+
   |  1   |    FRONT DOOR   |   -    | False  |
   |  2   |   GARAGE DOOR   |   -    | False  |
   |  3   |     SLIDING     |   -    | False  |
   |  4   | MOTION DETECTOR |   -    | False  |
   +------+-----------------+--------+--------+
  ```

  Arm to stay (level 2)
  ```
  $ concord232_client arm-stay
  ```
  Arm to away (level 3)
  ```
  $ concord232_client arm-away
  ```
  Disarm
  ```
  $ concord232_client disarm --master 1234
  ```

  Both stay (level 2) and away (level 3) alarms can take one of two
  options: silent arming, or instant arming.  Silent arming will not
  beep while the alarm is setting.  Instant arming has no delay.
  Clearly, this should only be used with away arming if you are already
  outside.
  Examples:
  Arm to stay with no delay
  ```
  $ concord232_client arm-stay-instant
  ```
  Arm to away without beeps
  ```
  $ concord232_client arm-away-silent
  ```

  Home Assistant will automatically download and install the pip3 library, but it only utilizes the Client to connect to the server.  I used the instructions [found here](http://www.raspberrypi-spy.co.uk/2015/10/how-to-autorun-a-python-script-on-boot-using-systemd/) for setting up the server to run automatically at boot time.


pkgId: 053b43c61f34bb75b7ec2c9843f6ce7428e24bcc99dbfd8b0cd1b7322a1ca021
name: python-concordancer
arch: src
version: 0.1.16
epoch: 0
release: 1
summary: Extract concordance lines from corpus with CQL
url: http://github.com/liao961120/concordancer
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-concordancer-0.1.16-1.src.rpm
checksum_type: sha256
description:
  買/VC 了/Di                 覺得/VK 材質/Na  很/Dfa 對/VH      覺得/VK        材質/Na
  “/PARENTHESISCATEGORY 不/D  戴/VC 錶/Na      世代/Na ”/VC      戴/VC          錶/Na
  聯名鞋/Na 趁著/P            過年/VA 期間/Na  穿出去/VB 四處/D  過年/VA        期間/Na
  走/VA  /WHITESPACE          燒/VC 錢/Na      啊/T ～/FW        燒/VC          錢/Na
  正/VH 韓/Nc                 賣/VD 家/Nc      裡面/Ncd 很/Dfa   賣/VD          家/Nc
  ```

  CQL search is supported through [cqls](https://github.com/liao961120/cqls), which implements a (quite useful) subset of CQL:
  - token: `[]`, `"我"`, `[word="我"]`, `[word!="我" & pos="N.*"]`
  - token-level quantifier: `+`, `*`, `?`, `{n,m}`
  - grouping: `("a" "b"? "c"){1,2}`
  - label: `lab1:[word="我" & pos="N.*"] lab2:("a" "b")`


pkgId: 14c64e0d54c65b90b73a568c2c908e23f5e1e14b41a5fb82e5b0efa06f594e70
name: python-concrete
arch: src
version: 4.18.2
epoch: 0
release: 1
summary: Python modules and scripts for working with Concrete
url: https://github.com/hltcoe/concrete-python
rpm_license: BSD License
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-concrete-4.18.2-1.src.rpm
checksum_type: sha256
description:
  Concrete-python is the Python interface to Concrete_, a
  natural language processing data format and set of service protocols
  that work across different operating systems and programming languages
  via `Apache Thrift`_.  Concrete-python contains generated Python
  classes, utility classes and functions, and scripts.  It does not contain the
  Thrift schema for Concrete, which can be found in the
  `Concrete GitHub repository`_.
  This document provides a quick tutorial of concrete-python installation and
  usage.  For more information, including an API reference and development
  information, please see the `online documentation`_.


pkgId: 7e75f9536e5bd21c89283b8c98240f9cd8690e592fd6c94e04ff2b27beb0ef48
name: python-concur
arch: src
version: 0.11.0
epoch: 0
release: 1
summary: Concur UI Framework for Python
url: https://potocpav.github.io/python-concur-docs/homepage.html
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-concur-0.11.0-1.src.rpm
checksum_type: sha256
description:
  <h1 align="center">
    Python Concur
  </h1>

  <p align="center">
     <img src="https://raw.githubusercontent.com/ajnsit/purescript-concur/master/docs/logo.png" height="100">
  </p>

  [![Build Status](https://travis-ci.com/potocpav/python-concur.svg?branch=master)](https://travis-ci.com/potocpav/python-concur)
  [![PyPI version](https://badge.fury.io/py/concur.svg)](https://pypi.org/project/concur)
  [![Python version](https://img.shields.io/pypi/pyversions/concur.svg)](https://pypi.org/project/concur)

  * [**API Reference**](https://potocpav.github.io/python-concur-docs/master/)
  * [**Documentation**](https://github.com/potocpav/python-concur-docs/)
  * [Homepage](https://potocpav.github.io/python-concur-docs/homepage.html)
  * [PyPI](https://pypi.org/project/concur/)
  * [Changelog](https://github.com/potocpav/python-concur/tree/master/CHANGELOG.md)

  Concur is a Python GUI framework based on synchronous generators.

  It is a port of [Concur for Purescript](https://github.com/ajnsit/purescript-concur), implemented on top of the [Dear ImGui](https://github.com/ocornut/imgui) C++ UI library.

  For introduction to core concepts, see the [documentation](file:///home/pavel/build/python-concur/docs/index.html#introduction). A more comprehensive introduction can be found in the [Documentation for the Haskell/Purescript versions](https://github.com/ajnsit/concur-documentation/blob/master/README.md). This obviously uses Haskell/Purescript syntax and semantics, but many of the concepts will apply to the Python version.

  Being an abstraction over ImGui, Concur is best used for debugging, prototyping and data analysis, rather than user-facing applications. ImGui functions can be used directly for any functionality that is not wrapped by Concur. See the [PyImGui docs](https://pyimgui.readthedocs.io/en/latest/) for additional widgets, or [ImGui itself](https://github.com/ocornut/imgui) if you need to poke really deep.



  The only dependencies are a C++ compiler, [GLFW](https://github.com/glfw/glfw) and Python >= 3.6. GLFW is included in major Linux distros, and there are [pre-compiled binaries](https://www.glfw.org/download.html) for Windows and OS X.

  ```sh
  sudo apt install libglfw3-dev # Ubuntu
  sudo pacman -S glfw           # Arch
  ```

  Concur can be installed using pip:

  ```sh
  pip install concur
  ```

  This command should produce a very simple GUI app:

  ```sh
  python -c 'import concur as c; c.main(c.button("Close"))'
  ```

  See the [introductory tutorial](https://github.com/potocpav/python-concur-docs/blob/master/Introduction.md) to get started, or use any of the [examples](https://github.com/potocpav/python-concur/tree/master/examples) as a starting point for your app.



  For Concur development, clone the repo and install it using pip:

  ```sh
  git clone https://github.com/potocpav/python-concur.git
  cd python-concur
  pip install -e.

  examples/all.py # Run the examples to verify installation
  ```

  To run the tests, you need PyTest and PyTest-benchmark:

  ```sh
  pip install pytest pytest-benchmark

  pytest # Run the tests


  SLOW_TEST=1 pytest
  ```

  To build documentation, install [pdoc3](https://pdoc3.github.io/pdoc/) (`pip install pdoc3`), clone the submodules (`git submodule update --init`), and run the script `./mkdocs.sh`.


  <p align="center">
  <img src="https://raw.githubusercontent.com/potocpav/python-concur-docs/master/screenshot.png">
  </p>


pkgId: ec30ba553066e466704480caf6a284439f1936f45d8f64d01f394cdf517f070a
name: python-concurrent-log-handler
arch: src
version: 0.9.25
epoch: 0
release: 1
summary: RotatingFileHandler replacement with concurrency, gzip and Windows support
url: https://github.com/Preston-Landers/concurrent-log-handler
rpm_license: Apache Software License
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-concurrent-log-handler-0.9.25-1.src.rpm
checksum_type: sha256
description:
  This package provides an additional log handler for Python's standard logging
  package (PEP 282). This handler will write log events to a log file which is
  rotated when the log file reaches a certain size. Multiple processes can
  safely write to the same log file concurrently. Rotated logs can be gzipped
  if desired. Both Windows and POSIX systems are supported. An optional threaded
  queue logging handler is provided to perform logging in the background.

  This is a fork of Lowell Alleman's ConcurrentLogHandler 0.9.1 which fixes
  a hanging/deadlocking problem. [See this](https://bugs.launchpad.net/python-concurrent-log-handler/+bug/1265150).

  Summary of other changes:

  * New: requires Python 3.6 or higher.
    * If you require support for Python 2.7, use version
      [0.9.22](https://github.com/Preston-Landers/concurrent-log-handler/releases/tag/0.9.22).
  * Renamed package to `concurrent_log_handler` (abbreviated CLH in this file.)
  * Provide `use_gzip` option to compress rotated logs
  * Support for Windows
  * Uses file locking to ensure exclusive write access
    Note: file locking is advisory, not a hard lock against external processes
  * More secure generation of random numbers for temporary filenames
  * Change the name of the lockfile to have .__ in front of it.
  * Provide a class for time-based rotation: [ConcurrentTimedRotatingFileHandler](#time-based-rotation-settings)
  * Provide an optional QueueListener / QueueHandler implementation for
    handling log events in a background thread.
  * Allow setting owner and mode permissions of rollover file on Unix
  * Depends on `portalocker` package, which (on Windows only) depends on `PyWin32`



  * [concurrent-log-handler on Github](https://github.com/Preston-Landers/concurrent-log-handler)
  * [concurrent-log-handler on the Python Package Index (PyPI)](https://pypi.org/project/concurrent-log-handler/)



  The main use case this is designed to support is when you have a Python application that runs in
  multiple processes, potentially on multiple hosts connected with a shared network drive, and you want
  to write all log events to a central log file and have those files rotated based on size and/or time,
  e.g. daily or hourly.

  However, this is not the only way to achieve shared logging from multiple processes. You can also
  centralize logging by using cloud logging services like Azure Log Monitor, Logstash, etc. Or you can
  implement your own remote logging server as shown here:

  [Logging cookbook: network](https://docs.python.org/3/howto/logging-cookbook.html#sending-and-receiving-logging-events-across-a-network)

  Concurrent-Log-Handler includes a QueueHandler and QueueListener implementation that can be used to
  perform logging in the background asynchronously, so the thread or process making the log statement doesn't have
  to wait for its completion. See [this section](#simple-example). Using that example code, each process still
  locks and writes the file separately, so there is no centralized writer. You could also write code to use
  QueueHandler and QueueListener to queue up log events within each process to be sent to a central server,
  instead of CLH's model where each process locks and writes to the log file.



  The main `ConcurrentRotatingFileHandler` class supports size-based rotation only.
  In addition, a `ConcurrentTimedRotatingFileHandler` class is provided that supports both time-based
  and size-based rotation. By default, it does hourly time-based rotation and no size rotation.
  See [this section](#time-based-rotation-settings) for more details.





  You can download and install the package with `pip` using the following command:

      pip install concurrent-log-handler

  This will also install the portalocker module, which on Windows in turn depends on pywin32.

  If installing from source, use the following command:

      python setup.py install



  If you plan to modify the code, you should follow this procedure:

  * Clone the repository
  * Create a virtual environment (`venv`) and activate it.
  * Install the package in editable mode with the [dev] option: `pip install -e .[dev]`

  * Run the tests:  `tox` or run `pytest` directly.

    Or manually run a single pass of the stress test with specific options:

  ```shell
  python tests/stresstest.py --help
  python tests/stresstest.py --gzip --num-processes 12 --log-calls=5000
  ```

  * To build a Python "wheel" for distribution, use the following:

  ```shell
  python setup.py clean --all build sdist bdist_wheel


  pip install twine
  twine upload dist/concurrent-log-handler-0.9.23.tar.gz dist/concurrent_log_handler-0.9.23-py3-none-any.whl
  ```



  Concurrent Log Handler (CLH) is designed to allow multiple processes to write to the same
  logfile in a concurrent manner. It is important that each process involved MUST follow
  these requirements:

  * You can't serialize a handler instance and reuse it in another process. This means you cannot, for
    example, pass a CLH handler instance from parent process to child process using
    the `multiprocessing` package in spawn mode (or similar techniques that use serialized objects).
    Each child process must initialize its own CLH instance.

  * When using the `multiprocessing` module in "spawn" (non-fork) mode, each child process must create
    its OWN instance of the handler (`ConcurrentRotatingFileHandler`). The child target function
    should call code that initializes a new CLH instance.

    * This requirement does not apply to threads within a given process. Different threads within a
      process can use the same CLH instance. Thread locking is handled automatically.

    * This also does not apply to `fork()` based child processes such as gunicorn --preload.
      Child processes of a fork() call should be able to inherit the CLH object instance.

    * This limitation exists because the CLH object can't be serialized, passed over a network or
      pipe, and reconstituted at the other end.

  * It is important that every process or thread writing to a given logfile must all use the same
    settings, especially related to file rotation. Also do not attempt to mix different handler
    classes writing to the same file, e.g. do not also use a `RotatingFileHandler` on the same file.

  * Special attention may need to be paid when the log file being written to resides on a network
    shared drive or a cloud synced folder (Dropbox, Google Drive, etc.). Whether the multiprocess
    advisory lock technique (via portalocker) works in these folders may depend on the details of
    your configuration.

    Note that a `lock_file_directory` setting (kwarg) now exists (as of v0.9.21) which lets you
    place the lockfile at a different location from the main logfile. This might solve problems
    related to trying to lock files in network shares or cloud folders (Dropbox, Google Drive, etc.)
    However, if multiple hosts are writing to the same shared logfile, they must also have access
    to the same lock file.

    Alternatively, you may be able to set your cloud sync software to ignore all `.lock` files.

  * A separate handler instance is needed for each individual log file. For instance, if your app
    writes to two different log files you will need to set up two CLH instances per process.



  Here is a simple direct usage example:

  ```python
  from logging import getLogger, INFO
  from concurrent_log_handler import ConcurrentRotatingFileHandler
  import os

  log = getLogger(__name__)

  logfile = os.path.abspath("mylogfile.log")

  rotateHandler = ConcurrentRotatingFileHandler(logfile, "a", 512 * 1024, 5)
  log.addHandler(rotateHandler)
  log.setLevel(INFO)

  log.info("Here is a very exciting log message, just for you")
  ```

  See also the file [src/example.py](src/example.py) for a configuration and usage example.
  This shows both the standard non-threaded non-async usage, and the use of the `asyncio`
  background logging feature. Under that option, when your program makes a logging statement,
  it is added to a background queue and may not be written immediately and synchronously. This
  queue can span multiple processes using `multiprocessing` or `concurrent.futures`, and spanning
  multiple hosts works due to the use of file locking on the log file. Note that with this async
  logging feature, currently there is no way for the caller to know when the logging statement
  completed (no "Promise" or "Future" object is returned).

  [QueueHandler](https://docs.python.org/3/library/logging.handlers.html#queuehandler)



  To use this module from a logging config file, use a handler entry like this:

  ```ini
  [handler_hand01]
  class = handlers.ConcurrentRotatingFileHandler
  level = NOTSET
  formatter = form01
  args = ("rotating.log", "a")
  kwargs = {'backupCount': 5, 'maxBytes': 1048576, 'use_gzip': True}
  ```

  That sets the files to be rotated at about 10 MB, and to keep the last 5 rotations.
  It also turns on gzip compression for rotated files.

  Please note that Python 3.7 and higher accepts keyword arguments (kwargs) in a logging
  config file, but earlier versions of Python only accept positional args.

  Note: you must have an `import concurrent_log_handler` before you call fileConfig(). For
  more information see Python docs on [log file formats](https://docs.python.org/3/library/logging.config.html#logging-config-fileformat)



  The size-based rotation limit (`maxBytes`) is not strict. The files may become slightly
  larger than `maxBytes`. How much larger depends on the size of the log message being
  written when the rollover occurs.

  By contrast, the base `RotatingLogHandler` class tries to ensure that the log file is
  always kept under `maxBytes` taking into account the size of the current log message being
  written. This limitation may be changed in the future.



  For best performance, avoid setting the `backupCount` (number of rollover files to keep) too
  high. What counts as "too high" is situational, but a good rule of thumb might be to keep
  around a maximum of 20 rollover files. If necessary, increase the `maxBytes` so that each
  file can hold more. Too many rollover files can slow down the rollover process due to the
  mass file renames, and the rollover occurs while the file lock is held for the main logfile.

  How big to allow each file to grow (`maxBytes`) is up to your needs, but generally a value of
  10 MB (1048576) to 100 MB (1048576) is reasonable.

  Gzip compression is turned off by default. If enabled it will reduce the storage needed for rotated
  files, at the cost of some minimal CPU overhead. Use of the background logging queue shown below
  can help offload the cost of logging to another thread.

  Sometimes you may need to place the lock file at a different location from the main log
  file. A `lock_file_directory` setting (kwarg) now exists (as of v0.9.21) which lets you
  place the lockfile at a different location. This can often solve problems related to trying
  to lock files in cloud folders (Dropbox, Google Drive, OneDrive, etc.) However, in
  order for this to work, each process writing to the log must have access to the same
  lock file location, even if they are running on different hosts.

  You can set the `namer` attribute of the handler to customize the naming of the rotated files,
  in line with the `BaseRotatingHandler` class. See the Python docs for
  [more details](https://docs.python.org/3.11/library/logging.handlers.html#logging.handlers.BaseRotatingHandler.namer).



  By default, the logfile will have line endings appropriate to the platform. On Windows
  the line endings will be CRLF ('\r\n') and on Unix/Mac they will be LF ('\n').

  It is possible to force another line ending format by using the newline and terminator
  arguments.

  The following would force Windows-style CRLF line endings on Unix:

      kwargs={'newline': '', 'terminator': '\r\n'}

  The following would force Unix-style LF line endings on Windows:

      kwargs={'newline': '', 'terminator': '\n'}



  An alternative class `ConcurrentTimedRotatingFileHandler` is also provided which supports
  time-based rotation, defaulting to hourly. Like the main class, it uses advisory file
  locking to both ensure that only one process/thread is writing to the log file at a time,
  and to coordinate the rollover time between processes.

  By default, it has `maxBytes` set to 0, which means that it will not rotate based on file size,
  but it is possible to set `maxBytes` to a value to limit the size of each file in addition
  to the time-based rotation. When files are rotated based on size, they may have an additional
  numeric suffix like `.1` added to the filename. Note that like with the main CLH class,
  the file size limits are *not* strictly adhered to.

  All the same settings are available for this class as for the main class, including
  `maxBytes`, `use_gzip`, `lock_file_directory`, `newline`, and `terminator`. However,
  the ordering of the arguments is different, so it's recommended to use keyword arguments
  when using or configuring this class. The arguments shared with `TimedRotatingFileHandler`
  are in the same order as the base class, and the extra CLH arguments come after that,
  although not in the exact same order due to some overlap.

  For configuration, see the [configuration section](#configuration) above, but substitute in
  `class=handlers.ConcurrentTimedRotatingFileHandler` and other appropriate settings
  like `when` and `interval`. See the Python docs for `TimedRotatingFileHandler` for
  [more details](https://docs.python.org/3.11/library/logging.handlers.html#logging.handlers.TimedRotatingFileHandler).



  To use the background logging queue, you must call this code at some point in your
  app after it sets up logging configuration. Please read the doc string in the
  file `concurrent_log_handler/queue.py` for more details. This requires Python 3.
  See also [src/example.py](src/example.py).

  ```python
  from concurrent_log_handler.queue import setup_logging_queues


  setup_logging_queues()
  ```

  This module is designed to function well in a multi-threaded or multi-processes
  concurrent environment. However, all writers to a given log file should be using
  the same class and the *same settings* at the same time, otherwise unexpected
  behavior may result during file rotation.

  This may mean that if you change the logging settings at any point you may need to
  restart your app service so that all processes are using the same settings at the same time.



  The `ConcurrentRotatingFileHandler` class is a drop-in replacement for
  Python's standard log handler `RotatingFileHandler`. This module uses file
  locking so that multiple processes can concurrently log to a single file without
  dropping or clobbering log events. This module provides a file rotation scheme
  like with `RotatingFileHandler`. Extra care is taken to ensure that logs
  can be safely rotated before the rotation process is started. (This module works
  around the file rename issue with `RotatingFileHandler` on Windows, where a
  rotation failure means that all subsequent log events are dropped).

  This module attempts to preserve log records at all cost. This means that log
  files will grow larger than the specified maximum (rotation) size. So if disk
  space is tight, you may want to stick with `RotatingFileHandler`, which will
  strictly adhere to the maximum file size.

  Important:

  If you have multiple instances of a script (or multiple scripts) all running at
  the same time and writing to the same log file, then *all* of the scripts should
  be using `ConcurrentRotatingFileHandler`. You should not attempt to mix
  and match `RotatingFileHandler` and `ConcurrentRotatingFileHandler`.
  The file locking is advisory only - it is respected by other Concurrent Log Handler
  instances, but does not protect against outside processes (or different Python logging
  file handlers) from writing to a log file in use.



  See [CHANGELOG.md](CHANGELOG.md)



  The original version was written by Lowell Alleman.

  Other contributors are listed in [CONTRIBUTORS.md](CONTRIBUTORS.md).



  See the [LICENSE file](LICENSE)


pkgId: 7a20a800c263f9e800d1b58798d4fe4ce9b13d6bcdfa4d425a59d51428b7368e
name: python-concurrent-openai
arch: src
version: 0.2.1
epoch: 0
release: 1
url: https://github.com/NoPaperMenu/concurrent-openai
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-concurrent-openai-0.2.1-1.src.rpm
checksum_type: sha256
summary:
  Python library for asynchronous interactions with the OpenAI API, enabling concurrent request handling. It simplifies building scalable, AI-powered applications by offering efficient, rate-limited access to OpenAI services. Perfect for developers seeking to integrate OpenAI's capabilities with minimal overhead.

description:
  The Concurrent OpenAI Manager is a pure Python library meticulously designed for developers seeking an optimal integration with OpenAI's APIs. This library is engineered to handle API requests with efficiency, ensuring compliance with rate limits and managing system resources effectively, all while providing transparent cost estimations for OpenAI services.



  Central to the library is a carefully crafted rate limiter, capable of managing the number of requests and tokens per minute. This ensures your application stays within OpenAI's usage policies, avoiding rate limit violations and potential service disruptions.


  The throttling mechanism is designed to prevent sudden surges of requests, spreading them evenly over time. This ensures a steady and predictable load on OpenAI's endpoints, contributing to a responsible utilization of API resources and avoiding the 429 errors that might occur if we simply do all the requests at once.


  To manage local system resources or limit parallelism, the library incorporates a semaphore mechanism. This allows developers to specify the maximum number of concurrent operations, ensuring balanced resource utilization and a responsive application performance. Useful when you want tot manage local resources (such as database connections or memory usage) or wish to limit parallelism to ensure a responsive user experience. By fine-tuning the semaphore value, you have control on the amount of coroutines that are on the Event Loop.


  A notable feature of the Concurrent OpenAI Manager is its built-in cost estimation. This functionality provides users with detailed insights into the cost implications of their API requests, including a breakdown of prompt and completion tokens used. Such transparency empowers users to manage their budget effectively and optimize their use of OpenAI's APIs.



  Integrating the Concurrent OpenAI Manager into your project is straightforward:

  ```bash
  $ pip install concurrent-openai
  ```


  1. Create a `.env` file in your project directory.
  2. Add an env variable named `OPENAI_API_KEY`.
  3. Test it out:
  ```python
  from concurrent_openai import process_completion_requests

  results = await process_completion_requests(
      prompts=[{"role": "user", "content": "Knock, knock!"}],
      model="gpt-4-0613",
      temperature=0.7,
      max_tokens=150,
      max_concurrent_requests=5,
      token_safety_margin=10,
  )

  for result in results:
      if result:
          print(result)
      else:
          print("Error processing request.")
  ```


pkgId: 8dbc051edc52a4dffe6d91c3ffa7813231f90051fbff97cefa3b9d76fc7de324
name: python-concurrent-progressbar
arch: src
version: 1.1.4
epoch: 0
release: 1
url: https://pypi.org/project/concurrent-progressbar/
rpm_license: None
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-concurrent-progressbar-1.1.4-1.src.rpm
checksum_type: sha256
summary:
  This package is used to track the processes executed in the program. It will show individual progress bar for each task alloted using this package. It runs each task in separate process.

description:
  - [About](#about)
  - [Getting Started](#getting_started)
  - [Usage](#usage)



  This package is used to track the processes executed in the program. It will show individual progress bar for each task alloted using this package. It runs each task in separate process.



  These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.



  pip install concurrent-progressbar


  ```
  import os
  from concurrent_progressbar.concurrent import Multithreading, MultiProcessing


  def target_1(i):
      pass

  def target_2(j):
      pass


  pool = Multithreading(
                      num_workers=os.cpu_count(),
                      target=[target_1, target_2],
                      args=[[(i,) for i in range(1000)], [(j,) for j in range(100)]]
      )

  pool.run()



  pool = Multiprocessing(
                      num_workers=os.cpu_count(),
                      target=[target_1, target_2],
                      args=[[(i,) for i in range(1000)], [(j,) for j in range(100)]]
      )

  pool.run()
  ```




  ```
  pool = Multiprocessing(
                      num_workers=os.cpu_count(),
                      target=[target_1, target_2],
                      args=[[(i,) for i in range(1000)], [(j,) for j in range(100)]],
                      task_desc=["my-task-1", "my-task-2"],
                      tasks_at_a_time=1
      )
   
   

  pool.run()

  ```


pkgId: 0398a95215f0442904e877fffc894e6338bbbc33d64242a4cdd1dc7cec1a30a7
name: python-concurrent_tasks
arch: src
version: 1.7.2
epoch: 0
release: 1
summary: Tools to run asyncio tasks concurrently.
url: https://github.com/gpajot/asyncio-concurrent-tasks
rpm_license: MIT
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-concurrent_tasks-1.7.2-1.src.rpm
checksum_type: sha256
description:
  [![tests](https://github.com/gpajot/asyncio-concurrent-tasks/actions/workflows/test.yml/badge.svg?branch=main&event=push)](https://github.com/gpajot/asyncio-concurrent-tasks/actions/workflows/test.yml?query=branch%3Amain+event%3Apush)
  [![version](https://img.shields.io/pypi/v/concurrent_tasks?label=stable)](https://pypi.org/project/concurrent_tasks/)
  [![python](https://img.shields.io/pypi/pyversions/concurrent_tasks)](https://pypi.org/project/concurrent_tasks/)

  Tooling to run asyncio tasks.
  - [Background task](#background-task)
  - [Periodic task](#periodic-task)
  - [Thread safe task pool](#thread-safe-task-pool)
  - [Threaded task pool](#threaded-task-pool)
  - [Restartable task](#restartable-task)
  - [Loop exception handler](#loop-exception-handler)



  Task that is running in the background until cancelled.
  Can be used as a context manager.

  Example usage:
  ```python
  import asyncio
  from typing import Callable, Awaitable
  from concurrent_tasks import BackgroundTask


  class HeartBeat(BackgroundTask):
      def __init__(self, interval: float, func: Callable[[], Awaitable]):
          super().__init__(self._run, interval, func)

      async def _run(self, interval: float, func: Callable[[], Awaitable]) -> None:
          while True:
              await func()
              await asyncio.sleep(interval)
  ```



  Task that is running periodically in the background until cancelled.
  Can be used as a context manager.
  There is no guarantee that the time between calls is strictly the interval if the function takes more time than the interval to execute.

  Example usage:
  ```python
  from typing import Callable, Awaitable
  from concurrent_tasks import PeriodicTask


  class HeartBeat(PeriodicTask):
      def __init__(self, interval: float, func: Callable[[], Awaitable]):
          super().__init__(interval, func)
  ```



  The goal is to be able to safely run tasks from other threads.

  Parameters:
  - `size` can be a positive integer to limit the number of tasks concurrently running.
  - `timeout` can be set to define a maximum running time for each time after which it will be cancelled.
  Note: this excludes time spent waiting to be started (time spent in the buffer).

  Example usage:
  ```python
  from concurrent_tasks import ThreadSafeTaskPool


  async def func():
      ...


  async with ThreadSafeTaskPool() as pool:
      
      result = await pool.run(func())
      
      future = pool.create_task(func())
  ```



  Run async tasks in a dedicated thread. It will have its own event loop.
  Under the hook, `ThreadSafeTaskPool` is used.

  Parameters:
  - `name` will be used as the thread's name.
  - `size` and `timeout` see `ThreadSafeTaskPool`.
  - `context_manager` can be optional context managers that will be entered when the loop has started
  and exited before the loop is stopped.

  > 💡 All tasks will be completed when the pool is stopped.

  > 💡 Blocking and async version are the same, prefer the async version if client code is async.



  > ⚠️ Asyncio primitives need to be instantiated with the proper event loop.

  To achieve that, use a context manager wrapping instantiation of objects:
  ```python
  from functools import partial

  from concurrent_tasks import ThreadedPoolContextManagerWrapper, AsyncThreadedTaskPool

  pool = AsyncThreadedTaskPool(context_manager=ThreadedPoolContextManagerWrapper(partial(MyObj, ...)))
  ```



  This can be used to run async functions in a dedicated event loop, while keeping it running to handle background tasks

  Example usage:
  ```python
  from concurrent_tasks import BlockingThreadedTaskPool


  async def func():
      ...


  with BlockingThreadedTaskPool() as pool:
      
      result = pool.run(func())
      
      future = pool.create_task(func())
  ```



  Threads can be useful in cooperation with asyncio to let the OS guarantee fair resource distribution between threads.
  This is especially useful in case you cannot know if called code will properly cooperate with the event loop.

  Example usage:
  ```python
  from concurrent_tasks import AsyncThreadedTaskPool


  async def func():
      ...


  async with AsyncThreadedTaskPool() as pool:
      
      result = await pool.run(func())
      
      future = pool.create_task(func())
  ```



  Task that can be started and cancelled multiple times until it can finally be completed.
  This is useful to handle pauses and retries when handling with a connection.

  > 💡 Use `functools.partial` to pass parameters to the function.

  Example usage:
  ```python
  from functools import partial
  from concurrent_tasks import RestartableTask

  async def send(data): ...

  task: RestartableTask[int] = RestartableTask(partial(send, b"\x00"), timeout=1)
  task.start()
  assert await task == 1




  task.cancel()

  task.start()

  task.set_result(1)
  ```



  Shut down process when an unhandled exception is caught or a signal is received.
  To make this a graceful stop, pass a `stop_func`.

  When creating multiple background tasks, exceptions raised within those will be forwarded directly to the event loop.
  In order to act on those exceptions, we need to use `loop.set_exception_handler`.

  > 💡 When a signal is received and the process is already shutting down, it will be force killed.

  Example minimalistic implementation:

  ```python
  import asyncio
  from concurrent_tasks import LoopExceptionHandler

  async def run():
      event = asyncio.Event()
      tasks = []
      async def _stop():
          await asyncio.gather(*tasks)
          event.set()
      async with LoopExceptionHandler(_stop):
          
          await event.wait()
  ```


pkgId: 9579a4b4131b103f409d6f62336b06c14e5ec22b517ec0a8a880b81f5977ffcb
name: python-concurrently
arch: src
version: 2
epoch: 0
release: 1
summary: Library helps easy write concurrent executed code blocks
url: https://github.com/sirkonst/concurrently
rpm_license: MIT License
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-concurrently-2-1.src.rpm
checksum_type: sha256
description:
  Library helps to easily write concurrent executed code blocks.
  Quick example:
      import asyncio
      from concurrently import concurrently
      async def amain(loop):
          """
          How to fetch some web pages with concurrently.
          """
          urls = [  # define pages urls
              'http://test/page_1',
              'http://test/page_2',
              'http://test/page_3',
              'http://test/page_4',
          ]
          results = {}
          
          
          @concurrently(2)
          async def fetch_urls():
              for url in urls:
                  
                  page = await fetch_page(url)
                  results[url] = page
          
          await fetch_urls()
          print(results)
      if __name__ == '__main__':
          loop = asyncio.get_event_loop()
          loop.run_until_complete(amain(loop))
  Decorator `@concurrently` makes to main thinks:
      * starts concurrent execution specified count of decorated function
      * returns special `waiter` object to control the running functions
  By default, the code runs as asyncio coroutines, but there are other supported
  ways to execute, by specifying the argument `engine`.


pkgId: 40fc7356a1d92524b773429a2a969bceeb608535919c8d70196dcbde7ddc09b6
name: python-conda-app
arch: src
version: 0.4.3
epoch: 0
release: 1
summary: Install applications with conda
url: https://foss.heptapod.net/fluiddyn/conda-app/
rpm_license: BSD
rpm_group: Unspecified
rpm_buildhost: localhost
location_href: Packages/python-conda-app-0.4.3-1.src.rpm
checksum_type: sha256
description:
  [![PyPI][pypi-badge]][pypi-link]
  [![Github Actions Unix][GH-badge-unix]][GH-link-unix]
  [![Github Actions Windows][GH-badge-windows]][GH-link-windows]
  [![Heptapod CI][Heptapod-badge]][Heptapod-link]



  `conda-app` is a tiny `conda` extension (actually a commandline tool using
  `conda` or `mamba`) to install applications in isolated environments. Like
  [pipx](https://github.com/pypa/pipx) but with conda environments.

  The main advantages are:

  - very simple **cross-platform** installation commands for Windows, macOS and
    Linux (and different shells, as bash, fish and zsh).

  - the applications are installed in **isolated** environments.

  - commands provided by the applications are **available system-wide**, i.e. even
    when the associated conda environment is not activated.

  - Installation from the `conda-forge` channel so there is **no need for
  compilation**.



  ```bash
  pip install conda-app
  ```



  Mercurial and common extensions (`hg-git` and `hg-evolve`) can be installed with:

  ```bash
  conda-app install mercurial
  ```

  Then, in **a new terminal** (on Windows, the "Conda Prompt"), the Mercurial
  command `hg` should be available so one can try `hg version -v`.

  This should also work:

  ```raw
  $ conda-app list
  Installed applications:
   ['mercurial', 'spyder', 'pandoc']

  $ conda-app uninstall pandoc
  ...
  ```

  [pypi-badge]: https://img.shields.io/pypi/v/conda-app.svg
  [pypi-link]: https://pypi.python.org/pypi/conda-app/
  [GH-badge-unix]: https://github.com/fluiddyn/conda-app/actions/workflows/unix.yml/badge.svg?branch=branch/default
  [GH-link-unix]: https://github.com/fluiddyn/conda-app/actions/workflows/unix.yml
  [GH-badge-windows]: https://github.com/fluiddyn/conda-app/actions/workflows/windows.yml/badge.svg?branch=branch/default
  [GH-link-windows]: https://github.com/fluiddyn/conda-app/actions/workflows/windows.yml
  [Heptapod-badge]: https://foss.heptapod.net/fluiddyn/conda-app/badges/branch/default/pipeline.svg
  [Heptapod-link]: https://foss.heptapod.net/fluiddyn/conda-app/-/pipelines


