Metadata-Version: 2.1
Name: sage-testsuite
Version: 0.1.0a3
Summary: testsuite for FATE
Home-page: https://fate.fedai.org/
License: Apache-2.0
Keywords: FATE,Federated Learning,Testsuite
Author: FederatedAI
Author-email: contact@FedAI.org
Requires-Python: >=3.6,<4.0
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Topic :: Software Development :: Testing
Requires-Dist: click (>=7.1.2,<8.0.0)
Requires-Dist: dataclasses (>=0.7,<0.8); python_version == "3.6"
Requires-Dist: loguru (>=0.5.1,<0.6.0)
Requires-Dist: prettytable (>=0.7.2,<0.8.0)
Requires-Dist: requests (>=2.24.0,<3.0.0)
Requires-Dist: requests_toolbelt (>=0.9.1,<0.10.0)
Requires-Dist: ruamel.yaml (>=0.16.10,<0.17.0)
Requires-Dist: sshtunnel (>=0.1.5,<0.2.0)
Project-URL: Documentation, https://fate.readthedocs.io/en/latest/?badge=latest
Project-URL: Repository, https://github.com/FederatedAI/FATE
Description-Content-Type: text/x-rst

testsuite
==============

A useful script to running FATE's testsuites.

quick start
-----------

1. (optional) create virtual env

   .. code-block:: bash

      python -m venv venv
      source venv/bin/activate


2. install fate_testsuite

   .. code-block:: bash

      pip install fate_testsuite
      fate_testsuite --help


3. new and edit the testsuite_config.yaml

   .. code-block:: bash

      # create a testsuite_config.yaml in current dir
      testsuite config new
      # edit priority config file with system default editor
      # filling some field according to comments
      fate_testsuite config edit


4. run some testsuites

   .. code-block:: bash

      fate_testsuite suite -i <path contains *testsuite.json>

5. useful logs or exception will be saved to logs dir with namespace showed in last step


testsuite_config.yaml examples
------------------------------

1. deploy multiple flow services in single node: `ip:port`, and run testsuite in different node:

   - party 9999, port 9380
   - party 10000, port 9381

   filling `ssh_tunnel` and comments out `local_services`

   .. code-block:: yaml

      ssh_tunnel:
          - address: ip:port
          - services:
              - address: 127.0.0.1:9380
                parties: [9999]
              - address: 127.0.0.1:9381
                parties: [10000]


2. deploy multiple flow services in single node: `ip:port`, and run testsuite in same node:

   - party 9999, port 9380
   - party 10000, port 9381

   filling `local_services` and commnets out `ssh_tunnel`

   .. code-block:: yaml

      local_services: # flow services in local
          - address: 127.0.0.1:9380
            parties: [9999]
          - address: 127.0.0.1:9381
            parties: [10000]


command options
---------------

1. exclude:

   .. code-block:: bash

      testsuite suite -i <path1 contains *testsuite.json> -e <path2 to exclude> -e <path3 to exclude> ...

   will run testsuites in `path1` but not in `path2` and `path3`

2. replace:

   .. code-block:: bash

      testsuite suite -i <path1 contains *testsuite.json> -r '{"maxIter": 5}'

   will find all key-value pair with key "maxIter" in `data conf` or `conf` or `dsl` and replace the value with 5
