Metadata-Version: 2.1
Name: zenml
Version: 0.8.1rc0
Summary: ZenML: Write production-ready ML code.
Home-page: https://zenml.io
License: Apache-2.0
Keywords: machine learning,production,pipeline,mlops,devops
Author: ZenML GmbH
Author-email: info@zenml.io
Requires-Python: >=3.7.1,<3.10
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: System :: Distributed Computing
Classifier: Typing :: Typed
Requires-Dist: analytics-python (>=1.4.0,<2.0.0)
Requires-Dist: apache-beam (>=2.30.0,<3.0.0)
Requires-Dist: click (>=8.0.1,<9.0.0)
Requires-Dist: click-params (>=0.3.0,<0.4.0)
Requires-Dist: distro (>=1.6.0,<2.0.0)
Requires-Dist: fastapi (>=0.75.0,<0.76.0)
Requires-Dist: gitpython (>=3.1.18,<4.0.0)
Requires-Dist: httplib2 (>=0.19.1,<0.20)
Requires-Dist: markupsafe (==1.1.1)
Requires-Dist: ml-pipelines-sdk (==1.8.0rc1)
Requires-Dist: nbconvert (==6.4.4)
Requires-Dist: pandas (>=1.1.5,<2.0.0)
Requires-Dist: pydantic (>=1.9.0,<2.0.0)
Requires-Dist: pyparsing (>=2.4.0,<3)
Requires-Dist: python-dateutil (>=2.8.1,<3.0.0)
Requires-Dist: pyyaml (>=5.4.1,<6.0.0)
Requires-Dist: rich[jupyter] (>=12.0.0,<13.0.0)
Requires-Dist: semver (>=2.13.0,<3.0.0)
Requires-Dist: sqlmodel (>=0.0.6,<0.1.0)
Requires-Dist: uvicorn[standard] (>=0.17.5,<0.18.0)
Project-URL: Documentation, https://docs.zenml.io
Project-URL: Repository, https://github.com/zenml-io/zenml
Description-Content-Type: text/markdown

<div align="center">
    <img src="https://zenml.io/assets/social/github.svg">
</div>

# ⏲️ Join the ZenML team on the MLOps Day

We are hosting a MLOps day where we'll be building a vendor-agnostic MLOps pipeline from scratch.

Sign up [here](https://www.eventbrite.com/e/zenml-mlops-day-join-us-in-building-a-vendor-agnostic-mlops-pipeline-tickets-336331515617) to join the entire ZenML team in showcasing the latest release, answering the community's questions, and live-coding vendor agnostic MLOps features with the ZenML framework!

# 👀 What is ZenML?

**ZenML** is an extensible, open-source MLOps framework to create
production-ready machine learning pipelines. Built for data scientists, it has a
simple, flexible syntax, is cloud- and tool-agnostic, and has
interfaces/abstractions that are catered towards ML workflows.

At its core, **ZenML pipelines execute ML-specific workflows** from sourcing
data to splitting, preprocessing, training, all the way to the evaluation of
results and even serving. There are many built-in batteries to support common ML
development tasks. ZenML is not here to replace the great tools that solve these
individual problems. Rather, it offers an **extensible framework** and a
standard abstraction to write and build your workflows.

🎉 **Version 0.8.0 out now!** [Check out the release notes
here](https://github.com/zenml-io/zenml/releases).

[![PyPI - Python
Version](https://img.shields.io/pypi/pyversions/zenml)](https://pypi.org/project/zenml/)
[![PyPI Status](https://pepy.tech/badge/zenml)](https://pepy.tech/project/zenml)
![GitHub](https://img.shields.io/github/license/zenml-io/zenml)
[![Codecov](https://codecov.io/gh/zenml-io/zenml/branch/main/graph/badge.svg)](https://codecov.io/gh/zenml-io/zenml)
[![Interrogate](docs/interrogate.svg)](https://interrogate.readthedocs.io/en/latest/)
[![Main Workflow
Tests](https://github.com/zenml-io/zenml/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/zenml-io/zenml/actions/workflows/ci.yml)

<div align="center">
Join our <a href="https://zenml.io/slack-invite" target="_blank">
    <img width="25" src="https://cdn3.iconfinder.com/data/icons/logos-and-brands-adobe/512/306_Slack-512.png" alt="Slack"/>
<b>Slack Community</b> </a> and become part of the ZenML family
</div>
<div align="center"> Give us a 
    <img width="25" src="https://cdn.iconscout.com/icon/free/png-256/github-153-675523.png" alt="Slack"/>
<b>GitHub star</b> to show your love
</div>
<div align="center"> 
    <b>NEW: </b> <a href="https://zenml.io/discussion" target="_blank"><img width="25" src="https://cdn1.iconfinder.com/data/icons/social-17/48/like-512.png" alt="Vote"/><b> Vote</b></a> on the next ZenML features 
</div>

<br>

# 🤖 Why use ZenML?

ZenML pipelines are designed to be written early on the development lifecycle.
Data scientists can explore their pipelines as they develop towards production,
switching stacks from local to cloud deployments with ease. You can read more
about why we started building ZenML [on our
blog](https://blog.zenml.io/why-zenml/). By using ZenML in the early stages of
your project, you get the following benefits:

- **Extensible** so you can build out the framework to suit your specific needs
- **Reproducibility** of training and inference workflows
- A **simple and clear** way to represent the steps of your pipeline in code
- **Batteries-included integrations**: bring all your favorite tools together
- Easy switch between local and cloud stacks
- Painless **deployment and configuration** of infrastructure

# 📖 Learn More

| ZenML Resources | Description |
| ------------- | - |
| 🧘‍♀️ **[ZenML 101]** | New to ZenML? Here's everything you need to know! |
| ⚛️ **[Core Concepts]** | Some key terms and concepts we use. |
| 🗃 **[Functional API Guide]** | Build production ML pipelines with simple functions. |
| 🚀 **[New in v0.8.0]** | New features, bug fixes. |
| 🗳 **[Vote for Features]** | Pick what we work on next! |
| 📓 **[Docs]** | Full documentation for creating your own ZenML pipelines. |
| 📒 **[API Reference]** | The detailed reference for ZenML's API. |
| 🍰 **[ZenBytes]** | A guided and in-depth tutorial on MLOps and ZenML. |
| 🗂️️ **[ZenFiles]** | End-to-end projects using ZenML. |
| ⚽️ **[Examples]** | Learn best through examples where ZenML is used? We've got you covered. |
| 📬 **[Blog]** | Use cases of ZenML and technical deep dives on how we built it. |
| 🔈 **[Podcast]** | Conversations with leaders in ML, released every 2 weeks. |
| 📣 **[Newsletter]** | We build ZenML in public. Subscribe to learn how we work. |
| 💬 **[Join Slack]** | Need help with your specific use case? Say hi on Slack! |
| 🗺 **[Roadmap]** | See where ZenML is working to build new features. |
| 🙋‍♀️ **[Contribute]** | How to contribute to the ZenML project and code base. |

[ZenML 101]: https://docs.zenml.io/
[Core Concepts]: https://docs.zenml.io/core-concepts
[Functional API Guide]: https://docs.zenml.io/v/docs/guides/functional-api
[New in v0.8.0]: https://github.com/zenml-io/zenml/releases
[Vote for Features]: https://zenml.io/discussion
[Docs]: https://docs.zenml.io/
[API Reference]: https://apidocs.zenml.io/
[ZenBytes]: https://github.com/zenml-io/zenbytes
[ZenFiles]: https://github.com/zenml-io/zenfiles
[Examples]: https://github.com/zenml-io/zenml/tree/main/examples
[Blog]: https://blog.zenml.io/
[Podcast]: https://podcast.zenml.io/
[Newsletter]: https://zenml.io/newsletter/
[Join Slack]: https://zenml.io/slack-invite/
[Roadmap]: https://zenml.io/roadmap
[Contribute]: https://github.com/zenml-io/zenml/blob/main/CONTRIBUTING.md

# 🎮 Features

### 1. 💪 Write local, run anywhere

You only need to write your core machine learning workflow code once, but you
can run it anywhere. We decouple your code from the environment and
infrastructure on which this code runs.

Switching from local experiments to cloud-based pipelines doesn't need to be
complicated. ZenML supports running pipelines wherever you want, for example by
using Kubeflow, one of our built-in integrations, or any orchestrator of your
choice. Switching from your local stack to a cloud stack is easy to do with our
CLI tool.

![You can run your pipelines locally or in the
cloud](docs/book/assets/core_concepts/concepts-3.png)

### 2. 🌈 All your MLOps stacks in one place

Once code is organized into a ZenML pipeline, you can supercharge your ML
development with [powerful
integrations](https://docs.zenml.io/features/integrations) on multiple [MLOps
stacks](https://docs.zenml.io/core-concepts). There are lots of moving parts for
all the MLOps tooling and infrastructure you require for ML in production and
ZenML aims to bring it all together under one roof.

We already support common use cases and integrations to standard ML tools via
our stack components, from orchestrators like Airflow and Kubeflow to model
deployment via MLflow or Seldon Core, to custom infrastructure for training your
models in the cloud and so on. If you want to learn more about our integrations,
check out [our Examples](https://github.com/zenml-io/zenml/tree/main/examples)
to see how they work.

![ZenML is the glue](docs/book/assets/stack-list.png)

### 3. 🛠 Extensibility

ZenML's Stack Components are built to support most machine learning use cases.
We offer a batteries-included initial installation that should serve many needs
and workflows, but if you need a special kind of monitoring tool added, for
example, or a different orchestrator to run your pipelines, ZenML is built as a
framework making it easy to extend and build out whatever you need.

![ZenML is fully extensible](docs/book/assets/extensibility.gif)

### 4. 🔍 Automated metadata tracking

ZenML tracks metadata for all the pipelines you run. This ensures that:

- Code is versioned
- Data is versioned
- Models are versioned
- Configurations are versioned

This also enables caching of the data that powers your pipelines which helps you
iterate quickly through ML experiments. (Read [our
blogpost](https://blog.zenml.io/caching-ml-pipelines/) to learn more!)

![Visualize your pipeline steps](docs/book/assets/dag-visualizer.png)

### 5. ➿ Continuous Training and Continuous Deployment (CT/CD)

Continuous Training (CT) refers to the paradigm where a team deploys training pipelines 
that run automatically to train models on new (fresh) data. Continuous Deployment (CD) 
refers to the paradigm where newly trained models are automatically deployed to a prediction 
service/server

ZenML enabled CT/CD by enabling the model preparation and model training with model deployment. 
With the built-in functionalities like Schedules, Model Deployers and Services you can 
create end-to-end ML workflows with Continuous Training and Deployment that deploys your 
model in a local environment with MLFlow integration or even in a production-grade environment 
like Kubernetes with our Seldon Core integration. You can also listed served models with the CLI:

![CI/CD/CT in ZenML](docs/book/assets/ct_cd_zenml.gif)

```
zenml served-models list
```

Read more about CT/CD in ZenML [here](https://blog.zenml.io/ci-ct-cd-with-zenml/).

# 🤸 Getting Started

## 💾 Install ZenML

*Requirements*: ZenML supports Python 3.7 and 3.8.

ZenML is available for easy installation into your environment via PyPI:

```bash
pip install zenml
```

Alternatively, if you’re feeling brave, feel free to install the bleeding edge:
**NOTE:** Do so on your own risk, no guarantees given!

```bash
pip install git+https://github.com/zenml-io/zenml.git@main --upgrade
```

ZenML is also available as a Docker image hosted publicly on
[DockerHub](https://hub.docker.com/r/zenmldocker/zenml). Use the following
command to get started in a bash environment:

```shell
docker run -it zenmldocker/zenml /bin/bash
```

## 🚅 Quickstart

The quickest way to get started is to create a simple pipeline.

#### Step 1: Initialize a ZenML repo

```bash
zenml init
zenml integration install sklearn -y # we use scikit-learn for this example
```

#### Step 2: Assemble, run, and evaluate your pipeline locally

```python
import numpy as np
from sklearn.base import ClassifierMixin

from zenml.integrations.sklearn.helpers.digits import get_digits, get_digits_model
from zenml.pipelines import pipeline
from zenml.steps import step, Output

@step
def importer() -> Output(
    X_train=np.ndarray, X_test=np.ndarray, y_train=np.ndarray, y_test=np.ndarray
):
    """Loads the digits array as normal numpy arrays."""
    X_train, X_test, y_train, y_test = get_digits()
    return X_train, X_test, y_train, y_test


@step
def trainer(
    X_train: np.ndarray,
    y_train: np.ndarray,
) -> ClassifierMixin:
    """Train a simple sklearn classifier for the digits dataset."""
    model = get_digits_model()
    model.fit(X_train, y_train)
    return model


@step
def evaluator(
    X_test: np.ndarray,
    y_test: np.ndarray,
    model: ClassifierMixin,
) -> float:
    """Calculate the accuracy on the test set"""
    test_acc = model.score(X_test, y_test)
    print(f"Test accuracy: {test_acc}")
    return test_acc


@pipeline
def mnist_pipeline(
    importer,
    trainer,
    evaluator,
):
    """Links all the steps together in a pipeline"""
    X_train, X_test, y_train, y_test = importer()
    model = trainer(X_train=X_train, y_train=y_train)
    evaluator(X_test=X_test, y_test=y_test, model=model)


pipeline = mnist_pipeline(
    importer=importer(),
    trainer=trainer(),
    evaluator=evaluator(),
)
pipeline.run()
```

# :racehorse: Get a guided tour with `zenml go`

For a slightly more in-depth introduction to ZenML, taught through Jupyter
notebooks, install `zenml` via pip as described above and type:

```shell
zenml go
```

This will spin up a Jupyter notebook that showcases the above example plus more
on how to use and extend ZenML.

# 👭 Collaborate with your team

ZenML is built to support teams working together. The underlying infrastructure
on which your ML workflows run can be shared, as can the data, assets and
artifacts that you need to enable your work. ZenML Profiles offer an easy way to
manage and switch between your stacks. The ZenML Server handles all the
interaction and sharing and you can host it wherever you'd like.

```
zenml server up
```

Read more about collaboration in ZenML [here](https://docs.zenml.io/collaborate/collaborate).

# 🍰 ZenBytes

[ZenBytes](https://github.com/zenml-io/zenbytes) is a series of short practical
MLOps lessons through ZenML and its various integrations. It is intended for
people looking to learn about MLOps generally, and also for ML practitioners who
want to get started with ZenML.

After you've run and understood the simple example above, your next port of call
is probably either the [fully-fleshed-out quickstart
example](https://github.com/zenml-io/zenml/tree/main/examples/quickstart) and
then to look at [the ZenBytes repository](https://github.com/zenml-io/zenbytes)
and notebooks.

# 🗂️ ZenFiles

ZenFiles are production-grade ML use-cases powered by ZenML. They are fully
fleshed out, end-to-end, projects that showcase ZenML's capabilities. They can
also serve as a template from which to start similar projects.

The ZenFiles project is fully maintained and can be viewed as a sister
repository of ZenML. Check it out [here](https://github.com/zenml-io/zenfiles).

# 🗺 Roadmap

ZenML is being built in public. The [roadmap](https://zenml.io/roadmap) is a
regularly updated source of truth for the ZenML community to understand where
the product is going in the short, medium, and long term.

ZenML is managed by a [core team](https://zenml.io/team) of developers that are
responsible for making key decisions and incorporating feedback from the
community. The team oversees feedback via various channels, and you can directly
influence the roadmap as follows:

- Vote on your most wanted feature on our [Discussion
  board](https://zenml.io/discussion). You can also request for new features here.
- Start a thread in our [Slack channel](https://zenml.io/slack-invite).

# 🙋‍♀️ Contributing & Community

We would love to develop ZenML together with our community! Best way to get
started is to select any issue from the [`good-first-issue`
label](https://github.com/zenml-io/zenml/labels/good%20first%20issue). If you
would like to contribute, please review our [Contributing
Guide](CONTRIBUTING.md) for all relevant details.

<br>

![Repobeats analytics
image](https://repobeats.axiom.co/api/embed/635c57b743efe649cadceba6a2e6a956663f96dd.svg
"Repobeats analytics image")


# 🆘 Where to get help

First point of call should be [our Slack group](https://zenml.io/slack-invite/).
Ask your questions about bugs or specific use cases and someone from the core
team will respond.

# 📜 License

ZenML is distributed under the terms of the Apache License Version 2.0. A
complete version of the license is available in the [LICENSE.md](LICENSE.md) in
this repository. Any contribution made to this project will be licensed under
the Apache License Version 2.0.

