Pipelines built on top of Allegro ClearML.
This repository contains machine learning pipelines mostly based on PyTorch. Every pipeline is designed to be published on a Allegro ClearML Kubernetes cluster on premise.
Each folder contains needed code and README for pipeline usage.
Further pipelines are welcome via pull request.
- mushrooms - Complete pipeline for a simple Pytorch model on a tabular mushrooms dataset.
- inat-2019 - Complete pipeline for a MobilenetV2 model on iNaturalist 2019 dataset [WIP].
Here some prerequisites needed to deploy this repo.
- Allegro ClearML >=0.17.4
- PyTorch >=1.7.1
Kubernetes installation can be done using ClearML chart at https://artifacthub.io/packages/helm/valeriano-manassero/clearml
A (hopefully) good alternative is the usage of GitOps paradigm with declaration of the entire cluster at https://github.com/valeriano-manassero/mlops-k8s-infra
Some python libraries are needed. it's possible to locally install them with:
pip install -r requirements.txt
requirements.txt files are on every pipeline folder. No need of manual install inside a ClearML task (instlllation should be automatic).