What is Argo Workflows?

Argo Workflows is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes. Argo Workflows is implemented as a Kubernetes CRD (Custom Resource Definition).

• Define workflows where each step in the workflow is a container.
• Model multi-step workflows as a sequence of tasks or capture the dependencies between tasks using a graph (DAG).
• Easily run compute intensive jobs for machine learning or data processing in a fraction of the time using Argo Workflows on Kubernetes.
• Run CI/CD pipelines natively on Kubernetes without configuring complex software development products.

Why Argo Workflows?

• Designed from the ground up for containers without the overhead and limitations of legacy VM and server-based environments.
• Cloud agnostic and can run on any Kubernetes cluster.
• Easily orchestrate highly parallel jobs on Kubernetes.
• Argo Workflows puts a cloud-scale supercomputer at your fingertips!

Quickstart

kubectl create namespace argo
kubectl apply -n argo -f https://raw.githubusercontent.com/argoproj/argo/stable/manifests/install.yaml

Who uses Argo Workflows?

Official Argo Workflows user list

Documentation

• Get started here
• How to write Argo Workflow specs
• How to configure your artifact repository

Features

• DAG or Steps based declaration of workflows
• Artifact support (S3, Artifactory, HTTP, Git, raw)
• Step level input & outputs (artifacts/parameters)
• Loops
• Parameterization
• Conditionals
• Timeouts (step & workflow level)
• Retry (step & workflow level)
• Resubmit (memoized)
• Suspend & Resume
• Cancellation
• K8s resource orchestration
• Exit Hooks (notifications, cleanup)
• Garbage collection of completed workflow
• Scheduling (affinity/tolerations/node selectors)
• Volumes (ephemeral/existing)
• Parallelism limits
• Daemoned steps
• DinD (docker-in-docker)
• Script steps

Community Blogs and Presentations

• Argo Ansible role: Provisioning Argo Workflows on OpenShift
• Argo Workflows vs Apache Airflow
• CI/CD with Argo on Kubernetes
• Running Argo Workflows Across Multiple Kubernetes Clusters
• Open Source Model Management Roundup: Polyaxon, Argo, and Seldon
• Producing 200 OpenStreetMap extracts in 35 minutes using a scalable data workflow
• Argo integration review
• TGI Kubernetes with Joe Beda: Argo workflow system
• Community meeting minutes and recordings

Project Resources

• Argo GitHub: https://github.com/argoproj
• Argo website: https://argoproj.github.io/
• Argo Slack: click here to join