This repository is associated to a series of blog posts where we build a general-purpose Big-Data platform running on Kubernetes.
In those posts we explain the reasoning behind our choice of tools, and introduce some key technology concepts. In this repository we build a demo of the platform, and we also provide some examples of usage.
The demo runs on a Kind cluster, and tools are configured without security or with the minimal default security.
The blog posts (in Spanish) can be found here:
See the deployment folder for instructions on how to install the tools on a Kubernetes cluster, and the examples folder for basic usage examples.
The docker-images folder contains the Dockerfiles used to prepare some of the images used in the installation and example yamls.
- Minio as object storage,
- Postgres, for less resource consuming data projects, and as back-end database for other tools, and
- Argo workflows as workflow manager.