Prerequisites
- Active Aiven account and project (https://aiven.io/)
- Aiven CLI (https://github.com/aiven/aiven-client)
- Terraform (https://learn.hashicorp.com/tutorials/terraform/install-cli)
- k8s cluster and kubectl command line tool
- Helm 3 (https://helm.sh/docs/intro/install/)
- jq command line tool (https://stedolan.github.io/jq/download/)
- kcat tool (https://github.com/edenhill/kcat)
- Jaeger K8s operator for tracing (https://jaegertracing.github.io/helm-charts/)
- Keda K8s operator for autoscaling (https://keda.sh/docs/2.10/deploy/#helm)
cd <project-root>/infra
terraform apply
./get-demo-secrets.sh <aiven-project-name>
cd <project-root>/database
./import-stations.sh
Example deployments are utilising OpenTelemetry tracing. Jaeger is now fully compatible with OTLP tracing protocol so we only need Jaeger Operator. Follow the instructions on Jaeger Operator Helm Chart repo https://github.com/jaegertracing/helm-charts/tree/main/charts/jaeger-operator
https://keda.sh/docs/2.10/deploy/
helm repo add kedacore https://kedacore.github.io/charts
helm repo update
helm install keda kedacore/keda --namespace keda --create-namespace
List of Helm releases should now look something like this
❯ helm list -A
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
cert-manager cert-manager 1 2023-04-13 10:52:16.765519 +0300 EEST deployed cert-manager-v1.11.1 v1.11.1
jaeger-operator observability 1 2023-08-15 15:26:01.075019 +0300 EEST deployed jaeger-operator-2.46.2 1.46.0
keda keda 1 2023-04-03 11:55:40.19998 +0300 EEST deployed keda-2.10.1 2.10.0
Your Kubernetes cluster is now ready for deploying the example applications.
cd <project-root>/k8s
kubectl create -f namespace.yaml
kubectl apply -k .
Jager UI provides user interface searching and visualiszing trace data. First forward Jeager port number 16686 to your host
kubectl -n tms-demo port-forward svc/simple-aiven-query 16686
Then you can open the localhost address http://localhost:16686 in your favourite Web browser. You will see the full path for observation event from receiving it as MQTT message and finally being processed by the Kafka Streams applications.
Follow instructions here