Hub-of-Hubs Nonk8s API

The REST API component of Hub-of-Hubs.

Rationale

While a Kubernetes Extension API server can be used to provide access to the items in the Hub-of-Hubs scalabale database, such a server would have the following drawbacks:

1. The API schema and URL parameters must confirm to the API schema of Kubernetes. In particular, no sort parameter could be passed for list operations (see list options, parsed by the API server code).
2. No advanced query capabilities (only label selectors of Kubernetes)
3. Watch from a revision version - hard to implmenent revision version mechanism for an SQL database (no build-in concept of revision versions in SQL)
4. The clients of a Kubernetes API server may try to cache all the resources, and can break as a result of caching a large number of resources.
5. Such API server must be REST (and not GRPC, for example).

Environment variables

The following environment variables are required for the most tasks below:

• REGISTRY, for example docker.io/vadimeisenbergibm.
• IMAGE_TAG, for example v0.1.0.

Build to run locally

make build

Run Locally

Set the following environment variables:

• DATABASE_URL - the URL of the database server
• CLUSTER_API_URL - the URL of the Kubernetes API server
• CLUSTER_API_CA_BUNDLE_PATH - the CA bundle for the Kubernetes API server. If not provided, verification of the server certificates is skipped.
• AUTHORIZATION_URL - the URL of the authorization server
• AUTHORIZATION_CA_BUNDLE_PATH - the CA bundle for the authorization server. If not provided, verification of the server certificates is skipped.
• KEY_PATH - the path to the file that contains the private key for this server's TLS.
• CERTIFICATE_PATH - the path to the file that contains the certificate for this server's TLS.

Set the DATABASE_URL according to the PostgreSQL URL format: postgres://YourUserName:YourURLEscapedPassword@YourHostname:5432/YourDatabaseName?sslmode=verify-full&pool_max_conns=50.

❗ Remember to URL-escape the password, you can do it in bash:

python -c "import sys, urllib as ul; print ul.quote_plus(sys.argv[1])" 'YourPassword'

Generate self-signed certificates:

mkdir testdata
openssl genrsa -out ./testdata/server.key 2048
openssl req -new -x509 -key ./testdata/server.key -out ./testdata/server.pem -days 365

Run the server

./bin/hub-of-hubs-nonk8s-api

Build image

make build-images

Deploy to a cluster

1. Create a secret with your database url:

   kubectl create secret generic hub-of-hubs-database-secret --kubeconfig $TOP_HUB_CONFIG -n open-cluster-management --from-literal=url=$DATABASE_URL
   

2. Deploy the operator:

   COMPONENT=$(basename $(pwd)) IMAGE_TAG=latest envsubst < deploy/operator.yaml.template | kubectl apply --kubeconfig $TOP_HUB_CONFIG -n open-cluster-management -f -
   

3. Deploy Ingress:

   COMPONENT=$(basename $(pwd)) envsubst < deploy/ingress.yaml.template | kubectl apply --kubeconfig $TOP_HUB_CONFIG -n open-cluster-management -f -
   

Test the ingress

Note that the port is 443 (the standard HTTPS port).

curl -ks  https://multicloud-console.apps.<the hub URL>/multicloud/hub-of-hubs-nonk8s-api/managedclusters  -H "Authorization: Bearer $TOKEN" | jq .[].metadata.name

Working with Kubernetes deployment

Show log:

kubectl logs -l name=$(basename $(pwd)) -n open-cluster-management

Execute commands on the container:

kubectl exec -it $(kubectl get pod -l name=$(basename $(pwd)) -o jsonpath='{.items..metadata.name}' -n open-cluster-management) \
-n open-cluster-management -- bash

Test (run the commands in this directory)

Add `example.com` to /etc/hosts as local host.

export TOKEN=<the OC token or Service Account token from its secret>

curl https://example.com:8080/managedclusters -w "%{http_code}\n" -H "Authorization: Bearer $TOKEN" --cacert ./certs/tls.crt

curl -s https://example.com:8080/managedclusters  -H "Authorization: Bearer $TOKEN" --cacert ./certs/tls.crt |
     jq .[].metadata.name

Exercise the deployed API

1. Define TOKEN and CLUSTER_URL environment variables. Get TOKEN from copy login command in the OpenShift console. Alternatively, if you want to run commands as a service account, check the secret of the service account which appears in the secrets field of the secret account, for example <sa name>-token-XXXX. Then run kubectl get secret <secret name> -o jsonpath="{.data.token}" | base64 -d to get the token from the secret and to decrypt it from base 64. CLUSTER_URL is the part of URL of the ACM console, after multicloud-console.apps.

2. Show the current identity:

   curl -k https://api.$CLUSTER_URL:6443/apis/user.openshift.io/v1/users/~ -H "Authorization: Bearer $TOKEN"
   

3. Show the managed clusters in Non-Kubernetes REST API:

   curl -ks https://multicloud-console.apps.$CLUSTER_URL/multicloud/hub-of-hubs-nonk8s-api/managedclusters -H "Authorization: Bearer $TOKEN" | jq .[].metadata.name | sort
   

4. Add a label a=b:

   curl -ks https://multicloud-console.apps.$CLUSTER_URL/multicloud/hub-of-hubs-nonk8s-api/managedclusters/cluster20 -H "Authorization: Bearer $TOKEN" -H 'Accept: application/json' -X PATCH -d '[{"op":"add","path":"/metadata/labels/a","value":"b"}]]' -w "%{http_code}\n"