Skip to content

Commit

Permalink
test
Browse files Browse the repository at this point in the history
  • Loading branch information
bmd1905 committed Aug 27, 2024
1 parent b4bfbeb commit 73cd4d7
Showing 1 changed file with 22 additions and 9 deletions.
31 changes: 22 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,13 @@ cd iac/terraform
terraform init
```

<video controls style="width: 100%; height: auto;">
<source" src="https://ffc2f9163061eaa84c660f92e3b06dcbbe78d37a01072cf1f4b3451-apidata.googleusercontent.com/download/storage/v1/b/promptalchemy/o/videos%2F1-init-cluster.mp4?jk=AXZI6Jy1boMVifNNgQdvnGsz9UxZF8HMIql1daLmJqtccxGPKHcowL3x3f7QUHNR0ARJ0txYpGyBYy29j2R78HQp8IMZ3--hFsfmXXb6WQc8rW2z2RwRiaTgrRfXtBPuc4ZdqwEM4OmlZf7CwGELNwM64JABNHPTDjUTN00ZVWyMOXxBZ-ViE6RVrqr6aQi4955tB01ECQ6WlxJV7ApC9abQG8vqD2zUKSfSfpppwfU9MSvrGjyTNEupkjTjjVC76G9-11hKHBzRgs_kW8kswGGeC3lNcd-xbWxe-L16FGWElfpGuNCqbKOC2UHGxd7y9VPrho3sSQsGEMoMP4tJVi5sWpQ_xQzJ5tF58Xba1OGLN6P1QPo6Q79-URTNRmOGJTo9ltwwBWwsooR1BO7LLEhmT8-TvboMZDrTfKOYjeOTg3_PWVeWrfnyzF1qGj0rY8_zuxy7wj-SXd7qnzX3FXJ8giMxjUgXS3EIUTVy2CO3pHvff_-ifH2D8rm_yCxsCz7qgChpH8N4oDiWAgOJDrFXKKAtvH2Gjoi7tC7DLKAC5DDkICLScGRVEr3nAdIF9kDjpZe-Utd28bd_R7K4bJA4KV-IgUOhBz8wgxoi4D_dkid6zg3O9f59pPPbN5-ujya2S-zjtKz-v_OVhUj3rk1GwuZaO0_LdMGoP8hbe0Sv7ErVRHrbKfw4OmloH_HB4KGFu_27tvUqyANKVl4vQCcxvHOPImLd0qwdnFCTLPilxhDAXYAdLgFBs3KT2NG_zyQ_PbVG5h81A3Ne3L_Zg5R7eFUwksjlgQNSTICyePCuoe-yYC8TuqfgUMPo1OdX-qwyvoK0xN7GMCeFc2e5k5TB4LUEanUh5i9Colzosu31-FBsmQ6Xswiv0I5X1TP0b-GZgIgI3Wb3QNtXp_3UczRut_y_CLtRMtnEK4yYW_nybZ4jEaWvC50-tBBOHAATQAjvFHyhxGtU8G_RSdAmHp0_ISaxS3nnYwqwOAOfyII14bdbJwfQi1gHuQKlY6dPQjyvfmTaUyfdkcUJr-PzbojenmM7981m1UOPMpoUSSvEKtaceQXS10Bs06TA8pPdZWr-CXD3rVcG4gjxHHWFSgLABQTO3Px8IQN2x5AV9vSI-H_An03sKMYU_JGvneEk161tikGdVIyFe91p5J5buXkfydkkBnaC9tNSKK-At3JmPnIqh2MZhRrrXpb5RGfCTEiUo0FB2r-jPf-EHaWSpSiq-QlnRq8jgdLt6Z91p11qa7ld9ByigNP43X8Fznsfs9It3nNwLsuDxLJ5QdDVMiM&isca=1" type="video/mp4">
Your browser does not support the video tag.
</video>



**Plan and Apply Configuration:**

Generate an execution plan to verify the resources that Terraform will create or modify, and then apply the configuration to set up the cluster:
Expand Down Expand Up @@ -170,6 +177,8 @@ kubens nginx-system
helm upgrade --install nginx-ingress ./deployments/nginx-ingress
```

Please story the Nginx Ingress Controller's IP address, as you'll need it later.

**2. Configure API Key Secret:**

Store your environment variables, such as API keys, securely in Kubernetes secrets. Create a namespace for model serving and create a secret from your `.env` file:
Expand All @@ -192,7 +201,16 @@ kubectl apply -f role.yaml
kubectl apply -f rolebinding.yaml
```

**4. Deploy LiteLLM:**
**4. Deploy caching service using Redis:**

Now, deploy the semantic caching service using Redis:
```bash
cd ./deployments/redis
helm dependency build
helm upgrade --install redis .
```

**5. Deploy LiteLLM:**

Deploy the [LiteLLM](https://github.com/BerriAI/litellm) service:

Expand All @@ -201,7 +219,7 @@ kubens model-serving
helm upgrade --install litellm ./deployments/litellm
```

**5. Deploy the Open WebUI:**
**6. Deploy the Open WebUI:**

Next, Deploy the web UI to your GKE cluster:

Expand All @@ -211,14 +229,9 @@ kubens model-serving
kubectl apply -f ./kubernetes/manifest/base
```

**6. Deploy semantic caching service using Redis:**
**7. Play around with the Application:**

Now, deploy the semantic caching service using Redis:
```bash
cd ./deployments/redis
helm dependency build
helm upgrade --install redis .
```
Open browser and navigate to the URL of your GKE cluster (e.g. `http://172.0.0.0` in step 1) to interact with the application.

### Continuous Integration/Continuous Deployment (CI/CD) with Jenkins and Ansible

Expand Down

0 comments on commit 73cd4d7

Please sign in to comment.