Skip to content

Commit

Permalink
test
Browse files Browse the repository at this point in the history
  • Loading branch information
bmd1905 committed Aug 27, 2024
1 parent b4bfbeb commit ea07bb3
Showing 1 changed file with 24 additions and 9 deletions.
33 changes: 24 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,15 @@ cd iac/terraform
terraform init
```

![Hehe](https://ff34a2b462fd9b6d77e2f1cfb2be0ca8287a65fe5558cdc17f83f20-apidata.googleusercontent.com/download/storage/v1/b/promptalchemy/o/videos%2F1-init-cluster.gif?jk=AXZI6JzsCiLtpo_lO2Y5yZ4quUgevRgK-J2l4qE6xjV5TiN4UrC07jdp23uuVSBETi8L_tTT74Pmew1nrkofpUm-DxTsMan5r1wVYlM4pZLrbmH87gfjJ0MNBL50hXxV2FIDVMDvI2GJ1X8m3wbZaHEdwUZdPjHGvGE6uThWC4cNLyGKV3fpsIZNsosfw-UzaODehVDjF3bBrHQ0O2pB1zZ3R0vDn_1Y3arNZiTeva34lVAclWRgHeEVozJsSW8k-ZrqcSZNX4xibqQykKT1ZaPhTp5SFbattQwvhA6-gw2WevHaZ14FDBNmgaOadu7RPfBjmumrhMhg3GlIGfupoSMhAoneEwhpig2TDG5bOVYvekbucSpAaFi3GHhXmiJBxHY3lVFv6aWwXRDT_rM8wjFFDiR8sX9NAomogxNdZ_EpySPfIX0Y8tOen6BUc9MdGIVjHVXDLIbQlp0teQRuJ3go2TdLWDi4dIqAU3sqt94kDF274qYRgahBSHB64dyLI32ajT4gsWeFdHLrR4c3H8F8XcSqxkY-4ZvEa39PirkQf_2KA_JpBp9qrMDqvU5YY1xhWidDezLOI7-PjcM5DvXwjCPDGIsQIhYnOBD7h5eK8YAicLhWVejCJqSbJZtRv1l7vHwNGu2x6s1m6biZKyLTq3r3YkcpoBDn6JbGYyUa-mBMrr9E3KNubWxyOxvhIemCI9U5dSCtbTDm7icVdWCW8qrKzW7g3UeRwd8C9YE2phr63DyfRuQn4ZDco5BIsCeZJfVjg4_wyDcyhLpkZEFSAdh4Y6Osh5U-w1Hagn1XehmUQbJvL_x3AkZIsAoekzbf1E1eOXuy5_mcsVh5X9OVVdAHV9Kc_KNcaoAIUrBfbQ0Zqb88gILeQ3RHQQGRbrrKSU1iTlgP6LkKoKH22sy6TYgz2ah_tk1wKqpr6jp12n1D32Kd2BFBnTtUVBs3tkPNEWyuUU-pa0Jj5VByT1zb60GtqrLafwVmQT54KJ5uMzS5QbLVin0boElx3Iddu63gLabPmv-MkDTpuHUKdWRP8ZWO09Swx03IMLvJhqmvdFByAHwu5lMhYtxZIN3QL2vJ0EeiJK8Jgxupjew3DCWGCkq9hVW82-IjT9VQ92-xeZtR7qWc3Y9vYfbJMXN-uOZf5TqamIuG4EaT4FrhX4I2g7oJ9VxybZWqKzpuWgyzeU2rtrM3EWHmBRGZxlJNprU-lp53Zb_VJD1e4OXOhi4KMzSdzVKVRPwX_O4rcqahUtkz9iinY94L5zNyilEm-L7CfiWtUbtROuHa8OEnjDg&isca=1)

<video controls>
<source src="https://ffc2f9163061eaa84c660f92e3b06dcbbe78d37a01072cf1f4b3451-apidata.googleusercontent.com/download/storage/v1/b/promptalchemy/o/videos%2F1-init-cluster.mp4?jk=AXZI6Jy1boMVifNNgQdvnGsz9UxZF8HMIql1daLmJqtccxGPKHcowL3x3f7QUHNR0ARJ0txYpGyBYy29j2R78HQp8IMZ3--hFsfmXXb6WQc8rW2z2RwRiaTgrRfXtBPuc4ZdqwEM4OmlZf7CwGELNwM64JABNHPTDjUTN00ZVWyMOXxBZ-ViE6RVrqr6aQi4955tB01ECQ6WlxJV7ApC9abQG8vqD2zUKSfSfpppwfU9MSvrGjyTNEupkjTjjVC76G9-11hKHBzRgs_kW8kswGGeC3lNcd-xbWxe-L16FGWElfpGuNCqbKOC2UHGxd7y9VPrho3sSQsGEMoMP4tJVi5sWpQ_xQzJ5tF58Xba1OGLN6P1QPo6Q79-URTNRmOGJTo9ltwwBWwsooR1BO7LLEhmT8-TvboMZDrTfKOYjeOTg3_PWVeWrfnyzF1qGj0rY8_zuxy7wj-SXd7qnzX3FXJ8giMxjUgXS3EIUTVy2CO3pHvff_-ifH2D8rm_yCxsCz7qgChpH8N4oDiWAgOJDrFXKKAtvH2Gjoi7tC7DLKAC5DDkICLScGRVEr3nAdIF9kDjpZe-Utd28bd_R7K4bJA4KV-IgUOhBz8wgxoi4D_dkid6zg3O9f59pPPbN5-ujya2S-zjtKz-v_OVhUj3rk1GwuZaO0_LdMGoP8hbe0Sv7ErVRHrbKfw4OmloH_HB4KGFu_27tvUqyANKVl4vQCcxvHOPImLd0qwdnFCTLPilxhDAXYAdLgFBs3KT2NG_zyQ_PbVG5h81A3Ne3L_Zg5R7eFUwksjlgQNSTICyePCuoe-yYC8TuqfgUMPo1OdX-qwyvoK0xN7GMCeFc2e5k5TB4LUEanUh5i9Colzosu31-FBsmQ6Xswiv0I5X1TP0b-GZgIgI3Wb3QNtXp_3UczRut_y_CLtRMtnEK4yYW_nybZ4jEaWvC50-tBBOHAATQAjvFHyhxGtU8G_RSdAmHp0_ISaxS3nnYwqwOAOfyII14bdbJwfQi1gHuQKlY6dPQjyvfmTaUyfdkcUJr-PzbojenmM7981m1UOPMpoUSSvEKtaceQXS10Bs06TA8pPdZWr-CXD3rVcG4gjxHHWFSgLABQTO3Px8IQN2x5AV9vSI-H_An03sKMYU_JGvneEk161tikGdVIyFe91p5J5buXkfydkkBnaC9tNSKK-At3JmPnIqh2MZhRrrXpb5RGfCTEiUo0FB2r-jPf-EHaWSpSiq-QlnRq8jgdLt6Z91p11qa7ld9ByigNP43X8Fznsfs9It3nNwLsuDxLJ5QdDVMiM&isca=1" type="video/mp4">
Your browser does not support the video tag.
</video>



**Plan and Apply Configuration:**

Generate an execution plan to verify the resources that Terraform will create or modify, and then apply the configuration to set up the cluster:
Expand Down Expand Up @@ -170,6 +179,8 @@ kubens nginx-system
helm upgrade --install nginx-ingress ./deployments/nginx-ingress
```

Please story the Nginx Ingress Controller's IP address, as you'll need it later.

**2. Configure API Key Secret:**

Store your environment variables, such as API keys, securely in Kubernetes secrets. Create a namespace for model serving and create a secret from your `.env` file:
Expand All @@ -192,7 +203,16 @@ kubectl apply -f role.yaml
kubectl apply -f rolebinding.yaml
```

**4. Deploy LiteLLM:**
**4. Deploy caching service using Redis:**

Now, deploy the semantic caching service using Redis:
```bash
cd ./deployments/redis
helm dependency build
helm upgrade --install redis .
```

**5. Deploy LiteLLM:**

Deploy the [LiteLLM](https://github.com/BerriAI/litellm) service:

Expand All @@ -201,7 +221,7 @@ kubens model-serving
helm upgrade --install litellm ./deployments/litellm
```

**5. Deploy the Open WebUI:**
**6. Deploy the Open WebUI:**

Next, Deploy the web UI to your GKE cluster:

Expand All @@ -211,14 +231,9 @@ kubens model-serving
kubectl apply -f ./kubernetes/manifest/base
```

**6. Deploy semantic caching service using Redis:**
**7. Play around with the Application:**

Now, deploy the semantic caching service using Redis:
```bash
cd ./deployments/redis
helm dependency build
helm upgrade --install redis .
```
Open browser and navigate to the URL of your GKE cluster (e.g. `http://172.0.0.0` in step 1) to interact with the application.

### Continuous Integration/Continuous Deployment (CI/CD) with Jenkins and Ansible

Expand Down

0 comments on commit ea07bb3

Please sign in to comment.