Issue: Configuring Microsoft OIDC with TLS Termination at External LB and DeployKF without SSL (tls.enabled=false)

[ehtishammubarik]

Hello,

I am working on setting up Kubeflow with DeployKF in an infrastructure that involves an external load balancer. We have deployed DeployKF with TLS disabled (tls.enabled=false). Our goal is to integrate Microsoft OIDC for authentication.

The challenge arises with the connector configuration for Microsoft OIDC. Microsoft requires the redirect URI to use HTTPS, but in our current setup, Dex is deployed with HTTP supported URLs. When we attempt to set tls.enabled=true for Dex and OAuth2 Proxy and redeploy, the OAuth2 Proxy encounters a "net/http tls handshake timeout" error. This occurs when trying to access the /dex/.well-known/openid-configuration endpoint, which is publicly available.

Here are the specifics of our setup:

Kubernetes Client Version: v1.28.2
Current Connector Configuration:

connectors: - type: microsoft id: microsoft name: ACESO config: clientID: XXXXX-XXXXX-XXXXXX clientSecret: XXXX~XlkXLkjXLKX redirectURI: "http://deploykf.domain.com/dex/callback" tenant: XXXXX-XXXXX-XXXXXX

I want to use
connectors: - type: microsoft id: microsoft name: ACESO config: clientID: XXXXX-XXXXX-XXXXXX clientSecret: XXXX~XlkXLkjXLKX redirectURI: "https://deploykf.domain.com/dex/callback" tenant: XXXXX-XXXXX-XXXXXX

However, this leads to failures in OAuth2 Proxy as mentioned above. I am seeking advice or suggestions on how to configure this correctly. Has anyone else encountered this issue or can provide pointers on how to resolve it?
Thank you in advance for your help.

[thesuperzapper]

@ehtishammubarik You should try and get your outer LoadBalancer Proxy working when deploykf_core.deploykf_istio_gateway.gateway.tls.enabled is true, rather than disabling TLS at the gateway level (which also tells oauth2-proxy to present itself as HTTP).

First, it is important to understand that deployKF will always redirect HTTP traffic to HTTPS when ...tls.enabled is true, read more about the implications of this in the "TLS Termination" note on this page.

The issue in your case is probably that your outer LoadBalancer Proxy is trying to talk to deployKF over HTTP, and is being redirected to HTTPS which is preventing traffic from reaching the gateway.

For example, if you are using Cloudflare, you would set "SSL/TLS Encryption Mode" to "Full":

---

The explanation of why your custom settings (...tls.enabled to false + modifying the oauth2-proxy to present itself as HTTPS) does not work is that in-mesh pods do NOT use the public internet to access the gateway, and instead talk through the Istio mesh (see the "In-Mesh Traffic to Gateway" note on this page), and when ...tls.enabled is false, the gateway will not be able to respond to HTTPS traffic.