Skip to content

EN_7_tenants_strict_Isolation_policy

Jump to bottom
myf5 edited this page Jan 29, 2022 · 3 revisions

k8s Tenant Strong Isolation Strategy Use case

Some CNIs, such as Kube-ovn, support different namespaces to be completely isolated at the network level, and even different namespaces can use overlapping CIDR (similar to different tenants having their own VPC and setting their own subnets, these subnets of different tenants can have overlapping address spaces with each other). F5 as an egress policy controller, support such a strict tenant isolation feature. F5 will create isolated configuration environments and isolated network traffic for different namespaces.

tenant mode

Policy setting

  1. According to the different CNI, set the network segment for different k8s namespace. Here, kube-ovn an example, the following ns-600and ns-900 have overlapping address segment192.168.60/24:
[root@ovnmaster tenantpolicy]# kubectl get subnets.kubeovn.io -o custom-columns=NAME:metadata.name,SUBNET:spec.cidrBlock,NAMESPACE:spec.namespaces
NAME          SUBNET            NAMESPACE
join          100.64.0.0/16     <none>
ovn-default   192.168.50.0/24   <none>
subnet600     192.168.60.0/24   [ns-600]
subnet700     192.168.70.0/24   [ns-700]
subnet800     192.168.80.0/24   [ns-800]
subnet900     192.168.60.0/24   [ns-900]
  1. In ns-900creating an external service to be accessed:
kind: ExternalService
apiVersion: kubeovn.io/v1alpha1
metadata:
   name: ns900-extsvc-f5se-io
   namespace: ns-900
spec:
  addresses:
    - f5se.io
  ports:
    - name: tcp-80
      protocol: TCP
      port: "80"
    - name: tcp-443
      protocol: TCP
      port: "443"
  1. In the ns-900creating namespace-level egress rule:
apiVersion: kubeovn.io/v1alpha1
kind: NamespaceEgressRule
metadata:
  name: ns900-allow-f5se-io
  namespace: ns-900
spec:
  action: accept-decisively
  externalServices:
    - ns900-extsvc-f5se-io

Verify

In the current cluster, ns-600and ns-900each contains the following rules. In ns-600the allowed access is baidu.com, and in the ns-900allowed access is f5se.io:

[root@ovnmaster tenantpolicy]# kubectl get namespaceegressrules.kubeovn.io -A
NAMESPACE   NAME                    ACTION              STATUS
ns-600      ns600-allow-baidu-bwc   accept-decisively   Success
ns-900      ns900-allow-f5se-io     accept-decisively   Success
[root@ovnmaster tenantpolicy]# kubectl get externalservices.kubeovn.io  -A
NAMESPACE     NAME                   ADDRESSES
kube-system   across-rd-pods         [192.168.0.0/16]
kube-system   globalextsvcdns        [8.8.8.8 114.114.114.114 223.5.5.5 10.96.0.10]
ns-600        ns600-baidu-bwc        [www.baidu.com]
ns-600        ns600-linjing-io       [linjing.io]
ns-900        ns900-extsvc-f5se-io   [f5se.io]

Enter ns-900the container, test www.baidu.com and f5se.io. You can see that f5se.io can be accessed, but baidu.com cannot be accessed:

[root@ovnmaster ~]# kubectl get pod -n ns-900
NAME                         READY   STATUS    RESTARTS   AGE
ns900-app-86d7f68cfc-9fv8b   1/1     Running   0          2d
[root@ovnmaster ~]# kubectl exec -it ns900-app-86d7f68cfc-9fv8b -n ns-900 -- sh

~ # curl -I f5se.io
HTTP/1.1 301 Moved Permanently
Server: GitHub.com
Content-Type: text/html
Location: https://f5se.io/
X-GitHub-Request-Id: 826E:64F2:B4DCA5:BF8C6B:61AC61A0
Content-Length: 162
Accept-Ranges: bytes
Date: Sun, 05 Dec 2021 06:52:17 GMT
Via: 1.1 varnish
Age: 0
Connection: keep-alive
X-Served-By: cache-hnd18723-HND
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1638687137.939326,VS0,VE149
Vary: Accept-Encoding
X-Fastly-Request-ID: 1692f5edac4b437f9607c683404c6917d444cf18


~ # curl -I www.baidu.com
^C

Enter ns-600the container, test www.baidu.com and f5se.io. It can be seen that f5se.io cannot be accessed, but baidu.com can be accessed:

[root@ovnmaster ~]# kubectl get pod -n ns-600
NAME                     READY   STATUS    RESTARTS   AGE
myapp-648bc84478-rk4l9   1/1     Running   1          3d22h
tmp-shell-ns600          1/1     Running   2          181d
[root@ovnmaster ~]# kubectl exec  -it tmp-shell-ns600 -n ns-600 -- sh

~ # curl -I f5se.io
^C


~ # curl -I www.baidu.com
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
Connection: keep-alive
Content-Length: 277
Content-Type: text/html
Date: Sun, 05 Dec 2021 06:55:16 GMT
Etag: "575e1f59-115"
Last-Modified: Mon, 13 Jun 2016 02:50:01 GMT
Pragma: no-cache
Server: bfe/1.0.8.18

Additional test:

In ns-900, publish service-level egress rules to allow service ns900-app-svcaccess baidu.com:

kind: ExternalService
apiVersion: kubeovn.io/v1alpha1
metadata:
   name: ns900-extsvc-baidu-com
   namespace: ns-900
spec:
  addresses:
    - baidu.com
  ports:
    - name: tcp-80
      protocol: TCP
      port: "80"
    - name: tcp-443
      protocol: TCP
      port: "443"
apiVersion: kubeovn.io/v1alpha1
kind: ServiceEgressRule
metadata:
  name: ns900-allow-baidu-com
  namespace: ns-900
spec:
  service: ns900-app-svc
  action: accept-decisively
  externalServices:
    - ns900-extsvc-baidu-com

verify:

[root@ovnmaster tenantpolicy]# kubectl get ep -n ns-900
NAME            ENDPOINTS         AGE
ns900-app-svc   192.168.60.1:80   8m5s
[root@ovnmaster ~]# kubectl exec -it ns900-app-86d7f68cfc-9fv8b -n ns-900 -- sh
~ # curl -I baidu.com
HTTP/1.1 200 OK
Date: Sun, 05 Dec 2021 07:19:52 GMT
Server: Apache
Last-Modified: Tue, 12 Jan 2010 13:48:00 GMT
ETag: "51-47cf7e6ee8400"
Accept-Ranges: bytes
Content-Length: 81
Cache-Control: max-age=86400
Expires: Mon, 06 Dec 2021 07:19:52 GMT
Connection: Keep-Alive
Content-Type: text/html

Next step

BIG-IP HA support

Clone this wiki locally