Skip to content

7.CES 租户强隔离

Jump to bottom
myf5 edited this page Jan 28, 2022 · 6 revisions

k8s 租户强隔离策略 Use case

部分CNI，如Kube-ovn支持不同的namespace的网络流量在网络层面完全隔离，甚至不同的namespace可以使用重叠地址网段（类似于不同租户都拥有自己的VPC，并自行设定各自的subnet，这些不同租户的subnet彼此之间可以存在重叠的地址空间）。F5作为出向策略控制器可以支持这样的强租户隔离特性。F5将为不同的namespace创建隔离的配置环境以及隔离的网络流量通信。满足等保要求中对于云环境场景的租户隔离访问控制要求。

tenant mode

策略设定方法

  1. 根据对应的CNI，设定不同k8s namespace的网段。以kube-ovn为例，以下ns-600与ns-900使用了重叠的地址空间:
[root@ovnmaster tenantpolicy]# kubectl get subnets.kubeovn.io -o custom-columns=NAME:metadata.name,SUBNET:spec.cidrBlock,NAMESPACE:spec.namespaces
NAME          SUBNET            NAMESPACE
join          100.64.0.0/16     <none>
ovn-default   192.168.50.0/24   <none>
subnet600     192.168.60.0/24   [ns-600]
subnet700     192.168.70.0/24   [ns-700]
subnet800     192.168.80.0/24   [ns-800]
subnet900     192.168.60.0/24   [ns-900]
  1. 在ns-900中创建要访问的外部服务：
kind: ExternalService
apiVersion: kubeovn.io/v1alpha1
metadata:
   name: ns900-extsvc-f5se-io
   namespace: ns-900
spec:
  addresses:
    - f5se.io
  ports:
    - name: tcp-80
      protocol: TCP
      port: "80"
    - name: tcp-443
      protocol: TCP
      port: "443"
  1. 在ns-900中创建namespace级访问策略规则
apiVersion: kubeovn.io/v1alpha1
kind: NamespaceEgressRule
metadata:
  name: ns900-allow-f5se-io
  namespace: ns-900
spec:
  action: accept-decisively
  externalServices:
    - ns900-extsvc-f5se-io

验证

在当前的集群中， ns-600和 ns-900分别包含了以下规则。在ns-600中容许访问baidu.com，而在ns-900中容许访问f5se.io:

[root@ovnmaster tenantpolicy]# kubectl get namespaceegressrules.kubeovn.io -A
NAMESPACE   NAME                    ACTION              STATUS
ns-600      ns600-allow-baidu-bwc   accept-decisively   Success
ns-900      ns900-allow-f5se-io     accept-decisively   Success

[root@ovnmaster tenantpolicy]# kubectl get externalservices.kubeovn.io  -A
NAMESPACE     NAME                   ADDRESSES
kube-system   across-rd-pods         [192.168.0.0/16]
kube-system   globalextsvcdns        [8.8.8.8 114.114.114.114 223.5.5.5 10.96.0.10]
ns-600        ns600-baidu-bwc        [www.baidu.com]
ns-600        ns600-linjing-io       [linjing.io]
ns-900        ns900-extsvc-f5se-io   [f5se.io]

进入ns-900中的容器，测试访问www.baidu.com以及f5se.io。可以看到f5se.io可以被访问，而baidu.com不能访问：

[root@ovnmaster ~]# kubectl get pod -n ns-900
NAME                         READY   STATUS    RESTARTS   AGE
ns900-app-86d7f68cfc-9fv8b   1/1     Running   0          2d
[root@ovnmaster ~]# kubectl exec -it ns900-app-86d7f68cfc-9fv8b -n ns-900 -- sh

~ # curl -I f5se.io
HTTP/1.1 301 Moved Permanently
Server: GitHub.com
Content-Type: text/html
Location: https://f5se.io/
X-GitHub-Request-Id: 826E:64F2:B4DCA5:BF8C6B:61AC61A0
Content-Length: 162
Accept-Ranges: bytes
Date: Sun, 05 Dec 2021 06:52:17 GMT
Via: 1.1 varnish
Age: 0
Connection: keep-alive
X-Served-By: cache-hnd18723-HND
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1638687137.939326,VS0,VE149
Vary: Accept-Encoding
X-Fastly-Request-ID: 1692f5edac4b437f9607c683404c6917d444cf18


~ # curl -I www.baidu.com
^C

进入ns-600中的容器，测试访问www.baidu.com以及f5se.io。可以看到f5se.io无法访问，而baidu.com可以访问：

[root@ovnmaster ~]# kubectl get pod -n ns-600
NAME                     READY   STATUS    RESTARTS   AGE
myapp-648bc84478-rk4l9   1/1     Running   1          3d22h
tmp-shell-ns600          1/1     Running   2          181d
[root@ovnmaster ~]# kubectl exec  -it tmp-shell-ns600 -n ns-600 -- sh

~ # curl -I f5se.io
^C


~ # curl -I www.baidu.com
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
Connection: keep-alive
Content-Length: 277
Content-Type: text/html
Date: Sun, 05 Dec 2021 06:55:16 GMT
Etag: "575e1f59-115"
Last-Modified: Mon, 13 Jun 2016 02:50:01 GMT
Pragma: no-cache
Server: bfe/1.0.8.18

附加测试：

在ns-900中，发布service级别的egress策略规则，容许服务ns900-app-svc 访问baidu.com:

kind: ExternalService
apiVersion: kubeovn.io/v1alpha1
metadata:
   name: ns900-extsvc-baidu-com
   namespace: ns-900
spec:
  addresses:
    - baidu.com
  ports:
    - name: tcp-80
      protocol: TCP
      port: "80"
    - name: tcp-443
      protocol: TCP
      port: "443"

apiVersion: kubeovn.io/v1alpha1
kind: ServiceEgressRule
metadata:
  name: ns900-allow-baidu-com
  namespace: ns-900
spec:
  service: ns900-app-svc
  action: accept-decisively
  externalServices:
    - ns900-extsvc-baidu-com

验证：

[root@ovnmaster tenantpolicy]# kubectl get ep -n ns-900
NAME            ENDPOINTS         AGE
ns900-app-svc   192.168.60.1:80   8m5s

[root@ovnmaster ~]# kubectl exec -it ns900-app-86d7f68cfc-9fv8b -n ns-900 -- sh
~ # curl -I baidu.com
HTTP/1.1 200 OK
Date: Sun, 05 Dec 2021 07:19:52 GMT
Server: Apache
Last-Modified: Tue, 12 Jan 2010 13:48:00 GMT
ETag: "51-47cf7e6ee8400"
Accept-Ranges: bytes
Content-Length: 81
Cache-Control: max-age=86400
Expires: Mon, 06 Dec 2021 07:19:52 GMT
Connection: Keep-Alive
Content-Type: text/html

下一步

F5 AFM高可用场景

Clone this wiki locally