How to build a sharding-proxy cluster？

[marker-wu]

How to build a sharding-proxy cluster？

[Crackersssss]

I was thinking about that, too.
My simple solution is to make ZooKeeper the configuration center and registry. LVS perform load balancing. It works ok. But I don't know if it really meets the clustering requirements

[Crackersssss]

This is my picture:

[marker-wu]

This is my picture:

Haha, I have the same idea as you. I tried to use Nginx to proxy proxy last night, but it didn’t work. I don’t know if there is a problem with my configuration.
Have you tried successfully?

[Crackersssss]

This is my picture:

Haha, I have the same idea as you. I tried to use Nginx to proxy proxy last night, but it didn’t work. I don’t know if there is a problem with my configuration.
Have you tried successfully?

My virtual IP does connect to all of my proxies, and it's polling

[marker-wu]

This is my picture:

Haha, I have the same idea as you. I tried to use Nginx to proxy proxy last night, but it didn’t work. I don’t know if there is a problem with my configuration.
Have you tried successfully?

My virtual IP does connect to all of my proxies, and it's polling

Can you share your Nginx configuration？

[Crackersssss]

This is my picture:

Haha, I have the same idea as you. I tried to use Nginx to proxy proxy last night, but it didn’t work. I don’t know if there is a problem with my configuration.
Have you tried successfully?

My virtual IP does connect to all of my proxies, and it's polling

Can you share your Nginx configuration？

Sorry, I'm using LVS.

[Crackersssss]

I can give you the steps of LVS if you want to use LVS too

[marker-wu]

I can give you the steps of LVS if you want to use LVS too

OK, let me see

[Crackersssss]

如果你也想用LVS我可以给你LVS的步骤

好的，让我看看

LB Server Configuration:

ifconfig network_card_name vip/mask up
ipvsadm -C
ipvsadm --set 30 5 60
ipvsadm -A -t vip:port -s rr -p 60
ipvsadm -a -t vip:port -r proxy_ip:port -g -w 1
#All proxy ip must be matched
ipvsadm -ln
ipvsadm -S > /etc/sysconfig/ipvsadm
systemctl restart ipvsadm

Web Server Configuration:
Disable ARP packet broadcasting:

cat >>/etc/sysctl.conf<<EOF 
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2 EOF
sysctl -p

Bind virtual IP addresses, add static routes, receive packets from the LB server, and forward the packets to the LO network adapter for processing:

ip addr add vip/mask dev lo 
route add -host proxy_ip dev lo

[marker-wu]

如果你也想用LVS我可以给你LVS的步骤

好的，让我看看

LB Server Configuration:

ifconfig network_card_name vip/mask up
ipvsadm -C
ipvsadm --set 30 5 60
ipvsadm -A -t vip:port -s rr -p 60
ipvsadm -a -t vip:port -r proxy_ip:port -g -w 1
#All proxy ip must be matched
ipvsadm -ln
ipvsadm -S > /etc/sysconfig/ipvsadm
systemctl restart ipvsadm

Web Server Configuration:
Disable ARP packet broadcasting:

cat >>/etc/sysctl.conf<<EOF 
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2 EOF
sysctl -p

Bind virtual IP addresses, add static routes, receive packets from the LB server, and forward the packets to the LO network adapter for processing:

ip addr add vip/mask dev lo 
route add -host proxy_ip dev lo

Haha, thank you, I built one with Haproxy and it worked！

[Crackersssss]

如果你也想用LVS我可以给你LVS的步骤

好的，让我看看

LB Server Configuration:

ifconfig network_card_name vip/mask up
ipvsadm -C
ipvsadm --set 30 5 60
ipvsadm -A -t vip:port -s rr -p 60
ipvsadm -a -t vip:port -r proxy_ip:port -g -w 1
#All proxy ip must be matched
ipvsadm -ln
ipvsadm -S > /etc/sysconfig/ipvsadm
systemctl restart ipvsadm

Web Server Configuration:
Disable ARP packet broadcasting:

cat >>/etc/sysctl.conf<<EOF 
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2 EOF
sysctl -p

Bind virtual IP addresses, add static routes, receive packets from the LB server, and forward the packets to the LO network adapter for processing:

ip addr add vip/mask dev lo 
route add -host proxy_ip dev lo

Haha, thank you, I built one with Haproxy and it worked！

congratulations

[tristaZero]

@marker-wu @crackersuperior Hi, everyone, thanks for your discussion, especially @crackersuperior 's diagram, looks nice.
I'd like to move this issue to the discussion section to make more people know about your talk. ;-)

[zhfeng]

Well done ! It definitely need to write a great article with Building a high available sharding-proxy with HAProxy or LVS.

[roodkcab]

I'd like to share our use case of ss as sidecar in Kubernetes.

Inject ShardingSphere as side car container to app pod, and let app connect to 127.0.0.1:3306 directly.
All ShardingSphere container consist of a cluster and listen to etcd metadata change.
We setup another ShardingSphere pod in db namespace to control metadata like schema, encrypt column and cipher, etc.

[tristaZero]

Looks cool!