ETCD multimaster cluster without kubernetes

[DemianTinkiel]

I'm setting up an ETCD cluster, unrelated to kubernetes. I would like to set it up such that there is 2 active leaders. Is there a way to achieve this? I have the following docker-compose

version: '3.7'

services:
  etcd1:
    hostname: "etcd1"
    networks:
      - nw1
    image: docker.io/bitnami/etcd:3
    environment:
      - ALLOW_NONE_AUTHENTICATION=yes
      - ETCD_NAME=etcd1
      - ETCD_INITIAL_ADVERTISE_PEER_URLS=http://etcd1:2380
      - ETCD_LISTEN_PEER_URLS=http://0.0.0.0:2380
      - ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379
      - ETCD_ADVERTISE_CLIENT_URLS=http://etcd1:2379
      - ETCD_INITIAL_CLUSTER_TOKEN=etcd-cluster
      - ETCD_INITIAL_CLUSTER=etcd1=http://etcd1:2380,etcd2=http://etcd2:2380
      - ETCD_INITIAL_CLUSTER_STATE=new
      - ETCD_ELECTION_TIMEOUT=2000
  etcd2:
    hostname: "etcd2"
    networks:
      - nw1
    image: docker.io/bitnami/etcd:3
    environment:
      - ALLOW_NONE_AUTHENTICATION=yes
      - ETCD_NAME=etcd2
      - ETCD_INITIAL_ADVERTISE_PEER_URLS=http://etcd2:2380
      - ETCD_LISTEN_PEER_URLS=http://0.0.0.0:2380
      - ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379
      - ETCD_ADVERTISE_CLIENT_URLS=http://etcd2:2379
      - ETCD_INITIAL_CLUSTER_TOKEN=etcd-cluster
      - ETCD_INITIAL_CLUSTER=etcd1=http://etcd1:2380,etcd2=http://etcd2:2380
      - ETCD_INITIAL_CLUSTER_STATE=new
      - ETCD_ELECTION_TIMEOUT=2000


networks:
  nw1:
    name: "nw1"

Starting this I see one of the nodes will be elected as leader but disconnecting them from each other via docker network disconnect nw1 etcd2 doesn't make either become their own leader. I tried the same with 6 nodes (with a split in the middle such that it would become 2 clusters of 3), same result.

I tried searching for this but all references seem to point to a multimaster kubernetes cluster. Is there a way to do it without kubernetes?

[SimFG]

Maybe you should map docker's port to local so the two containers can communicate.

[DemianTinkiel]

Maybe you should map docker's port to local so the two containers can communicate.

Thanks for the suggestion but that is not the problem, perhaps I didn't express myself correctly.

I specifically want to create a multi-master cluster and disconnect one instance from the other to see if it can handle split-brain problem after reconnecting the 2.

[SimFG]

If you want to handle this situation, three nodes are required at least.
https://etcd.io/docs/v2.3/admin_guide/#fault-tolerance-table

[DemianTinkiel]

If you want to handle this situation, three nodes are required at least. https://etcd.io/docs/v2.3/admin_guide/#fault-tolerance-table

As I said originally, I had created a cluster of 6 in such a way that upon network break it would yield 2 clusters of 3 nodes each

[SimFG]

[lavacat]

When you do 3:3 split, you don't have 4 nodes needed to establish quorum. If you want to test split brain your limit is 4:2.

If I understand your original question correctly - to make your 3 node clusters work after the split, you need to remove at least 1 disconnected peer from member list. Also your cluster should be started with --strict-reconfig-check=False to allow membership changes when quorum is lost. FYI I've never tested this myself. This isn't a good practice.