Building 3-Node Active-Active Redis Enterprise Cluster for Developers using Docker Desktop for Mac

Redis Enterprise Software (RS) offers Redis Cluster. RS Cluster is just a set of Redis nodes (OS with Redis installed). It is composed of identical nodes that are deployed within a data center or stretched across local availability zones. Redis cluster is self-managed, so all you have to do is create a database with required options and it abstracts out the pain of worrying about nodes/master/slave from you. Redis enterprise comes up with a fancy UI to interact with Redis Cluster.

Installing, Upgrading and uninstalling RS across those identical nodes is quite an easy process but developing globally distributed applications can be challenging, as developers have to think about race conditions and complex combinations of events under geo-failovers and cross-region write conflicts.

CRDB comes to a rescue..

CRDB refers to “Global Conflict-Free Replicated Database”. It is a new type of Redis Enterprise Software database that spans clusters. There can be one or more member databases across many clusters that form a conflict-free replicated database (CRDB)s. Each local database can have different shard count, replica count, and other database options but contain identical information in steady-state.



CRDB is a globally distributed database that spans multiple Redis Enterprise Software clusters. Each CRDB can have many CRDB Instances (Instances is a CRDB in each cluster that is participating in a global CRDBs) come with added smarts for handling globally distributed writes using the proven CRDT approach. CRDT research describes a set of techniques for creating systems that can handle conflicting writes. CRDBs are powered by Multi-Master Replication (MMR) which provides a straightforward and effective way to replicate your data between regions and simplify development of complex applications that can maintain correctness under geo-failovers and concurrent cross-region writes to the same data.

CRDBs simplify developing such applications by directly using built-in smarts for handling conflicting writes based on the data type in use. Instead of depending on just simplistic “last-writer-wins” type conflict resolution, geo-distributed CRDBs combines techniques defined in CRDT (conflict-free replicated data types) research with Redis types to provide smart and automatic conflict resolution based on the data types intent.

Geo-distributed Active-Active Redis App with CRBD

With Redis Active-Active, you can have a database that can spread across more than one participating clusters, and these clusters can belong to different geo-distributed Data Centers or Availability zones (AZs). For such a database, each cluster has one active master. This active master can read/write to the database.

As we have multiple clusters writing to the DB, there could be conflicts around whose write should win. To avoid that, Redis uses Last Write Wins resolution. (more details: Redis Active-Active). These kind of DBs are referred to as a conflict free replica database (CRDB).The major advantage is that applications hosted in different zones can access (read-write) database locally.

In short,

  • Redis Enterprise in its active-active mode is an ideal database for highly interactive, scalable, low-latency geo-distributed apps.
  • It’s architecture is based on the breakthrough academic research: conflict-free replicated data type (CRDT).
  • The Redis Enterprise implementation of CRDT is called Conflict-Free Replicated Database (CRDB).
  • With CRDBs, applications can read and write to the same data set from different geographical locations seamlessly and with latency less than 1 ms, without changing the way the application connects to the database.
  • Please note that CRDBs do not replicate the entire database, only the data. Database configurations, Lua scripts, and other configurations are not replicated.
  • CRDBs also provide disaster recovery and accelerated data read-access for geographically distributed users.

In my last blog, I showcased how to setup Redis Enterprise software on Play with Docker Platform in 5 Minutes. Under this blog post, I will show how to setup 3-Node Active-Active Redis Enterprise Cluster on top of Docker Desktop for Mac.

Pre-requisite

  • Install Docker Desktop for Mac using this link


  • Redis Enterprise Software Docker image works best when you provide a minimum of 2 cores and 6GB RAM per container. You can find additional minimum hardware and software requirements for Redis Enterprise Software in the product documentation

In case you install a fresh Redis Enterprise software, it consumes 985 MB by default. Hence, you need sufficient memory for your container to work smooth.


As this is just for demonstration, we will leverage 6GB memory. To set up Memory & CPU for Docker Desktop, you will need to click on “whale” icon on top right side of the screen, select “Preference” > Resources > Advanced as shown below:

Click on “Apply & Restart”.

Verifying Docker Engine is up and Running

[Captains-Bay]🚩 >  docker version
Client: Docker Engine - Community
 Version:           19.03.5
 API version:       1.40
 Go version:        go1.12.12
 Git commit:        633a0ea
 Built:             Wed Nov 13 07:22:34 2019
 OS/Arch:           darwin/amd64
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          19.03.5
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.12
  Git commit:       633a0ea
  Built:            Wed Nov 13 07:29:19 2019
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          v1.2.10
  GitCommit:        b34a5c8af56e510852c35414db4c1f4fa6172339
 runc:
  Version:          1.0.0-rc8+dev
  GitCommit:        3e425f80a8c931f88e6d94a8c831b9d5aa481657
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Installing Redis Enterprise Software

Let’s go ahead and create 3 Redis Enterprise Software(RS) containers on Mac. As we need to setup Active-Active RS Enterprise, we will create 3 bridge networks:

[Captains-Bay]🚩 >  cat network-setup.sh 
echo "Script to create bridge networks"
docker network create -d bridge redisnet1  --subnet=172.18.0.0/16 --gateway=172.18.0.1
docker network create -d bridge redisnet2  --subnet=172.19.0.0/16 --gateway=172.19.0.1
docker network create -d bridge redisnet3  --subnet=172.20.0.0/16 --gateway=172.20.0.1


[Captains-Bay]🚩 >  sh network-setup.sh 
Script to create bridge networks
6b47b8256b6408619f7d27d1adc867041239b42140dd7e932de0ea8ba4ccd7c
045fd51912441cd144dd875abf7ae016d35b9bdab895b3d58708e9952cdf4a6
cdc154be0ddaecd3165738b4d5a19d6c86a79d25e987f5d4325939baccefeb9

[Captains-Bay]🚩 >  docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
5dd879a59cac        bridge              bridge              local
06755d527174        host                host                local
9c44c79b0c10        none                null                local
6b47b8256b64        redisnet1           bridge              local
045fd5191244        redisnet2           bridge              local
cdc154be0dda        redisnet3           bridge              local

[Captains-Bay]🚩 >  cat rs-setup.sh 
docker run -d --cap-add sys_resource --network redisnet1 --name redis-node-01 -h redis-node-01  -p 8443:8443 -p 9443:9443 -p 12000:12000 --ip 172.18.0.2 redislabs/redis
docker run -d --cap-add sys_resource --network redisnet2 --name  redis-node-02 -h redis-node-02 -p 8444:8443 -p 9444:9443 -p 12001:12000 --ip 172.19.0.2 redislabs/redis
docker run -d --cap-add sys_resource --network redisnet3 --name  redis-node-03 -h redis-node-03 -p 8445:8443 -p 9445:9443 -p 12002:12000 --ip 172.20.0.2 redislabs/redis


[Captains-Bay]🚩 >  sh rs-setup.sh 
8ae80e7d06587b40eefd62e7c5439aaab60c00cb8e0f0a46b26471511de7839
aa379fe49187ad43bceae55223970d43899af998a0c0edace56749961055763
ddd6c3a7a7b604d554a53dfec079615ef042c009f607b4d81b7a162f978fd86

As you saw that we have chosen 8443,8444 and 8445 ports for the HTTPS connections, 12000, 12001 & 12002 ports for Redis client connections and 9443, 9444 & 9445 for REST API connections.

[Captains-Bay]🚩 >  docker network inspect redisnet1
[
    {
        "Name": "redisnet1",
        "Id": "6b47b8256b6408619f7d27d1adc867041239b42140dd7e932de0ea8ba4ccd7cb",
        "Created": "2020-02-07T06:32:31.222918188Z",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.18.0.0/16",
                    "Gateway": "172.18.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "8ae80e7d06587b40eefd62e7c5439aaab60c00cb8e0f0a46b26471511de78397": {
                "Name": "redis-node-01",
                "EndpointID": "56f7c2f313af5bb1dbc7b2899c17107a7e2b6f60511315a45d40886593d1f050",
                "MacAddress": "02:42:ac:12:00:02",
                "IPv4Address": "172.18.0.2/16",
                "IPv6Address": ""
            }
        },
        "Options": {},
        "Labels": {}
    }
]

Connecting the Networks

[Captains-Bay]🚩 >  cat connect-networks.sh 
echo "connecting networks to containers"

#connecting redisnet1 to node-02 and node-03
docker network connect redisnet1 redis-node-02
docker network connect redisnet1 redis-node-03

#connecting redisnet2 to node-03 & node-01
docker network connect redisnet2 redis-node-01
docker network connect redisnet2 redis-node-03

#connecting redisnet3 to node-01 & node-02
docker network connect redisnet3 redis-node-01
docker network connect redisnet3 redis-node-02


[Captains-Bay]🚩 >  sh connect-networks.sh 
connecting networks to containers


[Captains-Bay]🚩 >  docker network inspect redisnet1
[
    {
        "Name": "redisnet1",
        "Id": "6b47b8256b6408619f7d27d1adc867041239b42140dd7e932de0ea8ba4ccd7cb",
        "Created": "2020-02-07T06:32:31.222918188Z",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.18.0.0/16",
                    "Gateway": "172.18.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "321153d5db7ab4716d0ad4cc4a3d110b688db5eab59b9dd1705724b4f83618d2": {
                "Name": "redis-node-03",
                "EndpointID": "51a9390c011852b7cdc37b21dc606a58d50e2eb6026cfaa92847fc88600118ff",
                "MacAddress": "02:42:ac:12:00:04",
                "IPv4Address": "172.18.0.4/16",
                "IPv6Address": ""
            },
            "8ca5d39cac34bdb997702d7a6ec945a6ddaacda5a3f9df09bdf39efc502210a7": {
                "Name": "redis-node-02",
                "EndpointID": "fa49bda206b95a1a755fe2fe21b3e26f400229636e6b431a2fac0cb9c3f38960",
                "MacAddress": "02:42:ac:12:00:03",
                "IPv4Address": "172.18.0.3/16",
                "IPv6Address": ""
            },
            "93838562e851bf9a794d6abb3cb551a6abefdeaf1b9fa9d1226a9b0aad722635": {
                "Name": "redis-node-01",
                "EndpointID": "e4542306433feec430ffb3904e6f806c033038f186305fe3fdb35a5e957cfd66",
                "MacAddress": "02:42:ac:12:00:02",
                "IPv4Address": "172.18.0.2/16",
                "IPv6Address": ""
            }
        },
        "Options": {},
        "Labels": {}
    }
]

Setting up Redis Enterprise Cluster

[Captains-Bay]🚩 >  cat cluster-setup.sh 
#!/bin/bash

echo “Creating clusters”
docker exec -it redis-node-01 /opt/redislabs/bin/rladmin cluster create name cluster1.collabnix.com username ajeetraina@gmail.com password collab123
docker exec -it redis-node-02 /opt/redislabs/bin/rladmin cluster create name cluster2.collabnix.com username ajeetraina@gmail.com password collab123
docker exec -it redis-node-03 /opt/redislabs/bin/rladmin cluster create name cluster3.collabnix.com username ajeetraina@gmail.com password collab123 

[Captains-Bay]🚩 >  sh cluster-setup.sh 
“Creating clusters”
Creating a new cluster... ok
Creating a new cluster... ok
Creating a new cluster... ok
[Captains-Bay]🚩 >  

[Captains-Bay]🚩 >  cat crdb.sh 


echo “Creating a CRDB”
docker exec -it redis-node-01 /opt/redislabs/bin/crdb-cli crdb create –name mycrdb –memory-size 512mb –port 12000 –replication false –shards-count 1 –instance fqdn=cluster1.collabnix.com,username=ajeetraina@gmail.com,password=collab123 –instance fqdn=cluster2.collabnix.com,username=ajeetraina@gmail.com,password=collab123 –instance fqdn=cluster3.collabnix.com,username=ajeetraina@gmail.com,password=collab123
[Captains-Bay]🚩 >  sh crdb.sh 
“”
“Creating a CRDB”
Task 076be001-384e-4f1d-b446-a6081e6a5bf9 created
  ---> Status changed: queued -> started
  ---> CRDB GUID Assigned: crdb:8e3547c1-00a6-450d-9a5a-eb686ab8844e
redislabs@redis-node-01:~/bin$ rladmin status
CLUSTER NODES:
NODE:ID ROLE   ADDRESS    EXTERNAL_ADDRESS      HOSTNAME      SHARDS CORES FREE_RAM        PROVISIONAL_RAM VERSION   STATUS
*node:1 master 172.18.0.2 172.19.0.3,172.20.0.3 redis-node-01 0/100  2     680.22MB/5.81GB 0KB/4.76GB      5.4.10-22 OK    

DATABASES:
DB:ID    NAME         TYPE   STATUS    SHARDS    PLACEMENT      REPLICATION       PERSISTENCE       ENDPOINT    

ENDPOINTS:
DB:ID                   NAME                ID              NODE                ROLE                SSL         

SHARDS:
DB:ID        NAME       ID           NODE       ROLE       SLOTS        USED_MEMORY                STATUS       
redislabs@redis-node-01:~/bin$ 

Docker Desktop Dashboard

You can click on “Dashboard” under Docker Desktop to check CPU and Memory utilisation.

Cleaning Up

#!/bin/bash

docker rm -fv $(docker ps -a -q)
docker network rm collabnet1 collabnet2 collabnet3

Clap

(3)