Bartłomiej Gałęzowski
Bartłomiej Gałęzowski
Software Engineer

Hassle free Redis Cluster deployment using Docker

May 29, 202316 min read

Distributed Redis Cluster is an efficient solution for applications that require a scalable and reliable data store. Redis Cluster enables automatic data partitioning and replication within the cluster, ensuring high availability and fault tolerance. In this article, we will discuss how to run Redis Cluster using Docker.

We will start by preparing and describing in detail the Docker Compose file with Redis Cluster. To accomplish this, we will utilize the bitnami/redis-cluster image, which will enable us to run six containers (Redis Nodes), with one of them being designated as responsible for creating the cluster.

The cluster consists of nodes categorized as masters and slaves. Each master node can have one or multiple assigned slaves, which essentially act as replicas of the master. In the event of a master node failure, its corresponding slave node automatically takes over and becomes the new master. This mechanism ensures high availability within the cluster.

Composing a multi-container Redis Cluster

x-redis-base: &redis-base
  image: docker.io/bitnami/redis-cluster:7.0.10
  environment:
    ALLOW_EMPTY_PASSWORD: yes
    REDIS_NODES: redis-node-0 redis-node-1 redis-node-2 redis-node-3 redis-node-4 redis-node-5

The anchor contains the fundamental configuration for the Redis node. The presence of ALLOW_EMPTY_PASSWORD permits connecting to the cluster without requiring credentials, which aligns with our requirements.

Note that this article shows setup for development purposes, but we encourage to use authentication everywhere.

Additionally, REDIS_NODES comprises the hostnames of all nodes that will constitute the cluster, separated by spaces. Once we have created the anchor with the basic configuration, we can utilize it to generate five individual nodes.

redis-node-0:
  container_name: redis-node-0
  <<: *redis-base
redis-node-1:
  container_name: redis-node-1
  <<: *redis-base
redis-node-2:
  container_name: redis-node-2
  <<: *redis-base
redis-node-3:
  container_name: redis-node-3
  <<: *redis-base
redis-node-4:
  container_name: redis-node-4
  <<: *redis-base

The last, sixth node (redis-node-5), will have the specific role of establishing the cluster by setting the REDIS_CLUSTER_CREATOR parameter.

redis-node-5:
  container_name: redis-node-5
  <<: *redis-base
  depends_on:
    - redis-node-0
    - redis-node-1
    - redis-node-2
    - redis-node-3
    - redis-node-4
  environment:
    ALLOW_EMPTY_PASSWORD: yes
    REDIS_NODES: redis-node-0 redis-node-1 redis-node-2 redis-node-3 redis-node-4 redis-node-5
    REDIS_CLUSTER_REPLICAS: 1
    REDIS_CLUSTER_CREATOR: yes

I set REDIS_CLUSTER_REPLICAS: 1 what based on six nodes give us three masters and three slaves.

Now our docker-compose file looks like below:

x-redis-base: &redis-base
  image: docker.io/bitnami/redis-cluster:7.0.10
  environment:
    ALLOW_EMPTY_PASSWORD: yes
    REDIS_NODES: redis-node-0 redis-node-1 redis-node-2 redis-node-3 redis-node-4 redis-node-5

services:
  redis-node-0:
    container_name: redis-node-0
    <<: *redis-base
  redis-node-1:
    container_name: redis-node-1
    <<: *redis-base
  redis-node-2:
    container_name: redis-node-2
    <<: *redis-base
  redis-node-3:
    container_name: redis-node-3
    <<: *redis-base
  redis-node-4:
    container_name: redis-node-4
    <<: *redis-base

  redis-node-5:
    container_name: redis-node-5
    <<: *redis-base
    depends_on:
      - redis-node-0
      - redis-node-1
      - redis-node-2
      - redis-node-3
      - redis-node-4
    environment:
      ALLOW_EMPTY_PASSWORD: yes
      REDIS_NODES: redis-node-0 redis-node-1 redis-node-2 redis-node-3 redis-node-4 redis-node-5
      REDIS_CLUSTER_REPLICAS: 1
      REDIS_CLUSTER_CREATOR: yes

Let's execute docker-compose up to run the Redis Cluster.

redis-node-5  | >>> Performing Cluster Check (using node 172.18.0.2:6379)
redis-node-5  | M: 025420b382c7d631ebfe4959b346eaf255caeaba 172.18.0.2:6379
redis-node-5  |    slots:[0-5460] (5461 slots) master
redis-node-5  |    1 additional replica(s)
redis-node-5  | S: 1d693ab3d6b630768359615889bfcdc69642cea3 172.18.0.5:6379
redis-node-5  |    slots: (0 slots) slave
redis-node-5  |    replicates 025420b382c7d631ebfe4959b346eaf255caeaba
redis-node-5  | M: 9239ec8481a3ec411dd0f2f832d0f7a4b07472ef 172.18.0.6:6379
redis-node-5  |    slots:[10923-16383] (5461 slots) master
redis-node-5  |    1 additional replica(s)
redis-node-5  | S: 4d8b42ab42c2f589f31ea2310ccc630a899d71f1 172.18.0.7:6379
redis-node-5  |    slots: (0 slots) slave
redis-node-5  |    replicates 9c5e768c2aaae24e48bb62c0be8395d13592a940
redis-node-5  | S: 07fc9c307fc4fca6371e8832821638d412c8c350 172.18.0.4:6379
redis-node-5  |    slots: (0 slots) slave
redis-node-5  |    replicates 9239ec8481a3ec411dd0f2f832d0f7a4b07472ef
redis-node-5  | M: 9c5e768c2aaae24e48bb62c0be8395d13592a940 172.18.0.3:6379
redis-node-5  |    slots:[5461-10922] (5462 slots) master
redis-node-5  |    1 additional replica(s)
redis-node-5  | [OK] All nodes agree about slots configuration.
redis-node-5  | >>> Check for open slots...
redis-node-5  | >>> Check slots coverage...
redis-node-5  | [OK] All 16384 slots covered.
redis-node-5  | Cluster correctly created

Created!

Need support in Docker platform?

Whether it's integration, containerization or modernization, uninterrupted can handle it efficiently and seamlessly.

Contact Us
A person sitting and typing on a laptop keyboard

Connecting to the Redis Cluster

Next, let's consider a hypothetical scenario where we have an application that needs to connect to our Redis Cluster. This application is also executed within a Docker container so shares the same network. To simulate the application connection we will connect to our cluster using redis-cli from within one of the created containers.

Let’s run docker ps to list all containers.

❯ docker ps
CONTAINER ID   IMAGE                          COMMAND                  CREATED              STATUS              PORTS      NAMES
4a0e6c6ce02a   bitnami/redis-cluster:7.0.10   "/opt/bitnami/script…"   About a minute ago   Up About a minute   6379/tcp   redis-node-5
ff2570e6567c   bitnami/redis-cluster:7.0.10   "/opt/bitnami/script…"   About a minute ago   Up About a minute   6379/tcp   redis-node-2
18b9f0a68e87   bitnami/redis-cluster:7.0.10   "/opt/bitnami/script…"   About a minute ago   Up About a minute   6379/tcp   redis-node-1
2375812e5c93   bitnami/redis-cluster:7.0.10   "/opt/bitnami/script…"   About a minute ago   Up About a minute   6379/tcp   redis-node-0
492f879f178c   bitnami/redis-cluster:7.0.10   "/opt/bitnami/script…"   About a minute ago   Up About a minute   6379/tcp   redis-node-3
77e0549a66a8   bitnami/redis-cluster:7.0.10   "/opt/bitnami/script…"   About a minute ago   Up About a minute   6379/tcp   redis-node-4

And now use docker exec to go into the selected container:

❯ docker exec -it 4a0e6c6ce02a sh

Okay, now we are in and we can run redis-cli

$ redis-cli -c
127.0.0.1:6379>

It looks that everything is functioning properly. However, to verify whether our cluster is correctly configured, let's execute the CLUSTER NODES command to retrieve a list of all nodes within the cluster.

127.0.0.1:6379> CLUSTER NODES
82d2bee961c7a4b73f099550d72a232e55d12d60 172.18.0.2:6379@16379 master - 0 1683707451345 2 connected 5461-10922
e09bd0f03f71d5ca9e1db6804ac28f2470eb70db 172.18.0.7:6379@16379 myself,slave 82d2bee961c7a4b73f099550d72a232e55d12d60 0 1683707449000 2 connected
8cb609962e074807b56f1a01bbbde3e6a436e7ca 172.18.0.3:6379@16379 master - 0 1683707450331 1 connected 0-5460
7dd24d5ffc732189f7b32a5468d9b20ebc1f60d7 172.18.0.5:6379@16379 master - 0 1683707450000 3 connected 10923-16383
c02f2c7285074d9d5b23b8faf07ec287c6e94636 172.18.0.4:6379@16379 slave 8cb609962e074807b56f1a01bbbde3e6a436e7ca 0 1683707449316 1 connected
5105b2780cabbff516e0f834a694bcdca310e8b7 172.18.0.6:6379@16379 slave 7dd24d5ffc732189f7b32a5468d9b20ebc1f60d7 0 1683707448291 3 connected

Based on the provided information, we have a Redis Cluster consisting of three masters and three slaves, with each master having its corresponding slave. The "myself" indicator specifies the node to which we are currently connected.

Now, let's proceed to set a value in the cluster.

127.0.0.1:6379> SET key "value"
-> Redirected to slot [12539] located at 172.18.0.5:6379
OK

Voilà! Thanks to the -c flag, which runs redis-cli in cluster mode, we have been redirected to the master node responsible for slot 12539, allowing us to write our value.

No problemo, but…

Outside Docker Network

Imagine that your hypothetical application is not ready yet, and you would like to expose Redis Cluster outside of the Docker network. This will give you the possibility to develop application without the need to run it in Docker with every change you made. To achieve this, we will make some modifications to our cluster creator container (redis-node-5).

redis-node-5:
  container_name: redis-node-5
  <<: *redis-base
  depends_on:
    - redis-node-0
    - redis-node-1
    - redis-node-2
    - redis-node-3
    - redis-node-4
  ports:
    - 6379:6379
  environment:
    ALLOW_EMPTY_PASSWORD: yes
    REDIS_NODES: redis-node-0 redis-node-1 redis-node-2 redis-node-3 redis-node-4 redis-node-5
    REDIS_CLUSTER_REPLICAS: 1
    REDIS_CLUSTER_CREATOR: yes

We’ve added the "ports" keyword, which maps the local port 6379 to port 6379 inside the Docker network. As a result, we should now be able to connect to the Redis Cluster from the local network (local terminal).

❯ redis-cli -c
127.0.0.1:6379> CLUSTER NODES
4359876c953a0d1577b75203cddde191b2ac69bc 172.20.0.4:6379@16379 slave 5c7b8bd362ff414ac41861ea8cf5758a1d981bcc 0 1683709816000 3 connected
a95eb7d5b68e925bc8adff2ee79cf2384889a5c2 172.20.0.6:6379@16379 master - 0 1683709816687 2 connected 5461-10922
5c7b8bd362ff414ac41861ea8cf5758a1d981bcc 172.20.0.3:6379@16379 master - 0 1683709814651 3 connected 10923-16383
440ec475bf64e1496ba661b3f7ac49a77f2379cc 172.20.0.5:6379@16379 slave ce88b4c234d6e7203ef94c5970cd85ca38c4bce0 0 1683709816000 1 connected
de7b60d66e94a2905afd5d450bd2029690564f74 172.20.0.7:6379@16379 myself,slave a95eb7d5b68e925bc8adff2ee79cf2384889a5c2 0 1683709815000 2 connected
ce88b4c234d6e7203ef94c5970cd85ca38c4bce0 172.20.0.2:6379@16379 master - 0 1683709815665 1 connected 0-5460

And setting a value…

127.0.0.1:6379> SET key "value"
-> Redirected to slot [12539] located at 172.20.0.3:6379
Could not connect to Redis at 172.20.0.3:6379: Operation timed out
(75.00s)

Oops... Redis is unable to connect to the master behind 172.22.0.3:6379. However, if we think about it, the reason becomes apparent. As mentioned earlier, we were attempting to set a key, but we can not do that because we were connected to a slave node (indicated by the "myself" tag). When we asked Redis about the location of "our master," Redis responded with "Okay, you have to connect to 172.20.0.3:6379". However, since we didn't expose every node outside of the Docker network, we received an "Operation timed out" error, we don’t see other nodes. And here begins our problem...

Oh, wait! Are you Linux user? You’re lucky man!

In Linux we are able to use network host mode, in this mode the container shares the network stack of the host, meaning it uses the host's network interfaces directly, so allows us to use localhost.

All we have to do is to add network_mode: host for each node (I’ve added it in anchor), add REDIS_PORT_NUMBER environment for each node and modify REDIS_NODES.

version: "3.7"

x-redis-base: &redis-base
  image: docker.io/bitnami/redis-cluster:7.0.10
  network_mode: host

services:
  redis-node-0:
    container_name: redis-node-0
    <<: *redis-base
    environment:
      REDIS_PORT_NUMBER: 6370
      REDIS_NODES: 127.0.0.1:6370 127.0.0.1:6371 127.0.0.1:6372 127.0.0.1:6373 127.0.0.1:6374 127.0.0.1:6375
      ALLOW_EMPTY_PASSWORD: yes
  redis-node-1: 
    container_name: redis-node-1
    <<: *redis-base
    environment:
      REDIS_PORT_NUMBER: 6371
      REDIS_NODES: 127.0.0.1:6370 127.0.0.1:6371 127.0.0.1:6372 127.0.0.1:6373 127.0.0.1:6374 127.0.0.1:6375
      ALLOW_EMPTY_PASSWORD: yes
  redis-node-2:
    container_name: redis-node-2
    <<: *redis-base
    environment:
      REDIS_PORT_NUMBER: 6372 
      REDIS_NODES: 127.0.0.1:6370 127.0.0.1:6371 127.0.0.1:6372 127.0.0.1:6373 127.0.0.1:6374 127.0.0.1:6375
      ALLOW_EMPTY_PASSWORD: yes
  redis-node-3:
    container_name: redis-node-3
    <<: *redis-base
    environment:
      REDIS_PORT_NUMBER: 6373
      REDIS_NODES: 127.0.0.1:6370 127.0.0.1:6371 127.0.0.1:6372 127.0.0.1:6373 127.0.0.1:6374 127.0.0.1:6375
      ALLOW_EMPTY_PASSWORD: yes
  redis-node-4:
    container_name: redis-node-4
    <<: *redis-base
    environment:
      REDIS_PORT_NUMBER: 6374
      REDIS_NODES: 127.0.0.1:6370 127.0.0.1:6371 127.0.0.1:6372 127.0.0.1:6373 127.0.0.1:6374 127.0.0.1:6375
      ALLOW_EMPTY_PASSWORD: yes
  redis-node-5:
    container_name: redis-node-5
    <<: *redis-base
    depends_on:
      - redis-node-0
      - redis-node-1
      - redis-node-2
      - redis-node-3
      - redis-node-4
    environment:
      REDIS_PORT_NUMBER: 6375
      REDIS_NODES: 127.0.0.1:6370 127.0.0.1:6371 127.0.0.1:6372 127.0.0.1:6373 127.0.0.1:6374 127.0.0.1:6375
      ALLOW_EMPTY_PASSWORD: yes
      REDIS_CLUSTER_REPLICAS: 1
      REDIS_CLUSTER_CREATOR: yes

docker compose up and testing

~$ redis-cli -c -h 127.0.0.1 -p 6375
127.0.0.1:6375> CLUSTER NODES
88d08d5a84f7f9879623111c4b3bae028e465d0a 127.0.1.1:6373@16373 slave a816a5d3484177bb90c86e59d1a93a1e81d4bb2d 0 1683801476781 3 connected
69ab6dbf70298ffe19f388440703f30de177e59f 127.0.1.1:6370@16370 master - 0 1683801475761 1 connected 0-5460
b1ab22f03c0c99f17dce4980944d7180f94c154c 127.0.1.1:6371@16371 master - 0 1683801474743 2 connected 5461-10922
a816a5d3484177bb90c86e59d1a93a1e81d4bb2d 127.0.1.1:6372@16372 master - 0 1683801476000 3 connected 10923-16383
9a79480966b1da9a7f10e5c7304b0c3463e93176 127.0.1.1:6375@16375 myself,slave b1ab22f03c0c99f17dce4980944d7180f94c154c 0 1683801475000 2 connected
ebf22feb960eeaf1a7642deec1910a182e900c6f 127.0.1.1:6374@16374 slave 69ab6dbf70298ffe19f388440703f30de177e59f 0 1683801476000 1 connected

As you can see we have all nodes run on the same host, but on different ports. Let’s try set our test value.

127.0.0.1:6375> SET key "value"
-> Redirected to slot [12539] located at 127.0.1.1:6372
OK

Okay! We’ve redirected from the node running on 127.0.0.1:6375 to 127.0.0.1:6372, and were able to write the value without any issues.

Are you macOS user? Solution is more complex

Unlike Linux, other operating systems, such as macOS, spins-up Linux VM under the hood which makes real host networking unavailable. To resolve this problem, we need to expose each node outside the Docker network on the same host but on different ports. However, determining the exact host to set becomes challenging. We cannot use 127.0.0.1. Instead, we can obtain the external host IP and utilize it in both REDIS_NODES and REDIS_CLUSTER_ANNOUCE_IP configurations to ensure proper communication.

To receive your current external host IP you can use:

echo "$(route get uninterrupted.tech | grep interface | sed -e 's/.*: //' | xargs ipconfig getifaddr)"

My IP is 10.0.18.181 so Docker Compose will looks like below:

version: "3.7"

x-redis-base: &redis-base
  image: docker.io/bitnami/redis-cluster:7.0.10

services:
  redis-node-0:
    container_name: redis-node-0
    <<: *redis-base
    ports:
      - 6370:6370
      - 16370:16370
    environment:
      REDIS_CLUSTER_DYNAMIC_IPS: no
      REDIS_CLUSTER_ANNOUNCE_IP: 10.0.18.181
      REDIS_PORT_NUMBER: 6370
      ALLOW_EMPTY_PASSWORD: yes
      REDIS_NODES: 10.0.18.181:6370 10.0.18.181:6371 10.0.18.181:6372 10.0.18.181:6373 10.0.18.181:6374 10.0.18.181:6375
  redis-node-1:
    container_name: redis-node-1
    <<: *redis-base
    ports:
      - 6371:6371
      - 16371:16371
    environment:
      REDIS_CLUSTER_DYNAMIC_IPS: no
      REDIS_CLUSTER_ANNOUNCE_IP: 10.0.18.181
      REDIS_PORT_NUMBER: 6371
      ALLOW_EMPTY_PASSWORD: yes
      REDIS_NODES: 10.0.18.181:6370 10.0.18.181:6371 10.0.18.181:6372 10.0.18.181:6373 10.0.18.181:6374 10.0.18.181:6375
  redis-node-2:
    container_name: redis-node-2
    <<: *redis-base
    ports:
      - 6372:6372
      - 16372:16372
    environment:
      REDIS_CLUSTER_DYNAMIC_IPS: no
      REDIS_CLUSTER_ANNOUNCE_IP: 10.0.18.181
      REDIS_PORT_NUMBER: 6372
      ALLOW_EMPTY_PASSWORD: yes
      REDIS_NODES: 10.0.18.181:6370 10.0.18.181:6371 10.0.18.181:6372 10.0.18.181:6373 10.0.18.181:6374 10.0.18.181:6375
  redis-node-3:
    container_name: redis-node-3
    <<: *redis-base
    ports:
      - 6373:6373
      - 16373:16373
    environment:
      REDIS_CLUSTER_DYNAMIC_IPS: no
      REDIS_CLUSTER_ANNOUNCE_IP: 10.0.18.181
      REDIS_PORT_NUMBER: 6373
      ALLOW_EMPTY_PASSWORD: yes
      REDIS_NODES: 10.0.18.181:6370 10.0.18.181:6371 10.0.18.181:6372 10.0.18.181:6373 10.0.18.181:6374 10.0.18.181:6375
  redis-node-4:
    container_name: redis-node-4
    <<: *redis-base
    ports:
      - 6374:6374
      - 16374:16374
    environment:
      REDIS_CLUSTER_DYNAMIC_IPS: no
      REDIS_CLUSTER_ANNOUNCE_IP: 10.0.18.181
      REDIS_PORT_NUMBER: 6374
      ALLOW_EMPTY_PASSWORD: yes
      REDIS_NODES: 10.0.18.181:6370 10.0.18.181:6371 10.0.18.181:6372 10.0.18.181:6373 10.0.18.181:6374 10.0.18.181:6375
  redis-node-5:
    container_name: redis-node-5
    <<: *redis-base
    ports:
      - 6375:6375
      - 16375:16375
    environment:
      REDIS_CLUSTER_DYNAMIC_IPS: no
      REDIS_CLUSTER_ANNOUNCE_IP: 10.0.18.181
      REDIS_PORT_NUMBER: 6375
      ALLOW_EMPTY_PASSWORD: yes
      REDIS_NODES: 10.0.18.181:6370 10.0.18.181:6371 10.0.18.181:6372 10.0.18.181:6373 10.0.18.181:6374 10.0.18.181:6375
      REDIS_CLUSTER_REPLICAS: 1
      REDIS_CLUSTER_CREATOR: yes

After modified the REDIS_NODES configuration and added REDIS_CLUSTER_ANNOUCE_IP, we needed to set REDIS_CLUSTER_DYNAMIC_IPS to "no". Additionally, we extended the ports array to include new ports known as CLUSTER BUS PORTS. These ports are utilized for various purposes such as failure detection, configuration updates, failover authorization, and more. By default, the cluster bus port is calculated by adding 10000 to the data port (e.g. 16375).

Let’t connect to Redis Cluster and check cluster nodes:

❯ redis-cli -c -h 127.0.0.1 -p 6375
127.0.0.1:6375> CLUSTER NODES
275c337ffe6073071088bb0706a2536608c03fe9 192.168.1.184:6372@16372 master - 0 1683717016000 3 connected 10923-16383
0e779e30ce3879ee37b470b299a97d41c0e16779 192.168.1.184:6370@16370 master - 0 1683717018000 1 connected 0-5460
3e7dc06681938425f71a39a2a7d189b336132e4a 192.168.1.184:6374@16374 slave 0e779e30ce3879ee37b470b299a97d41c0e16779 0 1683717020706 1 connected
841b05c2890454b345da8380ee63fa41c03fae6b 192.168.1.184:6371@16371 master - 0 1683717020000 2 connected 5461-10922
44444520eb2bbc346ab4dba9b1c495973399a227 192.168.1.184:6375@16375 myself,slave 841b05c2890454b345da8380ee63fa41c03fae6b 0 1683717017000 2 connected
8f14d7d1faac0e131d315ee34c0c7fc3836716c9 192.168.1.184:6373@16373 slave 275c337ffe6073071088bb0706a2536608c03fe9 0 1683717019000 3 connected

Great! Each node has the same IP, different port and different bus port. Final set key attempt…

127.0.0.1:6375> SET key "value"
-> Redirected to slot [12539] located at 192.168.1.184:6372
OK

Excellent, everything works flawlessly, now we can connect to cluster from our app using 127.0.0.1 and any port from 6370 to 6375.

Conclusion

We have covered the process of setting up Redis Cluster inside Docker and exposing it externally. By following the steps outlined, you should now have a solid understanding of how to deploy and manage Redis Cluster within a Docker environment.

Additionally, we briefly touched upon Docker networking and discussed the limitations that can arise when using Docker on macOS. It's important to be aware of these limitations and consider alternative solutions if you encounter any issues specific to the macOS platform.

By exploring these concepts and working with Docker, you have taken a deeper dive into containerization and gained practical knowledge that can be applied to other projects and scenarios.

I hope this article has provided you with a clear understanding of Redis Cluster, Docker networking, and how to leverage these technologies together. Remember to keep experimenting and learning, as continuous improvement is key to mastering any technology.

Useful links

https://redis.io/docs/reference/cluster-spec/

https://redis.io/docs/management/scaling/

https://hub.docker.com/r/bitnami/redis-cluster/

https://docs.docker.com/network/

https://redis.io/docs/ui/cli/

RELATED POSTS
Tomasz Fidecki
Tomasz Fidecki
Managing Director | Technology

Templating Values in Kustomize: Unlocking the Potential of Dynamic Naming for Kubernetes Resources

Mar 13, 20247 min read
Article image
Tomasz Fidecki
Tomasz Fidecki
Managing Director | Technology

Maximizing Efficiency with Dev Containers: A Developer's Guide

Feb 22, 202419 min read
Article image
Bartłomiej Gałęzowski
Bartłomiej Gałęzowski
Software Engineer

Unleashing the power of serverless: a deep dive into web form deployment with our serverless plugin

Oct 12, 202313 min read
Article image