Redis Setup on Kubernetes

Redis is a popular and open-source in-memory database that supports multiple data structures like strings, hashes, lists, and sets. But similar to other tools, we can scale standalone redis to a particular extent and not beyond that. That’s why we have a cluster mode setup in which we can scale Redis nodes horizontally and then distribute data among those nodes.

Since Kubernetes is becoming buzz technology and people are using it to manage their applications, databases, and middlewares in a single place. So in this blog will see how we can deploy the Redis cluster in production mode in the Kubernetes cluster and test failover.

But if you want to deploy the redis cluster on VMs or bare metal, you can use the ansible role developed by us for redis cluster/standalone mode.

Ansible Galaxy

Challenges with Kubernetes

Kubernetes has made the deployment of stateful applications quite easy by StatefulSets. By using StatefulSets, we can easily deploy and scale any kind of stateful application like Kafka, Zookeeper, etc.
But in the case of redis, the setup is not straightforward, some additional things need to be taken care of:-

• We have to use the headless service of Redis because it’s a TCP-based service and the normal service is HTTP(Layer 7) based Loadbalancer. So in the case of headless service, no ClusterIP will be used and we have to rely on Pod IP.
• Redis doesn’t use DNS to form clusters instead of that it uses IP. So we cannot use the internal DNS name of headless service, instead of that, we have to use Pod IP to form a Redis cluster.
• In Kubernetes, Pod IP is dynamic and it can change after the pod restart, so in case of the restart the cluster will be malformed and the restarted pod will act as a lost node.

We know what you guys are thinking that life is not that easy with Kubernetes as well but we have tried to make it bearable by designing a solution that can deploy and manage the Redis cluster on Kubernetes within the minutes.

The Solution

As a solution, we have developed our in-house CRD(Custom Resource Definition) to deploy and manage Redis in standalone/cluster mode. So CRD is an amazing feature of Kubernetes, allowing us to create our own resources and APIs in Kubernetes. We are not going into the depth of the CRD but soon we will write a blog on CRD as well. Till that time you guys can read about CRD from the official documentation.

The API name we have created is “redis.opstreelabs.in/v1alpha1” This operator is also published under the OperatorHub catalog.
https://operatorhub.io/operator/redis-operator

So for deploying the redis-operator and setup we need a Kubernetes cluster 1.11+ and that’s it. Let’s deploy the redis operator first.

# Deploy the CRD API
$ kubectl apply -f https://github.com/OT-CONTAINER-KIT/redis-operator/raw/master/deploy/crds/redis.opstreelabs.in_redis_crd.yaml
# Create operator serviceaccount
$ kubectl apply -f https://github.com/OT-CONTAINER-KIT/redis-operator/raw/master/deploy/service_account.yaml
# Create operator role for access
$ kubectl apply -f https://github.com/OT-CONTAINER-KIT/redis-operator/raw/master/deploy/role.yaml
# Create rolebindings for binding the role and serviceaccount
$ kubectl apply -f https://github.com/OT-CONTAINER-KIT/redis-operator/raw/master/deploy/role_binding.yaml
# Deploy the operator deployment
$ kubectl apply -f https://github.com/OT-CONTAINER-KIT/redis-operator/raw/master/deploy/operator.yaml

In the above-defined manifest, we are deploying the Redis 3 node cluster with persistence.
The manifest which is using to deploy redis is explained here

https://ot-container-kit.github.io/redis-operator/guide/configuration.html

Note:- Make sure you change the default values as per your requirement like Resources, Storage.

# Deploy the cluster of Redis
$ kubectl apply -f redis-cluster.yaml

Then validate the Redis cluster is working fine or not.

$ kubectl exec -it redis-master-0 bash
$ redis-cli -c -a Opstree@1234
$ cluster nodes

Failover Testing

Before failover testing, we have to write some dummy data inside the Redis cluster, we can write the dummy data using the redis-cli.

$ set tony stark
$ get tony

Let’s restart the pod name “redis-master-0” and see the redis node behavior.

$ kubectl delete pod redis-master-0
$ kubectl exec -it redis-master-0 bash
$ redis-cli -c -a Opstree@1234
$ cluster nodes

So if you notice the output of the cluster nodes command, the node IP is updated and it’s connected as a master. Also, the data of the “tony” key persists in the redis cluster.

Conclusion

The redis-operator is not only limited to the Redis setup only but it also provides some other features like:-

• Password/Password-less setup
• Cluster/Standalone Redis setup
• Monitoring with redis exporter and Prometheus service discovery annotations
• Security context to manage system parameters
• Affinity and Priority class to manage node failure
• Database and caching mode support
• Secure Redis image

For further information on redis-exporter setup and use cases, you can refer to the documentation page.
https://ot-container-kit.github.io/redis-operator/#/

If you face any issues or you want to ask any questions, please feel free to use the comment section of this blog. Also, if you have some issues or feature requests regarding Redis Operator you can raise the issue for that in GitHub.
https://github.com/OT-CONTAINER-KIT/redis-operator

Thanks for reading, I’d really appreciate any feedback, please leave your comment below if you guys have any feedback.

Cheers till next time!!