Redis is a popular and opensource in-memory database that supports multiple data structures like strings, hashes, lists, and sets. But similar to other tools, we can scale standalone redis to a particular extent and not beyond that. That’s why we have a cluster mode setup in which we can scale Redis nodes horizontally and then distribute data among those nodes.
Since Kubernetes is becoming buzz technology and people are using it to manage their applications, databases, and middlewares at a single place. So in this blog, we will see how we can deploy the Redis cluster in production mode in the Kubernetes cluster and test failover.
But if you want to deploy the redis cluster on VMs or bare metal, you can use ansible role developed by us for redis cluster/standalone mode.
https://galaxy.ansible.com/opstree_devops/redis
Challenges with Kubernetes
Kubernetes has made the deployment of stateful application quite easy by StatefulSets. By using StatefulSets, we can easily deploy and scale any kind of stateful applications like Kafka, Zookeeper, etc.
But in the case of redis, the setup is not straightforward, there are some additional things which needs to be taken care:-
- We have to use the headless service of Redis because it’s a TCP based service and normal service is HTTP(Layer 7) based Loadbalancer. So in case of headless service, no ClusterIP will be used and we have to rely on Pod IP.
- Redis doesn’t use DNS to form clusters instead of that it uses IP. So we cannot use the internal DNS name of headless service, instead of that, we have to use Pod IP to form Redis cluster.
- In Kubernetes, Pod IP is dynamic and it can change after the pod restart, so in case of the restart the cluster will be malformed and the restarted pod will act as a lost node.
We know what you guys are thinking that life is not that easy with Kubernetes as well but we have tried to make it bearable by designing a solution that can deploy and manage the Redis cluster on Kubernetes within the minutes.
The Solution
As a solution, we have developed our in-house CRD(Custom Resource Definition) to deploy and manage Redis in standalone/cluster mode. So CRD is an amazing feature of Kubernetes which allows us to create our own resources and APIs in Kubernetes. We are not going in the depth of the CRD but soon we will write a blog on CRD as well. Till that time you guys can read about CRD from the official documentation.
The API name which we have created is “redis.opstreelabs.in/v1alpha1” and this operator is also published under the OperatorHub catalog.
https://operatorhub.io/operator/redis-operator
So for deploying the redis-operator and setup we need a Kubernetes cluster 1.11+ and that’s it. Let’s deploy the redis operator first.
# Deploy the CRD API $ kubectl apply -f https://github.com/OT-CONTAINER-KIT/redis-operator/raw/master/deploy/crds/redis.opstreelabs.in_redis_crd.yaml # Create operator serviceaccount $ kubectl apply -f https://github.com/OT-CONTAINER-KIT/redis-operator/raw/master/deploy/service_account.yaml # Create operator role for access $ kubectl apply -f https://github.com/OT-CONTAINER-KIT/redis-operator/raw/master/deploy/role.yaml # Create rolebindings for binding the role and serviceaccount $ kubectl apply -f https://github.com/OT-CONTAINER-KIT/redis-operator/raw/master/deploy/role_binding.yaml # Deploy the operator deployment $ kubectl apply -f https://github.com/OT-CONTAINER-KIT/redis-operator/raw/master/deploy/operator.yaml
Once the operator is deployed and running fine, we can deploy the Redis cluster.
---
apiVersion: redis.opstreelabs.in/v1alpha1
kind: Redis
metadata:
name: redis
spec:
mode: cluster
size: 3
global:
image: opstree/redis:v2.0
imagePullPolicy: IfNotPresent
password: "Opstree@1234"
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 100m
memory: 128Mi
master:
service:
type: ClusterIP
slave:
service:
type: ClusterIP
redisExporter:
enabled: true
image: quay.io/opstree/redis-exporter:1.0
imagePullPolicy: Always
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 100m
memory: 128Mi
storage:
volumeClaimTemplate:
spec:
storageClassName: csi-cephfs-sc
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 1Gi
selector: {}
In the above-defined manifest, we are deploying the Redis 3 node cluster with persistence.
The manifest which are using to deploy redis is explained here
https://ot-container-kit.github.io/redis-operator/#/configuration/configuration
Note:- Make sure you change the default values as per your requirement like Resources, Storage.
# Deploy the cluster of Redis $ kubectl apply -f redis-cluster.yaml
Then validate the Redis cluster is working fine or not.
$ kubectl exec -it redis-master-0 bash $ redis-cli -c -a Opstree@1234 $ cluster nodes
Failover Testing
Before failover testing, we have to write some dummy data inside the Redis cluster, we can write the dummy data using the redis-cli.
$ set tony stark $ get tony
Let’s restart the pod name “redis-master-0” and see the redis node behavior.
$ kubectl delete pod redis-master-0 $ kubectl exec -it redis-master-0 bash $ redis-cli -c -a Opstree@1234 $ cluster nodes
So if you notice the output of cluster nodes command, the node IP is updated and it’s connected as a master. Also, the data of the “tony” key still persists in the redis cluster.
Conclusion
The redis-operator is not only limited to the Redis setup only but it also provides some other features like:-
- Password/Password-less setup
- Cluster/Standalone Redis setup
- Monitoring with redis exporter and Prometheus service discovery annotations
- Security context to manage system parameters
- Affinity and Priority class to manage node failure
- Database and caching mode support
- Secure Redis image
For further information on redis-exporter setup and use cases, you can refer to the documentation page of it.
https://ot-container-kit.github.io/redis-operator/#/
If you face any issue or you want to ask any question, please feel free to use the comment section of this blog. Also, if you have some issues or feature requests regarding Redis Operator you can raise the issue for that in GitHub.
https://github.com/OT-CONTAINER-KIT/redis-operator
Thanks for reading, I’d really appreciate any and all feedback, please leave your comment below if you guys have any feedback.
Cheers till next time!!
Opstree is an End to End DevOps solution provider
How to setup service monitor which should be discovered by prometheus?