Redis refers to REmote DIctionary Server. It is an open source, in-memory Data Structure Store, used as a database, a caching layer or a message broker. Today Redis supports different kinds of abstract data structures, such as strings, lists, maps, sets, sorted sets, HyperLogLog, bitmaps, streams, and spatial indexes. Two week back, Redis 6.0 Release Candidate 1 was made available which comes with the below new features:
- ACLs
- RESP3
- Client side caching
- Threaded I/O
- Diskless replication on replicas
- Redis-benchmark cluster support + Redis-cli improvements
- Systemd support rewrite.
- Redis Cluster proxy was released with Redis 6 (but different repository).
- A Disque module was released with Redis 6 (but different repository).
In my subsequent blog posts, I will try to cover each of the above listed features in detail. Stay tuned ! Luckily, Redis 6 RC1 image is available on Docker Hub and one can access Dockerfile here.
How is Redis different from PostgreSQL or traditional SQL DB?
Redis is an in-memory database because it keeps the whole data set in memory, and answers all queries from memory. As RAM is faster than disks, this means Redis always has very fast reads. The only drawback is that the maximum size of the data set is limited by the available RAM. But nevertheless, Redis has built-in protections allowing the user to set a max limit to memory usage, using the maxmemory option in the configuration file to put a limit to the memory Redis can use. If this limit is reached Redis will start to reply with an error to write commands (but will continue to accept read-only commands), or you can configure it to evict keys when the max memory limit is reached in the case where you are using Redis for caching. Interesting, isn’t it?
Databases like PostgreSQL always keep the whole data set including indices on disk in a format that allows random access. Queries can be answered directly from the on-disk data. The database may load caches or indices into memory as an optimization. A larger difference between Redis and SQL databases is how they deal with writes, i.e. what durability guarantees they provide. There are a lot of tunable parameters here, so it’s not correct to say “an SQL database is always more durable than a Redis database”. However, Redis usually commits data to permanent storage on a periodic basis, whereas Postgres will usually commit before each transaction is marked as complete. This means Postgres is slower because it commits more frequently, but Redis usually has a time window where data loss may occur even when the client was told that their update was handled successfully. This data loss may or may not be an acceptable tradeoff in a given use case.
What’s so cool about key-value store?
Redis is based on the key-value model in which data is stored and fetched from Redis by key. Keybased access allows for extremely efficient access times and this model maps naturally to caching, with Redis providing the customary GET and SET semantics for interacting with the data.
Did you know? Redis can handle up to 232 keys, and was tested in practice to handle at least 250 million keys per instance. Every hash, list, set, and sorted set, can hold 232 elements. In other words your limit is likely the available memory in your system. Nevertheless, several of Redis’ commands operate on multiple keys. Multi-key operations provide better overall performance compared to performing the operations one after the other, because they require substantially less communication and administration.
Scalable shared-nothing clustering
Redis can be scaled horizontally to meet any increase in demand for RAM, computation or network resources. A Redis cluster is a set of processes, possibly on multiple nodes, that work together to provide the caching service. The cluster is made up of multiple Redis servers (i.e. shards), with each one of these being responsible for a subset of the cache’s keyspace. This allows scaling out the cluster simply by adding more shards to it and redistributing the data.
Under this blog post, we will test drive Redis 6.0 Release Candidate 1 for the first time and that too running on 5-Node Docker Swarm environment running on the browser. I will show you how to setup Redis Open Source using a single Docker Stack CLI. We will build a simple Python web application running on Docker Compose. The application uses the Flask framework and maintains a hit counter in Redis. Let’s get started:
Tested Infrastructure
- Play with Docker Environment
- Docker 19.03.4
- Docker Compose 1.23.2, build 1110ad0
- Redis 6.0 Release Candidate 1
To get started with Docker Swarm, you can use “Play with Docker”, aka PWD. It’s free of cost and open for all. You get maximum of 5 instances of Linux system to play around with Docker.
- Open Play with Docker labs on your browser
- Click on Icon near to Instance to choose 3 Managers & 2 Worker Nodes
- Wait for few seconds to bring up 5-Node Swarm Cluster
Clone the Repository
I have created a docker-compose file with the below contents:
version: "3"
services:
web:
# replace username/repo:tag with your name and image details
image: ajeetraina/redis-flask
build:
context: ./stackdemo
dockerfile: Dockerfile
deploy:
replicas: 5
restart_policy:
condition: on-failure
resources:
limits:
cpus: "0.1"
memory: 50M
ports:
- "8000:8000"
networks:
- webnet
visualizer:
image: dockersamples/visualizer:stable
ports:
- "8080:8080"
volumes:
- "/var/run/docker.sock:/var/run/docker.sock"
deploy:
placement:
constraints: [node.role == manager]
networks:
- webnet
redis:
image: redis:6.0-rc1
ports:
- "6379:6379"
volumes:
- data:/home/docker/data
deploy:
placement:
constraints: [node.role == manager]
command: redis-server --appendonly yes
networks:
- webnet
networks:
webnet:
volumes:
data:I have put the above code under Collabnix repository which you can pull and leverage it directly. Do follow the below steps:
git clone https://github.com/collabnix/dockerlabs
cd dockerlabs/solution/redis/viz-web-redis
docker stack deploy -c docker-compose.yml myredis
Verifying the Services
$ docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
ydgp8j56apek myredis_redis replicated 1/1 redis:3.0.6 *:6379->6379/tcp
ofqnb4282zo1 myredis_visualizer replicated 1/1 dockersamples/visualizer:stable *:8080->8080/tcp
bkxd3aklxhj7 myredis_web replicated 5/5 ajeetraina/redis-flask:latest *:8000->8000/tcp
Verifying if Redis is running successfully
$ docker service ps myredis_redis
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
robvimouagqj myredis_redis.1 redis:6.0-rc1 manager1 Running Running about a minute ago
Verifying the Redis Volume
$ docker volume inspect myredis_data
[
{
"CreatedAt": "2019-12-29T02:18:00Z",
"Driver": "local",
"Labels": {
"com.docker.stack.namespace": "myredis"
},
"Mountpoint": "/var/lib/docker/volumes/myredis_data/_data",
"Name": "myredis_data",
"Options": null,
"Scope": "local"
}
]
Checking the Redis logs
[manager1] (local) root@192.168.0.45 ~/dockerlabs/solution/redis/viz-web-redis
$ docker service ps myredis_redis
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
robvimouagqj myredis_redis.1 redis:6.0-rc1 manager1 Running Running about a minute ago
[manager1] (local) root@192.168.0.45 ~/dockerlabs/solution/redis/viz-web-redis
$ docker service ps myredis_redis
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
robvimouagqj myredis_redis.1 redis:6.0-rc1 manager1 Running Running about a minute ago
[manager1] (local) root@192.168.0.45 ~/dockerlabs/solution/redis/viz-web-redis
$ docker service logs -f myredis_redis
myredis_redis.1.robvimouagqj@manager1 | 1:C 29 Dec 2019 02:35:54.400 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
myredis_redis.1.robvimouagqj@manager1 | 1:C 29 Dec 2019 02:35:54.400 # Redis version=5.9.101, bits=64, commit=00000000, modified=0, pid=1, just started
myredis_redis.1.robvimouagqj@manager1 | 1:C 29 Dec 2019 02:35:54.400 # Configuration loaded
myredis_redis.1.robvimouagqj@manager1 | 1:M 29 Dec 2019 02:35:54.402 * Running mode=standalone, port=6379.
myredis_redis.1.robvimouagqj@manager1 | 1:M 29 Dec 2019 02:35:54.402 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
myredis_redis.1.robvimouagqj@manager1 | 1:M 29 Dec 2019 02:35:54.402 # Server initialized
myredis_redis.1.robvimouagqj@manager1 | 1:M 29 Dec 2019 02:35:54.402 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
myredis_redis.1.robvimouagqj@manager1 | 1:M 29 Dec 2019 02:35:54.402 * Ready to accept connectionsWhere is my Redis service running?
$ docker service ps myredis_redis
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
robvimouagqj myredis_redis.1 redis:6.0-rc1 manager1 Running Running 3 minutes ago
Inspecting Redis Service
$ docker service inspect myredis_redis
[
{
"ID": "hmistkdxnirdm5vq2f41aaqr9",
"Version": {
"Index": 127
},
"CreatedAt": "2019-12-29T02:35:47.7810801Z",
"UpdatedAt": "2019-12-29T02:35:47.78773254Z",
"Spec": {
"Name": "myredis_redis",
"Labels": {
"com.docker.stack.image": "redis:6.0-rc1",
"com.docker.stack.namespace": "myredis"
},
"TaskTemplate": {
"ContainerSpec": {
"Image": "redis:6.0-rc1@sha256:c2227b1e5c4755cb94f18eef10b34fb4eac116ce8c5ea0a40d0ca806927b8311",
"Labels": {
"com.docker.stack.namespace": "myredis"
},
"Args": [
"redis-server",
"--appendonly",
"yes"
],
"Privileges": {
"CredentialSpec": null,
"SELinuxContext": null
},
"Mounts": [
{
"Type": "volume",
"Source": "myredis_data",
"Target": "/home/docker/data",
"VolumeOptions": {
"Labels": {
"com.docker.stack.namespace": "myredis"
}
}
}
],
"StopGracePeriod": 10000000000,
"DNSConfig": {},
"Isolation": "default"
},
"Resources": {},
"RestartPolicy": {
"Condition": "any",
"Delay": 5000000000,
"MaxAttempts": 0
},
"Placement": {
"Constraints": [
"node.role == manager"
],
"Platforms": [
{
"Architecture": "amd64",
"OS": "linux"
},
{
"Architecture": "386",
"OS": "linux"
},
{
"Architecture": "ppc64le",
"OS": "linux"
},
{
"Architecture": "s390x",
"OS": "linux"
}
]
},
"Networks": [
{
"Target": "rolenrgn8nqibx2h16wd2tac6",
"Aliases": [
"redis"
]
}
],
"ForceUpdate": 0,
"Runtime": "container"
},
"Mode": {
"Replicated": {
"Replicas": 1
}
},
"UpdateConfig": {
"Parallelism": 1,
"FailureAction": "pause",
"Monitor": 5000000000,
"MaxFailureRatio": 0,
"Order": "stop-first"
},
"RollbackConfig": {
"Parallelism": 1,
"FailureAction": "pause",
"Monitor": 5000000000,
"MaxFailureRatio": 0,
"Order": "stop-first"
},
"EndpointSpec": {
"Mode": "vip",
"Ports": [
{
"Protocol": "tcp",
"TargetPort": 6379,
"PublishedPort": 6379,
"PublishMode": "ingress"
}
]
}
},
"Endpoint": {
"Spec": {
"Mode": "vip",
"Ports": [
{
"Protocol": "tcp",
"TargetPort": 6379,
"PublishedPort": 6379,
"PublishMode": "ingress"
}
]
},
"Ports": [
{
"Protocol": "tcp",
"TargetPort": 6379,
"PublishedPort": 6379,
"PublishMode": "ingress"
}
],
"VirtualIPs": [
{
"NetworkID": "sl1ecujt79razdyhjvmbohhjo",
"Addr": "10.255.0.26/16"
},
{
"NetworkID": "rolenrgn8nqibx2h16wd2tac6",
"Addr": "10.0.1.15/24"
}
]
}
}
]
$ docker exec -it 2db redis-cli --cluster help
Cluster Manager Commands:
create host1:port1 ... hostN:portN
--cluster-replicas <arg>
check host:port
--cluster-search-multiple-owners
info host:port
fix host:port
--cluster-search-multiple-owners
reshard host:port
--cluster-from <arg>
--cluster-to <arg>
--cluster-slots <arg>
--cluster-yes
--cluster-timeout <arg>
--cluster-pipeline <arg>
--cluster-replace
rebalance host:port
--cluster-weight <node1=w1...nodeN=wN>
--cluster-use-empty-masters
--cluster-timeout <arg>
--cluster-simulate
--cluster-pipeline <arg>
--cluster-threshold <arg>
--cluster-replace
add-node new_host:new_port existing_host:existing_port
--cluster-slave
--cluster-master-id <arg>
del-node host:port node_id
call host:port command arg arg .. arg
set-timeout host:port milliseconds
import host:port
--cluster-from <arg>
--cluster-copy
--cluster-replace
backup host:port backup_directory
help
For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.
$ curl localhost:8000
Hello World! I have been seen 6 times.
$ curl localhost:8000
Hello World! I have been seen 7 times.Visualizing Redis using Rebrow
If you are serious about monitoring your cluster health with real-time alerts, analyzing your cluster configuration, rebalance as necessary, managing addition of nodes, re-sharding, node deletion, and master-replica configuration, you must try out RedisInSight. It’s pretty cool and officially supported by Redis Labs. It works quite good if you are using Redis Enterprise.
As we are using Redis Open source, one of the promising tool I came across was Rebrow. It is a Python-Flask-based Browser for Redis Content. It is built for the developer who needs to look into a Redis store. It allows for inspection and deletion of keys and follows PubSub messages. It also displays some runtime and configuration information.
Let’s try to run Rebrow as Docker Swarm service and see if it really works.
$ docker service create --name myrebrow --publish 5001:5001 --replicas 2 marian/rebrow
p9qfx8bfmk7doamfxwy65eicu
overall progress: 2 out of 2 tasks
1/2: running
2/2: running
verify: Service converged
Wow ! It was really fast. Let’s open up web browser to see if it works without any issue. Just supply your Manager IP address and port as 6379. By now, it should connect to Redis server and show up the server status as shown below:
Click on “Keys” to find “hits” with a type “string”.
Once you click on “hits”, it should show you the total number of hits for the webpage.
In my next blog post, I will test drive Redis Open Source on Jetson Nano for the first time. Stay tuned for more exciting stuffs around Redis in near future.
Don’t miss this out !!
Redis Day Bengaluru is happening this January 21st-22nd at Taj Yesvantpur. It is a free, full-day, single-track event about anything and everything Redis. Its main purpose is for Redis users and developers to share technical knowledge and user stories. Check it out.
Comments are closed.