高阶测试人员：Redis-Shake工具研究（上）
因项目组需求，需要更换一套Redis集群，为减少开发工作量，调研了一些Redis数据迁移工具，后决定使用Redis-Shake进行集群数据迁移，前期进行了一些测试。本文主要包括两部分内容：
·Redis集群搭建和模拟数据初始化：为了模拟生产环境的新旧两套Redis集群，首先在本机上进行环境的搭建，并准备迁移所需要的测试数据。
·Redis-Shake测试：测试利用Redis-Shake进行数据迁移。
Redis集群搭建
在本部分，首先搭建一套3主3备的集群作为源集群，然后搭建一套5主5备的集群作为目标集群。为了简便起见，我们采用Docker来简化部署。
配置文件准备
首先，准备配置文件：
port ${PORT}
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
cluster-announce-ip 172.25.253.0
cluster-announce-port ${PORT}
cluster-announce-bus-port 1${PORT}
appendonly yes
requirepass 123456
masterauth 123456其中，port是Redis运行的端口，关闭protected-mode使得集群可以通信而不用配置bind，并配置集群的相关信息。
其中announce-port是命令端口，announce-bus-port是集群桥接端口，其余配置包括开权aof和集群密码，此处为了演示，使用简单密码。
此外，文件中的${PORT}是占位符，随后使用envsubst进行替换，以达到避免重复复制粘贴配置文件的效果。
接下来，我们准备脚本，为每个Redis节点生成一个文件夹，结构如下，其中conf保存配置文件，data保存数据文件：
├─7000
│├─conf
│└─data
#!/bin/zsh
for port in {7000..7005}
do
if [ ! -d "./${port}/conf" ]; then
mkdir -p ./${port}/conf
fi
PORT=${port} envsubst < ./redis-conf.template > ./${port}/conf/redis.conf
if [ ! -d "./${port}/data" ]; then
mkdir -p ./${port}/data
fi
done如上，7000-7005六个端口留作部署3主3备的源Redis集群，8000-8009十个端口留作部署5主5备的目的Redis集群。
准备好每个节点的文件夹后，需要启动Redis节点。此处，借助于docker-compose简化部署。
利用docker简化部署并构建集群
首先，准备源集群的docker-compose文件，如下：
version: "3.4"
x-image:
&default-image
redis:5.0.12
x-restart:
&default-restart
always
x-command:
&default-command
redis-server /etc/redis/redis.conf
services:
redis1-3:
image: *default-image
restart: *default-restart
container_name: redis3-m1
command: *default-command
volumes:
- ./7000/data:/data
- ./7000/conf/redis.conf:/etc/redis/redis.conf
ports:
- 7000:7000
- 17000:17000
networks:
- redisNet
redis2-3:
image: *default-image
restart: *default-restart
container_name: redis3-m2
command: *default-command
volumes:
- ./7001/data:/data
- ./7001/conf/redis.conf:/etc/redis/redis.conf
ports:
- 7001:7001
- 17001:17001
networks:
- redisNet
redis3-3:
image: *default-image
restart: *default-restart
container_name: redis3-m3
command: *default-command
volumes:
- ./7002/data:/data
- ./7002/conf/redis.conf:/etc/redis/redis.conf
ports:
- 7002:7002
- 17002:17002
networks:
- redisNet
redis4-3:
image: *default-image
restart: *default-restart
container_name: redis3-s1
command: *default-command
volumes:
- ./7003/data:/data
- ./7003/conf/redis.conf:/etc/redis/redis.conf
ports:
- 7003:7003
- 17003:17003
networks:
- redisNet
redis5-3:
image: *default-image
restart: *default-restart
container_name: redis3-s2
command: *default-command
volumes:
- ./7004/data:/data
- ./7004/conf/redis.conf:/etc/redis/redis.conf
ports:
- 7004:7004
- 17004:17004
networks:
- redisNet
redis6-3:
image: *default-image
restart: *default-restart
container_name: redis3-s3
command: *default-command
volumes:
- ./7005/data:/data
- ./7005/conf/redis.conf:/etc/redis/redis.conf
ports:
- 7005:7005
- 17005:17005
networks:
- redisNet
networks:
redisNet:
其中，源集群共启动了6个Redis实例，占据7000-7005六个端口，每个实例的配置和数据文件夹都挂载到对应端口下的对应文件夹中。
http://www.51testing.com/attachments/2021/07/15326825_202107021349213KqyF.png
如下，编辑好docker-compose文件后，可以启动源Redis集群所对应的6个Redis实例。
启动了相关的实例后，进入某个容器，进行集群构建。
5.0.12版本已经支持使用redis-cli创建集群，只需要将待构建集群的Redis实例全部作为参数传递，如下所示。
可以看出，首先进行了slot分配，接下来，分配主从节点，并为每个slave匹配相应的master，然后是集群的构建，检查后：
/mnt/c/Users/wangy/redis-cluster ? docker exec -it redis3-m1 /bin/bash
root@b7a3d4d2261d:/data# redis-cli --cluster create 172.25.253.0:7000 172.25.253.0:7001 172.25.253.0:7002 172.25.253.0:7003 172.25.253.0:7004 172.25.253.0:7005 --cluster-replicas 1 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...
Master -> Slots 0 - 5460
Master -> Slots 5461 - 10922
Master -> Slots 10923 - 16383
Adding replica 172.25.253.0:7004 to 172.25.253.0:7000
Adding replica 172.25.253.0:7005 to 172.25.253.0:7001
Adding replica 172.25.253.0:7003 to 172.25.253.0:7002
>>> Trying to optimize slaves allocation for anti-affinity
Some slaves are in the same host as their master
M: 9ac43c00a5a0efd5e35d6019673881e1794a9159 172.25.253.0:7000
slots: (5461 slots) master
M: c9170bddeb0da122efe7b1d36528ad3940341477 172.25.253.0:7001
slots: (5462 slots) master
M: 5bd620cedffe15bf58f2ad86fb5ab4274671e92c 172.25.253.0:7002
slots: (5461 slots) master
S: 50bd4229fe1d93a7d5fa9ed79ba5a7c8d12ad991 172.25.253.0:7003
replicates 5bd620cedffe15bf58f2ad86fb5ab4274671e92c
S: 54e45d4f5047d26bdd52b7e9312f8e60a4c3872b 172.25.253.0:7004
replicates 9ac43c00a5a0efd5e35d6019673881e1794a9159
S: 08fabb337b6a1393788749c904dd15071c29a692 172.25.253.0:7005
replicates c9170bddeb0da122efe7b1d36528ad3940341477
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
..
>>> Performing Cluster Check (using node 172.25.253.0:7000)
M: 9ac43c00a5a0efd5e35d6019673881e1794a9159 172.25.253.0:7000
slots: (5461 slots) master
1 additional replica(s)
S: 54e45d4f5047d26bdd52b7e9312f8e60a4c3872b 172.25.253.0:7004
slots: (0 slots) slave
replicates 9ac43c00a5a0efd5e35d6019673881e1794a9159
M: 5bd620cedffe15bf58f2ad86fb5ab4274671e92c 172.25.253.0:7002
slots: (5461 slots) master
1 additional replica(s)
M: c9170bddeb0da122efe7b1d36528ad3940341477 172.25.253.0:7001
slots: (5462 slots) master
1 additional replica(s)
S: 50bd4229fe1d93a7d5fa9ed79ba5a7c8d12ad991 172.25.253.0:7003
slots: (0 slots) slave
replicates 5bd620cedffe15bf58f2ad86fb5ab4274671e92c
S: 08fabb337b6a1393788749c904dd15071c29a692 172.25.253.0:7005
slots: (0 slots) slave
replicates c9170bddeb0da122efe7b1d36528ad3940341477
All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
All 16384 slots covered.同时，还可以连接到集群，使用cluster info查看集群状态：
root@b7a3d4d2261d:/data# redis-cli -p 7000 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:7000> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:350
cluster_stats_messages_pong_sent:348
cluster_stats_messages_sent:698
cluster_stats_messages_ping_received:343
cluster_stats_messages_pong_received:350
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:698
同理，我们准备一份搭建目的集群的docker-compose文件：
version: "3.4"
x-image:
&default-image
redis:5.0.12
x-restart:
&default-restart
always
x-command:
&default-command
redis-server /etc/redis/redis.conf
services:
redis1-5:
image: *default-image
restart: *default-restart
container_name: redis5-m1
command: *default-command
volumes:
- ./8000/data:/data
- ./8000/conf/redis.conf:/etc/redis/redis.conf
ports:
- 8000:8000
- 18000:18000
networks:
- redisNet
redis2-5:
image: *default-image
restart: *default-restart
container_name: redis5-m2
command: *default-command
volumes:
- ./8001/data:/data
- ./8001/conf/redis.conf:/etc/redis/redis.conf
ports:
- 8001:8001
- 18001:18001
networks:
- redisNet
redis3-5:
image: *default-image
restart: *default-restart
container_name: redis5-m3
command: *default-command
volumes:
- ./8002/data:/data
- ./8002/conf/redis.conf:/etc/redis/redis.conf
ports:
- 8002:8002
- 18002:18002
networks:
- redisNet
redis4-5:
image: *default-image
restart: *default-restart
container_name: redis5-m4
command: *default-command
volumes:
- ./8003/data:/data
- ./8003/conf/redis.conf:/etc/redis/redis.conf
ports:
- 8003:8003
- 18003:18003
networks:
- redisNet
redis5-5:
image: *default-image
restart: *default-restart
container_name: redis5-m5
command: *default-command
volumes:
- ./8004/data:/data
- ./8004/conf/redis.conf:/etc/redis/redis.conf
ports:
- 8004:8004
- 18004:18004
networks:
- redisNet
redis6-5:
image: *default-image
restart: *default-restart
container_name: redis5-s1
command: *default-command
volumes:
- ./8005/data:/data
- ./8005/conf/redis.conf:/etc/redis/redis.conf
ports:
- 8005:8005
- 18005:18005
networks:
- redisNet
redis7-5:
image: *default-image
restart: *default-restart
container_name: redis5-s2
command: *default-command
volumes:
- ./8006/data:/data
- ./8006/conf/redis.conf:/etc/redis/redis.conf
ports:
- 8006:8006
- 18006:18006
networks:
- redisNet
redis8-5:
image: *default-image
restart: *default-restart
container_name: redis5-s3
command: *default-command
volumes:
- ./8007/data:/data
- ./8007/conf/redis.conf:/etc/redis/redis.conf
ports:
- 8007:8007
- 18007:18007
networks:
- redisNet
redis9-5:
image: *default-image
restart: *default-restart
container_name: redis5-s4
command: *default-command
volumes:
- ./8008/data:/data
- ./8008/conf/redis.conf:/etc/redis/redis.conf
ports:
- 8008:8008
- 18008:18008
networks:
- redisNet
redis10-5:
image: *default-image
restart: *default-restart
container_name: redis5-s5
command: *default-command
volumes:
- ./8009/data:/data
- ./8009/conf/redis.conf:/etc/redis/redis.conf
ports:
- 8009:8009
- 18009:18009
networks:
- redisNet
networks:
redisNet:
页:
[1]
