前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >Redis6.0集群环境搭建

Redis6.0集群环境搭建

原创
作者头像
用户2141488
修改2021-02-10 12:10:27
2.6K0
修改2021-02-10 12:10:27
举报

Redis集群(Redis Cluster) 是Redis提供的分布式数据库方案,通过 分片(sharding) 来进行数据共享,并提供复制和故障转移功能。相比于主从复制、哨兵模式,Redis集群实现了较为完善的高可用方案,解决了存储能力受到单机限制,写操作无法负载均衡的问题。

本文是Redis集群学习的实践总结,详细介绍逐步搭建Redis集群环境的过程,并完成集群伸缩的实践。

测试环境

  • centos8.3
  • redis 6.0.6

redis安装

cat redis_install.sh

代码语言:javascript
复制
#!/bin/bash

yum update -y
cd /usr/local/src/
wget http://download.redis.io/releases/redis-6.0.6.tar.gz
tar zxvf redis-6.0.6.tar.gz
cd redis-6.0.6/
make
make install
ln -s /usr/local/src/redis-6.0.6/src/redis-server /usr/bin/redis-server
ln -s /usr/local/src/redis-6.0.6/src/redis-cli /usr/bin/redis-cli

集群搭建

因是测试,这里集群环境的所有节点全部位于同一个服务器上,共6个节点以端口号区分,3个主节点+3个从节点(分别是6379、6479、6579、6679、6779、6879)。

注意:从Redis 5.0以后的版本,集群管理软件 redis-trib.rb 被集成到 redis-cli 客户端工具中。

集群的搭建可以分为四步:

  1. 启动节点:将节点以集群方式启动(此时节点是独立的)
  2. 节点握手:将独立的节点连成网络
  3. 槽(solts)分配:将16384个槽位分配给主节点,以达到分片保存数据库键值对的效果。
  4. 主从复制:为从节点指定主节点。

每个节点初始状态仍为 master服务器,唯一不同的是:使用 Cluster 模式启动。需要对配置文件进行修改,以端口号为6379的节点为例,主要修改如下几项:

  • port 6379 # 启动端口
  • cluster-enabled yes # 开启集群
  • pidfile /var/run/redis_6379.pid # pid存储目录
  • cluster-config-file "node-6379.conf" # 集群节点配置文件,这个文件是不能手动编辑的.确保每一个集群节点的配置文件不通
  • logfile "redis-server-6379.log" # 日志存储目录
  • dbfilename "dump-6379.rdb" # 数据存储
  • daemonize yes # 守护进程
  • appendonly yes # 持久化

为了方便,可以使用`sed`进行批量替换,下面的批量部署脚本可以参考下

代码语言:javascript
复制
#!/bin/bash


# 安装redis6.0
if [[ `redis-server -v` == "Redis server"* ]]; then
    echo "redis已安装"
else
    yum update -y
    cd /usr/local/src/
    wget http://download.redis.io/releases/redis-6.0.6.tar.gz
    tar zxvf redis-6.0.6.tar.gz
    cd redis-6.0.6/
    make && make install
    ln -s /usr/local/src/redis-6.0.6/src/redis-server /usr/bin/redis-server
    ln -s /usr/local/src/redis-6.0.6/src/redis-cli /usr/bin/redis-cli
fi



# 定义安装目录
install_pash="/usr/local/src/redis-6.0.6"

mkdir /var/log/redis 	# 创建日志目录
mkdir ${install_pash}/dump.rdb 	# 创建数据存储目录
mkdir ${install_pash}/nodes.conf 	# 创建节点配置文件目录
cp ${install_pash}/redis.conf ${install_pash}/redis.conf.bak 	# 备份原始配置文件



# 设置端口号
ports_arr=(6379 6479 6579 6679 6779 6879)

for ports in ${ports_arr[@]}
do
    cp ${install_pash}/redis.conf ${install_pash}/redis-$ports.conf
    # 定义启动端口
    sed -i 's#port 6379#port '${ports}'#g' ${install_pash}/redis-$ports.conf
    # 守护进程(后台运行)
    sed -i 's/daemonize no/daemonize yes/g' ${install_pash}/redis-$ports.conf
    # 持久化
    sed -i 's/appendonly no/appendonly yes/g' ${install_pash}/redis-$ports.conf
    # 保存数据的AOF文件名
    sed -i 's#appendfilename "appendonly.aof"#appendfilename "appendonly_'${ports}'.aof"#g' ${install_pash}/redis-$ports.conf
    # 开启集群
    sed -i 's/# cluster-enabled yes/cluster-enabled yes/g' ${install_pash}/redis-$ports.conf
    # 日志文件
    sed -i 's#logfile ""#logfile "/var/log/redis/'${ports}'.log"#g' ${install_pash}/redis-$ports.conf
    # pid文件
    sed -i 's#pidfile /var/run/redis_6379.pid#pidfile /var/run/redis_'${ports}'.pid#g' ${install_pash}/redis-$ports.conf
    # 数据存储文件所在目录
    sed -i 's@dir ./@dir "'${install_pash}'/dump.rdb/"@g' ${install_pash}/redis-$ports.conf
    # 数据存储文件
    sed -i 's@dbfilename dump.rdb@dbfilename "dump-'${ports}'.rdb"@g' ${install_pash}/redis-$ports.conf
    # 集群节点配置文件,这个文件是不能手动编辑的.确保每一个集群节点的配置文件不同
    sed -i 's@# cluster-config-file nodes-6379.conf@cluster-config-file '${install_pash}'/nodes.conf/nodes-'${ports}'.conf@g' ${install_pash}/redis-$ports.conf

    # 启动redis服务
    redis-server ${install_pash}/redis-$ports.conf

done

sleep 3

# 查看redis服务是否启动
ps -aux | grep redis

其中 cluster-config-file 参数指定了集群配置文件的位置,每个节点在运行过程中,会维护一份集群配置文件;每当集群信息发生变化时(如增减节点),集群内所有节点会将最新信息更新到该配置文件;当节点重启后,会重新读取该配置文件,获取集群信息,可以方便的重新加入到集群中。也就是说,当Redis节点以集群模式启动时,会首先寻找是否有集群配置文件,如果有则使用文件中的配置启动,如果没有,则初始化配置并将配置保存到文件中。集群配置文件由Redis节点维护,不需要人工修改。

1.1 启动节点

为6个节点修改好相应的配置文件后,即可利用 redis-server redis-xxxx.conf 工具启动6个服务器(xxxx表示端口号,对应相应的配置文件)。利用ps命令查看进程:

代码语言:javascript
复制
~]# ps -axu | grep redis
root       59134  0.0  0.2  61788  9636 ?        Ssl  21:47   0:00 redis-server 127.0.0.1:6379 [cluster]
root       59136  0.0  0.2  61788  9684 ?        Ssl  21:47   0:00 redis-server 127.0.0.1:6479 [cluster]
root       59142  0.0  0.2  61788  9704 ?        Ssl  21:47   0:00 redis-server 127.0.0.1:6579 [cluster]
root       59148  0.0  0.2  61788  9724 ?        Ssl  21:47   0:00 redis-server 127.0.0.1:6679 [cluster]
root       59154  0.0  0.2  61788  9628 ?        Ssl  21:47   0:00 redis-server 127.0.0.1:6779 [cluster]
root       59160  0.0  0.2  61788  9704 ?        Ssl  21:47   0:00 redis-server 127.0.0.1:6879 [cluster]
root       59182  0.0  0.0  12112  1068 pts/0    S+   21:47   0:00 grep --color=auto redis

1.2 节点握手

1.1中将每个节点启动后,节点间是相互独立的,他们都处于一个只包含自己的集群当中,以端口号6379的服务器为例,利用 CLUSTER NODES 查看当前集群包含的节点。

代码语言:javascript
复制
[root@VM-0-10-centos ~]# redis-cli -p 6379 -c cluster nodes
162be0c0e035e9dfb52765e41a33069cd021f8b0 :6379@16379 myself,master - 0 0 0 connected

我们需要将各个独立的节点连接起来,构成一个包含多个节点的集群,使用 CLUSTER MEET 命令。

代码语言:javascript
复制
[root@VM-0-15-centos ~]# redis-cli -p 6379 -c cluster meet 127.0.0.1 6479
OK
[root@VM-0-15-centos ~]# redis-cli -p 6379 -c cluster meet 127.0.0.1 6579
OK
[root@VM-0-15-centos ~]# redis-cli -p 6379 -c cluster meet 127.0.0.1 6679
OK
[root@VM-0-15-centos ~]# redis-cli -p 6379 -c cluster meet 127.0.0.1 6779
OK
[root@VM-0-15-centos ~]# redis-cli -p 6379 -c cluster meet 127.0.0.1 6879
OK

再次查看此时集群中包含的节点情况:

代码语言:javascript
复制
[root@VM-0-15-centos ~]# redis-cli -p 6379 -c cluster nodes
693de0512d73a16c7525e3d85e79f6da4fcf67b2 127.0.0.1:6779@16779 master - 0 1611474390792 4 connected
d3a3d410da3db56d33dd34df8eb8f6fd3cdbced6 127.0.0.1:6579@16579 master - 0 1611474389000 0 connected
c4dbef9824daa77c553e716a968b1628d78f5377 127.0.0.1:6679@16679 master - 0 1611474389789 3 connected
162be0c0e035e9dfb52765e41a33069cd021f8b0 127.0.0.1:6379@16379 myself,master - 0 1611474387000 1 connected
92f8bc8d863e73b94b04d3f612eebf6c8b9e4834 127.0.0.1:6479@16479 master - 0 1611474389000 2 connected
1b403710ba735ad76533caa3d8f5627cf45bbda3 127.0.0.1:6879@16879 master - 0 1611474388786 5 connected

可以发现此时6个节点均作为主节点加入到集群中, CLUSTER NODES 返回的结果各项含义如下:

<id> <ip:port@cport> <flags> <master> <ping-sent> <pong-recv> <config-epoch> <link-state> <slot> <slot> ... <slot>

  • 节点id: 由40个16进制字符串组成,节点id只在集群初始化时创建一次,然后保存到集群配置文件(即前文提到的cluster-config-file)中,以后节点重新启动时会直接在集群配置文件中读取。
  • port@cport: 前者为普通端口,用于为客户端提供服务;后者为集群端口,分配方法为:普通端口+10000,只用于节点间的通讯。

其余各项的详细解释可以参考文档cluster nodes(集群节点)

1.3 槽(solts)分配(分片)

Redis集群通过分片(sharding)的方式保存数据库的键值对,整个数据库被分为16384个槽(slot),数据库每个键都属于这16384个槽的一个,集群中的每个节点都可以处理0个或者最多16384个slot。

槽是数据管理和迁移的基本单位。当数据库中的16384个槽都分配了节点时,集群处于上线状态(ok);如果有任意一个槽没有分配节点,则集群处于下线状态(fail)。

注意,只有主节点有处理槽的能力,如果将槽指派步骤放在主从复制之后,并且将槽位分配给从节点,那么集群将无法正常工作(处于下线状态)。

命令格式:redis-cli -h <节点ip> -p <节点端口> cluster addslots <指派的solts数量>

代码语言:javascript
复制
~]# redis-cli -h 127.0.0.1 -p 6379 cluster addslots {0..5000}
OK
~]# redis-cli -h 127.0.0.1 -p 6579 cluster addslots {5001..10000}
OK
~]# redis-cli -h 127.0.0.1 -p 6779 cluster addslots {10001..16383}
OK

solts指派后集群中节点情况如下:

代码语言:javascript
复制
[root@VM-0-15-centos ~]# redis-cli -p 6379 -c cluster nodes
693de0512d73a16c7525e3d85e79f6da4fcf67b2 127.0.0.1:6779@16779 master - 0 1611474535000 4 connected 10001-16383
d3a3d410da3db56d33dd34df8eb8f6fd3cdbced6 127.0.0.1:6579@16579 master - 0 1611474536216 0 connected 5001-10000
c4dbef9824daa77c553e716a968b1628d78f5377 127.0.0.1:6679@16679 master - 0 1611474533000 3 connected
162be0c0e035e9dfb52765e41a33069cd021f8b0 127.0.0.1:6379@16379 myself,master - 0 1611474532000 1 connected 0-5000
92f8bc8d863e73b94b04d3f612eebf6c8b9e4834 127.0.0.1:6479@16479 master - 0 1611474535000 2 connected
1b403710ba735ad76533caa3d8f5627cf45bbda3 127.0.0.1:6879@16879 master - 0 1611474535214 5 connected

查看集群info

代码语言:javascript
复制
[root@VM-0-15-centos ~]# redis-cli -p 6379 -c cluster info
cluster_state:ok            # 集群处于上线状态
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:5
cluster_my_epoch:1
cluster_stats_messages_ping_sent:295
cluster_stats_messages_pong_sent:327
cluster_stats_messages_meet_sent:5
cluster_stats_messages_sent:627
cluster_stats_messages_ping_received:327
cluster_stats_messages_pong_received:300
cluster_stats_messages_received:627

1.4 主从复制

上述步骤后,集群节点均作为主节点存在,仍不能实现Redis的高可用,配置主从复制之后,才算真正实现了集群的高可用功能。

CLUSTER REPLICATE <node_id> 用来让集群中接收命令的节点成为 node_id 所指定节点的从节点,并开始对主节点进行复制。

命令格式:redis-cli -p <从节点端口> cluster replicate <主节点node_id>

代码语言:javascript
复制
[root@VM-0-15-centos ~]# redis-cli -p 6479 cluster replicate 162be0c0e035e9dfb52765e41a33069cd021f8b0
OK
[root@VM-0-15-centos ~]# redis-cli -p 6679 cluster replicate d3a3d410da3db56d33dd34df8eb8f6fd3cdbced6
OK
[root@VM-0-15-centos ~]# redis-cli -p 6879 cluster replicate 693de0512d73a16c7525e3d85e79f6da4fcf67b2
OK

[root@VM-0-15-centos ~]# redis-cli -p 6379 -c cluster nodes
693de0512d73a16c7525e3d85e79f6da4fcf67b2 127.0.0.1:6779@16779 master - 0 1611474855000 4 connected 10001-16383
d3a3d410da3db56d33dd34df8eb8f6fd3cdbced6 127.0.0.1:6579@16579 master - 0 1611474857101 0 connected 5001-10000
c4dbef9824daa77c553e716a968b1628d78f5377 127.0.0.1:6679@16679 slave d3a3d410da3db56d33dd34df8eb8f6fd3cdbced6 0 1611474856097 0 connected
162be0c0e035e9dfb52765e41a33069cd021f8b0 127.0.0.1:6379@16379 myself,master - 0 1611474856000 1 connected 0-5000
92f8bc8d863e73b94b04d3f612eebf6c8b9e4834 127.0.0.1:6479@16479 slave 162be0c0e035e9dfb52765e41a33069cd021f8b0 0 1611474855000 1 connected
1b403710ba735ad76533caa3d8f5627cf45bbda3 127.0.0.1:6879@16879 slave 693de0512d73a16c7525e3d85e79f6da4fcf67b2 0 1611474856000 4 connected

1.5 在集群中执行命令

集群此时处于上线状态,可以通过客户端向集群中的节点发送命令。接收命令的节点会计算出命令要处理的键属于哪个solts,并检查这个solts是否指派给自己。

  • 如果键所在的slot刚好指派给了当前节点,会直接执行这个命令。
  • 否则,节点向客户端返回 MOVED 错误,指引客户端转向 redirect 至正确的节点,并再次发送此前的命令。

此处,我们利用 CLUSTER KEYSLOT 查看到键 foo 所在槽号为12182(被分配在6579节点),当对此键操作时,会被重定向到相应的节点。

代码语言:javascript
复制
127.0.0.1:6379> set ddk ddv
-> Redirected to slot [7560] located at 127.0.0.1:6579
OK

值得注意的是,当我们将命令通过客户端发送给一个从节点时,命令会被重定向至对应的主节点。

代码语言:javascript
复制
127.0.0.1:6479> set aak aav
-> Redirected to slot [2445] located at 127.0.0.1:6379
OK

1.6 集群故障转移

集群中主节点下线时,复制此主节点的所有的从节点将会选出一个节点作为新的主节点,并完成故障转移。和主从复制的配置相似。当原先的从节点再次上线,需要手动将宕机启动后的节点加入集群

下面模拟6379节点宕机的情况(将其shutdown),可以观察到其从节点6679将作为新的主节点继续工作。

# 6379节点宕机前

代码语言:javascript
复制
127.0.0.1:6379> cluster nodes
693de0512d73a16c7525e3d85e79f6da4fcf67b2 127.0.0.1:6779@16779 master - 0 1611475290285 4 connected 10001-16383
d3a3d410da3db56d33dd34df8eb8f6fd3cdbced6 127.0.0.1:6579@16579 master - 0 1611475289000 0 connected 5001-10000
c4dbef9824daa77c553e716a968b1628d78f5377 127.0.0.1:6679@16679 slave d3a3d410da3db56d33dd34df8eb8f6fd3cdbced6 0 1611475287000 0 connected
162be0c0e035e9dfb52765e41a33069cd021f8b0 127.0.0.1:6379@16379 myself,master - 0 1611475288000 1 connected 0-5000
92f8bc8d863e73b94b04d3f612eebf6c8b9e4834 127.0.0.1:6479@16479 slave 162be0c0e035e9dfb52765e41a33069cd021f8b0 0 1611475287278 1 connected
1b403710ba735ad76533caa3d8f5627cf45bbda3 127.0.0.1:6879@16879 slave 693de0512d73a16c7525e3d85e79f6da4fcf67b2 0 1611475289282 4 connected

上面可以看出6379节点的id:162be0c0e035e9dfb52765e41a33069cd021f8b0

对应从节点6679的节点id:92f8bc8d863e73b94b04d3f612eebf6c8b9e4834

kill掉6379节点的服务" kill -9 PID"

代码语言:javascript
复制
[root@VM-0-15-centos src]# ps -ef | grep redis
root       58341       1  0 15:40 ?        00:00:01 redis-server 127.0.0.1:6379 [cluster]
root       58343       1  0 15:40 ?        00:00:01 redis-server 127.0.0.1:6479 [cluster]
root       58345       1  0 15:40 ?        00:00:01 redis-server 127.0.0.1:6579 [cluster]
root       58347       1  0 15:40 ?        00:00:01 redis-server 127.0.0.1:6679 [cluster]
root       58353       1  0 15:40 ?        00:00:01 redis-server 127.0.0.1:6779 [cluster]
root       58355       1  0 15:40 ?        00:00:01 redis-server 127.0.0.1:6879 [cluster]
[root@VM-0-15-centos src]# kill -9 58341
[root@VM-0-15-centos src]# ps -ef | grep redis
root       58343       1  0 15:40 ?        00:00:01 redis-server 127.0.0.1:6479 [cluster]
root       58345       1  0 15:40 ?        00:00:01 redis-server 127.0.0.1:6579 [cluster]
root       58347       1  0 15:40 ?        00:00:01 redis-server 127.0.0.1:6679 [cluster]
root       58353       1  0 15:40 ?        00:00:01 redis-server 127.0.0.1:6779 [cluster]
root       58355       1  0 15:40 ?        00:00:01 redis-server 127.0.0.1:6879 [cluster]

# 6379节点宕机后(可以看到6379节点的状态是 disconnected(断开))

代码语言:javascript
复制
127.0.0.1:6479> cluster nodes
162be0c0e035e9dfb52765e41a33069cd021f8b0 127.0.0.1:6379@16379 master,fail - 1611475424650 1611475420000 1 disconnected
693de0512d73a16c7525e3d85e79f6da4fcf67b2 127.0.0.1:6779@16779 master - 0 1611475548000 4 connected 10001-16383
1b403710ba735ad76533caa3d8f5627cf45bbda3 127.0.0.1:6879@16879 slave 693de0512d73a16c7525e3d85e79f6da4fcf67b2 0 1611475549000 4 connected
92f8bc8d863e73b94b04d3f612eebf6c8b9e4834 127.0.0.1:6479@16479 myself,master - 0 1611475549000 6 connected 0-5000
c4dbef9824daa77c553e716a968b1628d78f5377 127.0.0.1:6679@16679 slave d3a3d410da3db56d33dd34df8eb8f6fd3cdbced6 0 1611475550098 0 connected
d3a3d410da3db56d33dd34df8eb8f6fd3cdbced6 127.0.0.1:6579@16579 master - 0 1611475549095 0 connected 5001-10000

恢复6379节点服务,再看下集群节点状态

代码语言:javascript
复制
127.0.0.1:6479> cluster nodes
162be0c0e035e9dfb52765e41a33069cd021f8b0 :0@0 master,fail,noaddr - 1611475424650 1611475420000 1 disconnected
693de0512d73a16c7525e3d85e79f6da4fcf67b2 127.0.0.1:6779@16779 master - 0 1611475808942 4 connected 10001-16383
1b403710ba735ad76533caa3d8f5627cf45bbda3 127.0.0.1:6879@16879 slave 693de0512d73a16c7525e3d85e79f6da4fcf67b2 0 1611475809944 4 connected
92f8bc8d863e73b94b04d3f612eebf6c8b9e4834 127.0.0.1:6479@16479 myself,master - 0 1611475807000 6 connected 0-5000
c4dbef9824daa77c553e716a968b1628d78f5377 127.0.0.1:6679@16679 slave d3a3d410da3db56d33dd34df8eb8f6fd3cdbced6 0 1611475807939 0 connected
d3a3d410da3db56d33dd34df8eb8f6fd3cdbced6 127.0.0.1:6579@16579 master - 0 1611475808000 0 connected 5001-10000

看到6379节点还是 disconnected 状态,可以看到6379节点服务是成功启动的

代码语言:javascript
复制
[root@VM-0-15-centos src]# redis-cli -p 6379 -c cluster nodes
b36cfffaff5d9bdd4cf06f3cb482d0acda381c19 127.0.0.1:6379@16379 myself,master - 0 0 0 connected

注意:观察到重新启动后的 node_id 和之前的不一样

6379宕机前的节点id:162be0c0e035e9dfb52765e41a33069cd021f8b0

6379宕机后的节点id:b36cfffaff5d9bdd4cf06f3cb482d0acda381c19

由此断定可能是重新启动节点后 node_id 发生改变导致集群无法识别,现在需要咱们手动将集群内失效的6379节点移除,加入新的6379节点

1.6.1 从集群移除节点

使用 forget 命令,剔除集群失效的6379节点(也就是:162be0c0e035e9dfb52765e41a33069cd021f8b0),因为是5个节点,所以需要每个节点挨个执行cluster forget命令

代码语言:javascript
复制
[root@VM-0-15-centos src]# redis-cli -c -p 6479 cluster forget 162be0c0e035e9dfb52765e41a33069cd021f8b0
OK
[root@VM-0-15-centos src]# redis-cli -c -p 6579 cluster forget 162be0c0e035e9dfb52765e41a33069cd021f8b0
OK
[root@VM-0-15-centos src]# redis-cli -c -p 6679 cluster forget 162be0c0e035e9dfb52765e41a33069cd021f8b0
OK
[root@VM-0-15-centos src]# redis-cli -c -p 6779 cluster forget 162be0c0e035e9dfb52765e41a33069cd021f8b0
OK
[root@VM-0-15-centos src]# redis-cli -c -p 6879 cluster forget 162be0c0e035e9dfb52765e41a33069cd021f8b0
OK

这里估计有小伙伴就发问了,为什么不用 cluster del-node 命令删除节点, 用 forget 命令还要每个节点里都执行一次,用del-node命令可以删除节点,但是 del-node 命令的格式是:cluster del-node 《host:port》 《node_id 》;这时候集群内的失效节点没有对应的host:port,所以得用 forget 命令;

可以看到失效的节点已从集群内移除

代码语言:javascript
复制
[root@VM-0-15-centos src]# redis-cli -p 6479 -c cluster nodes
693de0512d73a16c7525e3d85e79f6da4fcf67b2 127.0.0.1:6779@16779 master - 0 1611476542374 4 connected 10001-16383
1b403710ba735ad76533caa3d8f5627cf45bbda3 127.0.0.1:6879@16879 slave 693de0512d73a16c7525e3d85e79f6da4fcf67b2 0 1611476542000 4 connected
92f8bc8d863e73b94b04d3f612eebf6c8b9e4834 127.0.0.1:6479@16479 myself,master - 0 1611476543000 6 connected 0-5000
c4dbef9824daa77c553e716a968b1628d78f5377 127.0.0.1:6679@16679 slave d3a3d410da3db56d33dd34df8eb8f6fd3cdbced6 0 1611476543376 0 connected
d3a3d410da3db56d33dd34df8eb8f6fd3cdbced6 127.0.0.1:6579@16579 master - 0 1611476542000 0 connected 5001-10000

1.6.2 添加节点

命令格式:redis-cli -c -p 《集群任意节点port》 --cluster add-node 《需要加入集群的ip:port》 《集群任意节点ip:port》 --cluster-slave --cluster-master-id 《master node_id》

下面命令意思是:将6379节点作为6479节点的从节点加入集群

代码语言:javascript
复制
[root@VM-0-15-centos ~]# redis-cli -c -p 6479 --cluster add-node 127.0.0.1:6379 127.0.0.1:6479 --cluster-slave --cluster-master-id 92f8bc8d863e73b94b04d3f612eebf6c8b9e4834
>>> Adding node 127.0.0.1:6379 to cluster 127.0.0.1:6479
>>> Performing Cluster Check (using node 127.0.0.1:6479)
M: 92f8bc8d863e73b94b04d3f612eebf6c8b9e4834 127.0.0.1:6479
   slots:[0-5000] (5001 slots) master
M: 693de0512d73a16c7525e3d85e79f6da4fcf67b2 127.0.0.1:6779
   slots:[10001-16383] (6383 slots) master
   1 additional replica(s)
S: 1b403710ba735ad76533caa3d8f5627cf45bbda3 127.0.0.1:6879
   slots: (0 slots) slave
   replicates 693de0512d73a16c7525e3d85e79f6da4fcf67b2
S: c4dbef9824daa77c553e716a968b1628d78f5377 127.0.0.1:6679
   slots: (0 slots) slave
   replicates d3a3d410da3db56d33dd34df8eb8f6fd3cdbced6
M: d3a3d410da3db56d33dd34df8eb8f6fd3cdbced6 127.0.0.1:6579
   slots:[5001-10000] (5000 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 127.0.0.1:6379 to make it join the cluster.
Waiting for the cluster to join

>>> Configure node as replica of 127.0.0.1:6479.
[OK] New node added correctly.

可以看到6379节点已成功加入集群并作为6479的从节点

代码语言:javascript
复制
[root@VM-0-15-centos src]# redis-cli -p 6479 -c cluster nodes
693de0512d73a16c7525e3d85e79f6da4fcf67b2 127.0.0.1:6779@16779 master - 0 1611477298000 4 connected 10001-16383
b36cfffaff5d9bdd4cf06f3cb482d0acda381c19 127.0.0.1:6379@16379 slave 92f8bc8d863e73b94b04d3f612eebf6c8b9e4834 0 1611477299456 6 connected
1b403710ba735ad76533caa3d8f5627cf45bbda3 127.0.0.1:6879@16879 slave 693de0512d73a16c7525e3d85e79f6da4fcf67b2 0 1611477299000 4 connected
92f8bc8d863e73b94b04d3f612eebf6c8b9e4834 127.0.0.1:6479@16479 myself,master - 0 1611477301000 6 connected 0-5000
c4dbef9824daa77c553e716a968b1628d78f5377 127.0.0.1:6679@16679 slave d3a3d410da3db56d33dd34df8eb8f6fd3cdbced6 0 1611477301461 0 connected
d3a3d410da3db56d33dd34df8eb8f6fd3cdbced6 127.0.0.1:6579@16579 master - 0 1611477300459 0 connected 5001-10000

前文提到 cluster-config-file 会记录下集群节点的状态,打开其中一个节点6379的配置文件 nodes-6379.conf ,可以看到 CLUSTER NODES 所示信息均被保存在配置文件中:

代码语言:javascript
复制
[root@VM-0-15-centos src]# cat nodes-6379.conf 
693de0512d73a16c7525e3d85e79f6da4fcf67b2 127.0.0.1:6779@16779 master - 0 1611478049263 4 connected 10001-16383
c4dbef9824daa77c553e716a968b1628d78f5377 127.0.0.1:6679@16679 slave d3a3d410da3db56d33dd34df8eb8f6fd3cdbced6 0 1611478047000 0 connected
d3a3d410da3db56d33dd34df8eb8f6fd3cdbced6 127.0.0.1:6579@16579 master - 0 1611478048259 0 connected 5001-10000
1b403710ba735ad76533caa3d8f5627cf45bbda3 127.0.0.1:6879@16879 slave 693de0512d73a16c7525e3d85e79f6da4fcf67b2 0 1611478050268 4 connected
92f8bc8d863e73b94b04d3f612eebf6c8b9e4834 :0@0 master,fail,noaddr - 1611478026188 1611478021158 6 disconnected
b36cfffaff5d9bdd4cf06f3cb482d0acda381c19 127.0.0.1:6379@16379 myself,master - 0 1611478048000 8 connected 0-5000
vars currentEpoch 8 lastVoteEpoch 0

2、集群伸缩实践

集群伸缩的关键在于对集群的进行重新分片,实现槽位在节点间的迁移。本节将以在集群中添加节点和删除节点为例,对槽迁移进行实践。

2.1 添加节点

2.1.1 启动节点

按照 1.1 中介绍的步骤,启动7079和7179节点。

2.1.2 节点握手(加入集群)

借助 redis-cli --cluster add-node 命令分别添加节点7079(主)和7179(从)。

代码语言:javascript
复制
# 添加 7079(主)节点
[root@VM-0-6-centos ~]# redis-cli --cluster add-node 127.0.0.1:7079 127.0.0.1:6379
>>> Adding node 127.0.0.1:7079 to cluster 127.0.0.1:6379
>>> Performing Cluster Check (using node 127.0.0.1:6379)
M: 4830b60b806cc944550e564ef807192f00d03620 127.0.0.1:6379
   slots:[0-5000] (5001 slots) master
   1 additional replica(s)
S: aa97f087a96af31ad95768c693a3152019ee4557 127.0.0.1:6879
   slots: (0 slots) slave
   replicates 1d24fcbec775f2c7aff2bc09c29879963700e9af
M: 1d24fcbec775f2c7aff2bc09c29879963700e9af 127.0.0.1:6779
   slots:[10001-16383] (6383 slots) master
   1 additional replica(s)
M: 171c8b624e045dbc8adf517568e602948a0f2f61 127.0.0.1:6579
   slots:[5001-10000] (5000 slots) master
   1 additional replica(s)
S: a3d763ab71d8aa0fc10edc0ac9f4520b82bc6a3d 127.0.0.1:6679
   slots: (0 slots) slave
   replicates 171c8b624e045dbc8adf517568e602948a0f2f61
S: 02777dacff2e2c2c7dabe750ec80baf3333499a7 127.0.0.1:6479
   slots: (0 slots) slave
   replicates 4830b60b806cc944550e564ef807192f00d03620
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 127.0.0.1:7079 to make it join the cluster.
[OK] New node added correctly.

可以看到7079节点已添加到集群,node_id:7a06d29c98d6eaaeb39129c79259267adceb65bd

代码语言:javascript
复制
127.0.0.1:6379> CLUSTER NODES
171c8b624e045dbc8adf517568e602948a0f2f61 127.0.0.1:6579@16579 myself,master - 0 1612870545000 2 connected 5001-10000
aa97f087a96af31ad95768c693a3152019ee4557 127.0.0.1:6879@16879 slave 1d24fcbec775f2c7aff2bc09c29879963700e9af 0 1612870544495 4 connected
a3d763ab71d8aa0fc10edc0ac9f4520b82bc6a3d 127.0.0.1:6679@16679 slave 171c8b624e045dbc8adf517568e602948a0f2f61 0 1612870545496 2 connected
7a06d29c98d6eaaeb39129c79259267adceb65bd 127.0.0.1:7079@17079 master - 0 1612870544000 0 connected
1d24fcbec775f2c7aff2bc09c29879963700e9af 127.0.0.1:6779@16779 master - 0 1612870540000 4 connected 10001-16383
4830b60b806cc944550e564ef807192f00d03620 127.0.0.1:6379@16379 master - 0 1612870546498 3 connected 0-5000
02777dacff2e2c2c7dabe750ec80baf3333499a7 127.0.0.1:6479@16479 slave 4830b60b806cc944550e564ef807192f00d03620 0 1612870542000 3 connected

# 添加 7179(从)节点,并指定7079节点为其主节点

代码语言:javascript
复制
[root@VM-0-6-centos ~]# redis-cli --cluster add-node 127.0.0.1:7179 127.0.0.1:6379 --cluster-slave --cluster-master-id 7a06d29c98d6eaaeb39129c79259267adceb65bd
>>> Adding node 127.0.0.1:7179 to cluster 127.0.0.1:6379
>>> Performing Cluster Check (using node 127.0.0.1:6379)
M: 4830b60b806cc944550e564ef807192f00d03620 127.0.0.1:6379
   slots:[0-5000] (5001 slots) master
   1 additional replica(s)
M: 7a06d29c98d6eaaeb39129c79259267adceb65bd 127.0.0.1:7079
   slots: (0 slots) master
S: aa97f087a96af31ad95768c693a3152019ee4557 127.0.0.1:6879
   slots: (0 slots) slave
   replicates 1d24fcbec775f2c7aff2bc09c29879963700e9af
M: 1d24fcbec775f2c7aff2bc09c29879963700e9af 127.0.0.1:6779
   slots:[10001-16383] (6383 slots) master
   1 additional replica(s)
M: 171c8b624e045dbc8adf517568e602948a0f2f61 127.0.0.1:6579
   slots:[5001-10000] (5000 slots) master
   1 additional replica(s)
S: a3d763ab71d8aa0fc10edc0ac9f4520b82bc6a3d 127.0.0.1:6679
   slots: (0 slots) slave
   replicates 171c8b624e045dbc8adf517568e602948a0f2f61
S: 02777dacff2e2c2c7dabe750ec80baf3333499a7 127.0.0.1:6479
   slots: (0 slots) slave
   replicates 4830b60b806cc944550e564ef807192f00d03620
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 127.0.0.1:7179 to make it join the cluster.
Waiting for the cluster to join

>>> Configure node as replica of 127.0.0.1:7079.
[OK] New node added correctly.

可以看到7179节点已添加到集群,并是7079的从节点

代码语言:javascript
复制
127.0.0.1:6379> CLUSTER NODES
171c8b624e045dbc8adf517568e602948a0f2f61 127.0.0.1:6579@16579 myself,master - 0 1612870706000 2 connected 5001-10000
aa97f087a96af31ad95768c693a3152019ee4557 127.0.0.1:6879@16879 slave 1d24fcbec775f2c7aff2bc09c29879963700e9af 0 1612870708770 4 connected
a3d763ab71d8aa0fc10edc0ac9f4520b82bc6a3d 127.0.0.1:6679@16679 slave 171c8b624e045dbc8adf517568e602948a0f2f61 0 1612870707768 2 connected
43b20254222d025f7ec7413e7b9559c9222afd88 127.0.0.1:7179@17179 slave 7a06d29c98d6eaaeb39129c79259267adceb65bd 0 1612870707000 0 connected
7a06d29c98d6eaaeb39129c79259267adceb65bd 127.0.0.1:7079@17079 master - 0 1612870707000 0 connected
1d24fcbec775f2c7aff2bc09c29879963700e9af 127.0.0.1:6779@16779 master - 0 1612870704000 4 connected 10001-16383
4830b60b806cc944550e564ef807192f00d03620 127.0.0.1:6379@16379 master - 0 1612870706767 3 connected 0-5000
02777dacff2e2c2c7dabe750ec80baf3333499a7 127.0.0.1:6479@16479 slave 4830b60b806cc944550e564ef807192f00d03620 0 1612870705765 3 connected

2.2 重新分片

借助 redis-cli --cluster reshard 命令对集群重新分片,使得各节点槽位均衡(从节点6779中迁移一些slot到节点7079中)。需要指定:

  • 移动的槽位数:2776
  • 接收槽位的目标节点ID:节点7079的node_id:7a06d29c98d6eaaeb39129c79259267adceb65bd
  • 移出槽位的源节点ID:节点6779的node_id:1d24fcbec775f2c7aff2bc09c29879963700e9af

命令格式:

redis-cli --cluster reshard 《 ip:port(集群任意节点) 》 --cluster-from 《移出槽位的源节点ID,多个用逗号分隔》 --cluster-to 《接收槽位的目标节点ID》 --cluster-slots 《移动的槽位数》 --cluster-yes

代码语言:javascript
复制
[root@VM-0-6-centos ~]# redis-cli --cluster reshard 127.0.0.1:6379 --cluster-from 1d24fcbec775f2c7aff2bc09c29879963700e9af --cluster-to 7a06d29c98d6eaaeb39129c79259267adceb65bd --cluster-slots 2776 --cluster-yes 
>>> Performing Cluster Check (using node 127.0.0.1:6379)
M: 4830b60b806cc944550e564ef807192f00d03620 127.0.0.1:6379
   slots:[0-5000] (5001 slots) master
   1 additional replica(s)
M: 7a06d29c98d6eaaeb39129c79259267adceb65bd 127.0.0.1:7079
   slots:[10001-12766] (2766 slots) master
   1 additional replica(s)
S: aa97f087a96af31ad95768c693a3152019ee4557 127.0.0.1:6879
   slots: (0 slots) slave
   replicates 1d24fcbec775f2c7aff2bc09c29879963700e9af
M: 1d24fcbec775f2c7aff2bc09c29879963700e9af 127.0.0.1:6779
   slots:[12767-16383] (3617 slots) master
   1 additional replica(s)
M: 171c8b624e045dbc8adf517568e602948a0f2f61 127.0.0.1:6579
   slots:[5001-10000] (5000 slots) master
   1 additional replica(s)
S: a3d763ab71d8aa0fc10edc0ac9f4520b82bc6a3d 127.0.0.1:6679
   slots: (0 slots) slave
   replicates 171c8b624e045dbc8adf517568e602948a0f2f61
S: 43b20254222d025f7ec7413e7b9559c9222afd88 127.0.0.1:7179
   slots: (0 slots) slave
   replicates 7a06d29c98d6eaaeb39129c79259267adceb65bd
S: 02777dacff2e2c2c7dabe750ec80baf3333499a7 127.0.0.1:6479
   slots: (0 slots) slave
   replicates 4830b60b806cc944550e564ef807192f00d03620
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

Ready to move 10 slots.
  Source nodes:
    M: 1d24fcbec775f2c7aff2bc09c29879963700e9af 127.0.0.1:6779
       slots:[12767-16383] (3617 slots) master
       1 additional replica(s)
  Destination node:
    M: 7a06d29c98d6eaaeb39129c79259267adceb65bd 127.0.0.1:7079
       slots:[10001-12766] (2766 slots) master
       1 additional replica(s)
  Resharding plan:
    Moving slot 12767 from 1d24fcbec775f2c7aff2bc09c29879963700e9af
    ......
    ......
    Moving slot 12776 from 1d24fcbec775f2c7aff2bc09c29879963700e9af
Moving slot 12767 from 127.0.0.1:6779 to 127.0.0.1:7079: 
    ......
    ......
Moving slot 12776 from 127.0.0.1:6779 to 127.0.0.1:7079:

c

查看7079节点的solt数量

代码语言:javascript
复制
127.0.0.1:6579> CLUSTER NODES
171c8b624e045dbc8adf517568e602948a0f2f61 127.0.0.1:6579@16579 myself,master - 0 1612871458000 2 connected 5001-10000
aa97f087a96af31ad95768c693a3152019ee4557 127.0.0.1:6879@16879 slave 1d24fcbec775f2c7aff2bc09c29879963700e9af 0 1612871459000 4 connected
a3d763ab71d8aa0fc10edc0ac9f4520b82bc6a3d 127.0.0.1:6679@16679 slave 171c8b624e045dbc8adf517568e602948a0f2f61 0 1612871460976 2 connected
43b20254222d025f7ec7413e7b9559c9222afd88 127.0.0.1:7179@17179 slave 7a06d29c98d6eaaeb39129c79259267adceb65bd 0 1612871455969 7 connected
7a06d29c98d6eaaeb39129c79259267adceb65bd 127.0.0.1:7079@17079 master - 0 1612871460000 7 connected 10001-12776
1d24fcbec775f2c7aff2bc09c29879963700e9af 127.0.0.1:6779@16779 master - 0 1612871458000 4 connected 12777-16383
4830b60b806cc944550e564ef807192f00d03620 127.0.0.1:6379@16379 master - 0 1612871458973 3 connected 0-5000
02777dacff2e2c2c7dabe750ec80baf3333499a7 127.0.0.1:6479@16479 slave 4830b60b806cc944550e564ef807192f00d03620 0 1612871457972 3 connected

2.3 移除节点

主节点要先把solts迁移走(执行 2.2 的步骤将主节点的solt移走)再删除,从节点可以直接删除,利用 redis-cli --cluster del-node 命令依次删除从节点7079和主节点7179

命令格式

redis-cli --cluster del-node 《 ip:port(集群任意节点) 》《要移除的节点id》

代码语言:javascript
复制
# 将7079主节点的2776个solts分给6779主节点
[root@VM-0-6-centos ~]# redis-cli --cluster reshard 127.0.0.1:6379 --cluster-from 7a06d29c98d6eaaeb39129c79259267adceb65bd --cluster-to 1d24fcbec775f2c7aff2bc09c29879963700e9af --cluster-slots 2776 --cluster-yes
......
......
# 可以看到7079主节点已没有solts
127.0.0.1:6379> CLUSTER NODES
171c8b624e045dbc8adf517568e602948a0f2f61 127.0.0.1:6579@16579 myself,master - 0 1612871704000 2 connected 5001-10000
aa97f087a96af31ad95768c693a3152019ee4557 127.0.0.1:6879@16879 slave 1d24fcbec775f2c7aff2bc09c29879963700e9af 0 1612871704385 8 connected
a3d763ab71d8aa0fc10edc0ac9f4520b82bc6a3d 127.0.0.1:6679@16679 slave 171c8b624e045dbc8adf517568e602948a0f2f61 0 1612871702000 2 connected
43b20254222d025f7ec7413e7b9559c9222afd88 127.0.0.1:7179@17179 slave 1d24fcbec775f2c7aff2bc09c29879963700e9af 0 1612871705388 8 connected
7a06d29c98d6eaaeb39129c79259267adceb65bd 127.0.0.1:7079@17079 master - 0 1612871702000 7 connected
1d24fcbec775f2c7aff2bc09c29879963700e9af 127.0.0.1:6779@16779 master - 0 1612871705000 8 connected 10001-16383
4830b60b806cc944550e564ef807192f00d03620 127.0.0.1:6379@16379 master - 0 1612871703382 3 connected 0-5000
02777dacff2e2c2c7dabe750ec80baf3333499a7 127.0.0.1:6479@16479 slave 4830b60b806cc944550e564ef807192f00d03620 0 1612871706394 3 connected

# 删除7079和7179节点
[root@VM-0-6-centos ~]# redis-cli --cluster del-node 127.0.0.1:6379 7a06d29c98d6eaaeb39129c79259267adceb65bd
>>> Removing node 7a06d29c98d6eaaeb39129c79259267adceb65bd from cluster 127.0.0.1:6379
>>> Sending CLUSTER FORGET messages to the cluster...
>>> Sending CLUSTER RESET SOFT to the deleted node.
[root@VM-0-6-centos ~]# redis-cli --cluster del-node 127.0.0.1:6379 43b20254222d025f7ec7413e7b9559c9222afd88
>>> Removing node 43b20254222d025f7ec7413e7b9559c9222afd88 from cluster 127.0.0.1:6379
>>> Sending CLUSTER FORGET messages to the cluster...
>>> Sending CLUSTER RESET SOFT to the deleted node.

# 查看当前节点,可以看到7079和7179节点已从节点移除
127.0.0.1:6379> CLUSTER NODES
171c8b624e045dbc8adf517568e602948a0f2f61 127.0.0.1:6579@16579 myself,master - 0 1612872200000 2 connected 5001-10000
aa97f087a96af31ad95768c693a3152019ee4557 127.0.0.1:6879@16879 slave 1d24fcbec775f2c7aff2bc09c29879963700e9af 0 1612872198000 8 connected
a3d763ab71d8aa0fc10edc0ac9f4520b82bc6a3d 127.0.0.1:6679@16679 slave 171c8b624e045dbc8adf517568e602948a0f2f61 0 1612872198165 2 connected
1d24fcbec775f2c7aff2bc09c29879963700e9af 127.0.0.1:6779@16779 master - 0 1612872201000 8 connected 10001-16383
4830b60b806cc944550e564ef807192f00d03620 127.0.0.1:6379@16379 master - 0 1612872201171 3 connected 0-5000
02777dacff2e2c2c7dabe750ec80baf3333499a7 127.0.0.1:6479@16479 slave 4830b60b806cc944550e564ef807192f00d03620 0 1612872199166 3 connected

2.4 设置主从关系

一个主节点可以有多个从节点,下面加入7279节点作为6379主节点的第二个从节点(一般第二个从节点用作数据备份)

命令格式

redis-cli -p 《要添加的从节点端口》 cluster replicate 《对应主节点的node_id》

代码语言:javascript
复制
# 添加7179节点到集群
[root@VM-0-6-centos ~]# redis-cli --cluster add-node 127.0.0.1:7279 127.0.0.1:6379

# 设置主从关系
[root@VM-0-6-centos ~]# redis-cli -p 7279 cluster replicate 4830b60b806cc944550e564ef807192f00d03620
OK

# 查看当前集群主从关系,可以看到6379节点有7179和7279两个从节点
127.0.0.1:6579> CLUSTER NODES
171c8b624e045dbc8adf517568e602948a0f2f61 127.0.0.1:6579@16579 myself,master - 0 1612874189000 2 connected 6725-10000
aa97f087a96af31ad95768c693a3152019ee4557 127.0.0.1:6879@16879 slave 1d24fcbec775f2c7aff2bc09c29879963700e9af 0 1612874189000 10 connected
4830b60b806cc944550e564ef807192f00d03620 127.0.0.1:6379@16379 master - 0 1612874189098 3 connected 3363-5000
02777dacff2e2c2c7dabe750ec80baf3333499a7 127.0.0.1:6479@16479 slave 7a06d29c98d6eaaeb39129c79259267adceb65bd 0 1612874192103 9 connected
52eb187093695b5939630813e46e59eb48ec7489 127.0.0.1:7279@17279 slave 4830b60b806cc944550e564ef807192f00d03620 0 1612874190000 3 connected
a3d763ab71d8aa0fc10edc0ac9f4520b82bc6a3d 127.0.0.1:6679@16679 slave 171c8b624e045dbc8adf517568e602948a0f2f61 0 1612874188097 2 connected
1d24fcbec775f2c7aff2bc09c29879963700e9af 127.0.0.1:6779@16779 master - 0 1612874190100 10 connected 5905-6724 12288-16383
7a06d29c98d6eaaeb39129c79259267adceb65bd 127.0.0.1:7079@17079 master - 0 1612874191101 9 connected 0-3362 5001-5904 10001-12287
43b20254222d025f7ec7413e7b9559c9222afd88 127.0.0.1:7179@17179 slave 4830b60b806cc944550e564ef807192f00d03620 0 1612874188000 3 connected

2.5 均衡各主节点的solts

可以看到集群里各主节点的solts不均衡,这里新加一个主节点分摊其它3个主节点的solts,使其每个节点的solts都一样

代码语言:javascript
复制
[root@VM-0-6-centos ~]# redis-cli --cluster check 127.0.0.1:6379
127.0.0.1:6379 (4830b60b...) -> 0 keys | 5001 slots | 1 slaves.
127.0.0.1:6779 (1d24fcbe...) -> 0 keys | 6383 slots | 1 slaves.
127.0.0.1:6579 (171c8b62...) -> 1 keys | 5000 slots | 1 slaves.

添加7079主节点,使用--cluster rebalance命令进行平衡集群节点slot数量

命令格式

redis-cli --cluster rebalance 《 ip:port(7079节点) 》 --cluster-use-empty-masters

代码语言:javascript
复制
# 添加7079节点到集群
[root@VM-0-6-centos ~]# redis-cli --cluster add-node 127.0.0.1:7079 127.0.0.1:6379

# 让7079节点均衡其他节点的solts
[root@VM-0-6-centos ~]# redis-cli --cluster rebalance 127.0.0.1:7079 --cluster-use-empty-masters
>>> Performing Cluster Check (using node 127.0.0.1:7079)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Rebalancing across 4 nodes. Total weight = 4.00
Moving 2287 slots from 127.0.0.1:6779 to 127.0.0.1:7079
###############......###############

# 查看当前集群每个主节点的solts,可以看到每个主节点solts都是一样的
[root@VM-0-6-centos ~]# redis-cli --cluster check 127.0.0.1:6379
127.0.0.1:6379 (4830b60b...) -> 0 keys | 4096 slots | 1 slaves.
127.0.0.1:7079 (7a06d29c...) -> 0 keys | 4096 slots | 0 slaves.
127.0.0.1:6779 (1d24fcbe...) -> 0 keys | 4096 slots | 1 slaves.
127.0.0.1:6579 (171c8b62...) -> 1 keys | 4096 slots | 1 slaves.

2.6 给各主节点设置权重

各节点的机器配置不同,因此需要对配置低的节点权重设置的小一些,减少机器负载,使用--cluster rebalance --cluster-weight命令给各主节点设置权重;

命令格式

redis-cli --cluster rebalance 《ip:port(集群内任意节点都可以)》 --cluster-weight 《node_id=N node_id=N node_id=N 》--cluster-simulate(模拟rebalance操作,不会真正执行迁移操作:可选)

代码语言:javascript
复制
[root@VM-0-6-centos ~]# redis-cli --cluster rebalance 127.0.0.1:6379 --cluster-weight 4830b60b806cc944550e564ef807192f00d03620=1 171c8b624e045dbc8adf517568e602948a0f2f61=2 1d24fcbec775f2c7aff2bc09c29879963700e9af=3 7a06d29c98d6eaaeb39129c79259267adceb65bd=4
>>> Performing Cluster Check (using node 127.0.0.1:6379)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Rebalancing across 4 nodes. Total weight = 10.00
Moving 2458 slots from 127.0.0.1:6379 to 127.0.0.1:7079
###############......###############

# 检查当前集群solts的分布情况,可以看到设置的权重越大,分配的solts越多;
[root@VM-0-6-centos ~]# redis-cli --cluster check 127.0.0.1:6379
127.0.0.1:6379 (4830b60b...) -> 0 keys | 1638 slots | 1 slaves.
127.0.0.1:7079 (7a06d29c...) -> 0 keys | 6554 slots | 0 slaves.
127.0.0.1:6779 (1d24fcbe...) -> 0 keys | 4916 slots | 1 slaves.
127.0.0.1:6579 (171c8b62...) -> 1 keys | 3276 slots | 1 slaves.
[OK] 1 keys in 4 masters.

3、集群常用命令

代码语言:javascript
复制
[root@VM-0-3-centos ~]# redis-cli --cluster help
Cluster Manager Commands:
  create         host1:port1 ... hostN:portN			# 创建集群
                 --cluster-replicas <arg>				# 从节点个数
  check          host:port								# 检查集群
                 --cluster-search-multiple-owners		# 检查是否有槽同时被分配给了多个节点
  info           host:port								# 查看集群状态
  fix            host:port								# 修复集群
                 --cluster-search-multiple-owners		# 修复槽的重复分配问题
                 --cluster-fix-with-unreachable-masters
  reshard        host:port								# 指定集群的任意一节点进行迁移slot,重新分slots
                 --cluster-from <arg>					# 需要从哪些源节点上迁移slot,可从多个源节点完成迁移,以逗号隔开,传递的是节点的node                                                           id,还可以直接传递--from all,这样源节点就是集群的所有节点,不传递该参数的话,则会在                                                           迁移过程中提示用户输入
                 --cluster-to <arg>						# slot需要迁移的目的节点的node id,目的节点只能填写一个,不传递该参数的话,则会在迁移过                                                         程中提示用户输入
                 --cluster-slots <arg>					# 需要迁移的slot数量,不传递该参数的话,则会在迁移过程中提示用户输入。
                 --cluster-yes							# 指定迁移时的确认输入
                 --cluster-timeout <arg>				# 设置migrate命令的超时时间
                 --cluster-pipeline <arg>				# 设置migrate命令的超时时间
                 --cluster-replace						# 是否直接replace到目标节点
  rebalance      host:port								# 指定集群的任意一节点进行平衡集群节点slot数量 
                 --cluster-weight <node1=w1...nodeN=wN>	# 指定集群节点的权重
                 --cluster-use-empty-masters			# 设置可以让没有分配slot的主节点参与,默认不允许
                 --cluster-timeout <arg>				# 设置migrate命令的超时时间
                 --cluster-simulate						# 模拟rebalance操作,不会真正执行迁移操作
                 --cluster-pipeline <arg>				# 定义cluster getkeysinslot命令一次取出的key数量,默认值为10
                 --cluster-threshold <arg>				# 迁移的slot阈值超过threshold,执行rebalance操作
                 --cluster-replace						# 是否直接replace到目标节点
  add-node       new_host:new_port existing_host:existing_port	# 添加节点,把新节点加入到指定的集群,默认添加主节点
                 --cluster-slave						# 新节点作为从节点,默认随机一个主节点
                 --cluster-master-id <arg>				# 给新节点指定主节点
  del-node       host:port node_id						# 删除给定的一个节点,成功后关闭该节点服务
  call           host:port command arg arg .. arg		# 在集群的所有节点执行相关命令
  set-timeout    host:port milliseconds					# 设置cluster-node-timeout
  import         host:port								# 将外部redis数据导入集群
                 --cluster-from <arg>					# 将指定实例的数据导入到集群
                 --cluster-copy							# migrate时指定copy
                 --cluster-replace						# migrate时指定replace
  backup         host:port backup_directory
  help           

For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.	#对于check、fix、reshard、del node、set timeout,您可以指定集群中任何工作节点的主机和端口。

原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。

如有侵权,请联系 cloudcommunity@tencent.com 删除。

原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。

如有侵权,请联系 cloudcommunity@tencent.com 删除。

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
目录
  • 测试环境
  • redis安装
  • 集群搭建
    • 1.1 启动节点
      • 1.2 节点握手
        • 1.3 槽(solts)分配(分片)
          • 1.4 主从复制
            • 1.5 在集群中执行命令
              • 1.6 集群故障转移
                • 1.6.1 从集群移除节点
                • 1.6.2 添加节点
            • 2、集群伸缩实践
              • 2.1 添加节点
                • 2.1.1 启动节点
                • 2.1.2 节点握手(加入集群)
              • 2.2 重新分片
                • 2.3 移除节点
                  • 2.4 设置主从关系
                    • 2.5 均衡各主节点的solts
                      • 2.6 给各主节点设置权重
                      • 3、集群常用命令
                      相关产品与服务
                      云数据库 Redis
                      腾讯云数据库 Redis(TencentDB for Redis)是腾讯云打造的兼容 Redis 协议的缓存和存储服务。丰富的数据结构能帮助您完成不同类型的业务场景开发。支持主从热备,提供自动容灾切换、数据备份、故障迁移、实例监控、在线扩容、数据回档等全套的数据库服务。
                      领券
                      问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档
                      http://www.vxiaotou.com