发表文章

[最新] Redis - Setup Redis Cluster

chuckchen1222 5月前 2

创建集群
1. 修改配置文件
绑定地址: bind 192.168.3.51
监听端口:port 7000
后台运行:daemonize yes
开启Cluster:cluster-enabled yes
集群配置文件:cluster-config-file nodes-7000.conf。这个配置文件不是要我们去配的,而是Redis运行时保存配置的文件,所以我们也不可以修改这个文件。
集群超时时间:cluster-node-timeout 15000。结点超时多久则认为它宕机了。
槽是否全覆盖:cluster-require-full-coverage no。默认是yes,只要有结点宕机导致16384个槽没全被覆盖,整个集群就全部停止服务,所以一定要改为no
输出日志:logfile “./redis.log”

2. 将配置文件复制到
cd /usr/local/redis/
mkdir cfg-cluster
cd cfg-cluster
mkdir 7001 7002 7003 7004 7005 7006

将配置文件复制,并改端口 port ,cluster-config-file 参数为对应端口号
for ((i=1; i<=6; ++i)); do
cp redis.conf cfg-cluster/700$i/redis.conf.700$i
done


3. 安装redis cluster
yum -y install ruby
wget http://github.com/rubygems/rubygems/releases/download/v2.2.3/rubygems-2.2.3.tgztar xzvf rubygems-2.2.3.tgz cd rubygems-2.2.3ruby setup.rb --no-rdoc --no-ri
wget http://rubygems.org/downloads/redis-3.2.1.gemgem install redis-3.2.1.gem --local --no-rdoc --no-ri

4. 启动节点
for ((i=1; i<=6; ++i)); do
cd cfg-cluster/700$i && ../../bin/redis-server redis.conf.700$i && cd /usr/local/redis/;
done

5. 启动cluster
# bin/redis-trib.rb create --replicas 1 192.168.3.51:7001 192.168.3.51:7002 192.168.3.51:7003 192.168.3.51:7004 192.168.3.51:7005 192.168.3.51:7006
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.3.51:7001
192.168.3.51:7002
192.168.3.51:7003
Adding replica 192.168.3.51:7005 to 192.168.3.51:7001
Adding replica 192.168.3.51:7006 to 192.168.3.51:7002
Adding replica 192.168.3.51:7004 to 192.168.3.51:7003
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: b150c4ba51d4a3293ba97ba1ba260951daab7817 192.168.3.51:7001
slots:0-5460 (5461 slots) master
M: 4ffeb5a4209456520787dcbe3e929a8d2ea5f19b 192.168.3.51:7002
slots:5461-10922 (5462 slots) master
M: b573aa9fc5f51bd9f5fe49d162f145a89c6015a9 192.168.3.51:7003
slots:10923-16383 (5461 slots) master
S: c4c44644e6f9564d97b627bf4f912bc883824b1a 192.168.3.51:7004
replicates 4ffeb5a4209456520787dcbe3e929a8d2ea5f19b
S: c8383a589a00529148abe19952a38f63c0b927e8 192.168.3.51:7005
replicates b573aa9fc5f51bd9f5fe49d162f145a89c6015a9
S: 4dfb77acd69d783c44f894e29b5205c6e4e0ece7 192.168.3.51:7006
replicates b150c4ba51d4a3293ba97ba1ba260951daab7817
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join......
>>> Performing Cluster Check (using node 192.168.3.51:7001)
M: b150c4ba51d4a3293ba97ba1ba260951daab7817 192.168.3.51:7001
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: 4dfb77acd69d783c44f894e29b5205c6e4e0ece7 192.168.3.51:7006
slots: (0 slots) slave
replicates b150c4ba51d4a3293ba97ba1ba260951daab7817
M: 4ffeb5a4209456520787dcbe3e929a8d2ea5f19b 192.168.3.51:7002
slots:5461-10922 (5462 slots) master
1 additional replica(s)
M: b573aa9fc5f51bd9f5fe49d162f145a89c6015a9 192.168.3.51:7003
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: c4c44644e6f9564d97b627bf4f912bc883824b1a 192.168.3.51:7004
slots: (0 slots) slave
replicates 4ffeb5a4209456520787dcbe3e929a8d2ea5f19b
S: c8383a589a00529148abe19952a38f63c0b927e8 192.168.3.51:7005
slots: (0 slots) slave
replicates b573aa9fc5f51bd9f5fe49d162f145a89c6015a9
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

集群重启
1. 需要手工kill所有进程。
ps -ef | grep redis | awk '{print $2}' | xargs kill
2. 启动redis
for ((i=1; i<=6; ++i)); do
cd cfg-cluster/700$i && ../../bin/redis-server redis.conf.700$i && cd /usr/local/redis/;
done
3. 启动集群
bin/redis-trib.rb create --replicas 1 192.168.3.51:7001 192.168.3.51:7002 192.168.3.51:7003 192.168.3.51:7004 192.168.3.51:7005 192.168.3.51:7006



测试:
插入测试数据
for ((i=0; i<10; ++i))
do
bin/redis-cli -c -h 192.168.3.51 -p 7001 set foo$i bar$i
done

查看数据会在cluster中的不同节点自动切换
bin/redis-cli -c -h 192.168.3.51 -p 7001
192.168.3.51:7001> get foo1
-> Redirected to slot [13431] located at 192.168.3.51:7003
"bar"
192.168.3.51:7003> get foo1
"bar1"
192.168.3.51:7003> get foo2
-> Redirected to slot [1044] located at 192.168.3.51:7001
"bar2"
192.168.3.51:7001> get foo2
"bar2"
192.168.3.51:7001> get foo3
"bar3"
192.168.3.51:7001> get foo4
-> Redirected to slot [9426] located at 192.168.3.51:7002
"bar4"
192.168.3.51:7002> get foo4
"bar4"

启动新节点,添加到集群
复制参数文件
cd cfg-cluster
cp 7001/redis.conf.7001 7011/redis.conf.7011
cp 7001/redis.conf.7001 7012/redis.conf.7012
修改配置文件的port

../../bin/redis-server 7011/redis.conf.7011
../../bin/redis-server 7012/redis.conf.7012

将节点7011 加入cluster 7001
[root@dbw21ac redis]# bin/redis-trib.rb add-node 192.168.3.51:7011 192.168.3.51:7001
>>> Adding node 192.168.3.51:7011 to cluster 192.168.3.51:7001
>>> Performing Cluster Check (using node 192.168.3.51:7001)
M: b150c4ba51d4a3293ba97ba1ba260951daab7817 192.168.3.51:7001
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: 4dfb77acd69d783c44f894e29b5205c6e4e0ece7 192.168.3.51:7006
slots: (0 slots) slave
replicates b150c4ba51d4a3293ba97ba1ba260951daab7817
M: 4ffeb5a4209456520787dcbe3e929a8d2ea5f19b 192.168.3.51:7002
slots:5461-10922 (5462 slots) master
1 additional replica(s)
M: b573aa9fc5f51bd9f5fe49d162f145a89c6015a9 192.168.3.51:7003
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: c4c44644e6f9564d97b627bf4f912bc883824b1a 192.168.3.51:7004
slots: (0 slots) slave
replicates 4ffeb5a4209456520787dcbe3e929a8d2ea5f19b
S: c8383a589a00529148abe19952a38f63c0b927e8 192.168.3.51:7005
slots: (0 slots) slave
replicates b573aa9fc5f51bd9f5fe49d162f145a89c6015a9
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.3.51:7011 to make it join the cluster.
[OK] New node added correctly.
[root@dbw21ac redis]# bin/redis-cli -c -h 192.168.3.51 -p 7001 cluster nodes
4dfb77acd69d783c44f894e29b5205c6e4e0ece7 192.168.3.51:7006@17006 slave b150c4ba51d4a3293ba97ba1ba260951daab7817 0 1531203607371 6 connected
59bdd7ab4c2a9a76163d2770ff59ef25c6da3d85 192.168.3.51:7011@17011 master - 0 1531203605365 0 connected
4ffeb5a4209456520787dcbe3e929a8d2ea5f19b 192.168.3.51:7002@17002 master - 0 1531203606000 2 connected 5461-10922
b573aa9fc5f51bd9f5fe49d162f145a89c6015a9 192.168.3.51:7003@17003 master - 0 1531203605000 3 connected 10923-16383
c4c44644e6f9564d97b627bf4f912bc883824b1a 192.168.3.51:7004@17004 slave 4ffeb5a4209456520787dcbe3e929a8d2ea5f19b 0 1531203606369 4 connected
b150c4ba51d4a3293ba97ba1ba260951daab7817 192.168.3.51:7001@17001 myself,master - 0 1531203603000 1 connected 0-5460
c8383a589a00529148abe19952a38f63c0b927e8 192.168.3.51:7005@17005 slave b573aa9fc5f51bd9f5fe49d162f145a89c6015a9 0 1531203604000 5 connected
[root@dbw21ac redis]# bin/redis-trib.rb add-node --slave --master-id 59bdd7ab4c2a9a76163d2770ff59ef25c6da3d85 192.168.3.51:7012 192.168.3.51:7001
>>> Adding node 192.168.3.51:7012 to cluster 192.168.3.51:7001
>>> Performing Cluster Check (using node 192.168.3.51:7001)
M: b150c4ba51d4a3293ba97ba1ba260951daab7817 192.168.3.51:7001
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: 4dfb77acd69d783c44f894e29b5205c6e4e0ece7 192.168.3.51:7006
slots: (0 slots) slave
replicates b150c4ba51d4a3293ba97ba1ba260951daab7817
M: 59bdd7ab4c2a9a76163d2770ff59ef25c6da3d85 192.168.3.51:7011
slots: (0 slots) master
0 additional replica(s)
M: 4ffeb5a4209456520787dcbe3e929a8d2ea5f19b 192.168.3.51:7002
slots:5461-10922 (5462 slots) master
1 additional replica(s)
M: b573aa9fc5f51bd9f5fe49d162f145a89c6015a9 192.168.3.51:7003
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: c4c44644e6f9564d97b627bf4f912bc883824b1a 192.168.3.51:7004
slots: (0 slots) slave
replicates 4ffeb5a4209456520787dcbe3e929a8d2ea5f19b
S: c8383a589a00529148abe19952a38f63c0b927e8 192.168.3.51:7005
slots: (0 slots) slave
replicates b573aa9fc5f51bd9f5fe49d162f145a89c6015a9
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.3.51:7012 to make it join the cluster.
Waiting for the cluster to join.
>>> Configure node as replica of 192.168.3.51:7011.
[OK] New node added correctly.
[root@dbw21ac redis]# bin/redis-cli -c -h 192.168.3.51 -p 7001 cluster nodes
00daf29162d146958ad861fa36db697480dc3c69 192.168.3.51:7012@17012 slave 59bdd7ab4c2a9a76163d2770ff59ef25c6da3d85 0 1531203784000 0 connected
4dfb77acd69d783c44f894e29b5205c6e4e0ece7 192.168.3.51:7006@17006 slave b150c4ba51d4a3293ba97ba1ba260951daab7817 0 1531203780000 6 connected
59bdd7ab4c2a9a76163d2770ff59ef25c6da3d85 192.168.3.51:7011@17011 master - 0 1531203783036 0 connected
4ffeb5a4209456520787dcbe3e929a8d2ea5f19b 192.168.3.51:7002@17002 master - 0 1531203784039 2 connected 5461-10922
b573aa9fc5f51bd9f5fe49d162f145a89c6015a9 192.168.3.51:7003@17003 master - 0 1531203785044 3 connected 10923-16383
c4c44644e6f9564d97b627bf4f912bc883824b1a 192.168.3.51:7004@17004 slave 4ffeb5a4209456520787dcbe3e929a8d2ea5f19b 0 1531203781000 4 connected
b150c4ba51d4a3293ba97ba1ba260951daab7817 192.168.3.51:7001@17001 myself,master - 0 1531203783000 1 connected 0-5460
c8383a589a00529148abe19952a38f63c0b927e8 192.168.3.51:7005@17005 slave b573aa9fc5f51bd9f5fe49d162f145a89c6015a9 0 1531203782033 5 connected

resharding
redis-trib.rb reshard可以交互式地迁移Slot。
通过命令在动转移。
./redis-trib.rb reshard <host>:<port> --from <node-id> --to <node-id> --slots --yes
转移前:
[root@dbw21ac redis]# bin/redis-cli -c -h 192.168.3.51 -p 7001 keys "*"
1) "foo7"
2) "foo6"
3) "foo2"
4) "foo3"
[root@dbw21ac redis]# bin/redis-cli -c -h 192.168.3.51 -p 7002 keys "*"
1) "foo8"
2) "foo4"
3) "foo0"
[root@dbw21ac redis]# bin/redis-cli -c -h 192.168.3.51 -p 7003 keys "*"
1) "foo5"
2) "foo1"
3) "foo9"
[root@dbw21ac redis]# bin/redis-cli -c -h 192.168.3.51 -p 7004 keys "*"
1) "foo0"
2) "foo8"
3) "foo4"
[root@dbw21ac redis]# bin/redis-cli -c -h 192.168.3.51 -p 7005 keys "*"
1) "foo9"
2) "foo5"
3) "foo1"
[root@dbw21ac redis]# bin/redis-cli -c -h 192.168.3.51 -p 7006 keys "*"
1) "foo3"
2) "foo2"
3) "foo6"
4) "foo7"
[root@dbw21ac redis]# bin/redis-cli -c -h 192.168.3.51 -p 7011 keys "*"
(empty list or set)
[root@dbw21ac redis]# bin/redis-cli -c -h 192.168.3.51 -p 7012 keys "*"
(empty list or set)
[root@dbw21ac redis]# bin/redis-trib.rb reshard 192.168.3.51:7001
>>> Performing Cluster Check (using node 192.168.3.51:7001)
M: b150c4ba51d4a3293ba97ba1ba260951daab7817 192.168.3.51:7001
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: 00daf29162d146958ad861fa36db697480dc3c69 192.168.3.51:7012
slots: (0 slots) slave
replicates 59bdd7ab4c2a9a76163d2770ff59ef25c6da3d85
S: 4dfb77acd69d783c44f894e29b5205c6e4e0ece7 192.168.3.51:7006
slots: (0 slots) slave
replicates b150c4ba51d4a3293ba97ba1ba260951daab7817
M: 59bdd7ab4c2a9a76163d2770ff59ef25c6da3d85 192.168.3.51:7011
slots: (0 slots) master
1 additional replica(s)
M: 4ffeb5a4209456520787dcbe3e929a8d2ea5f19b 192.168.3.51:7002
slots:5461-10922 (5462 slots) master
1 additional replica(s)
M: b573aa9fc5f51bd9f5fe49d162f145a89c6015a9 192.168.3.51:7003
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: c4c44644e6f9564d97b627bf4f912bc883824b1a 192.168.3.51:7004
slots: (0 slots) slave
replicates 4ffeb5a4209456520787dcbe3e929a8d2ea5f19b
S: c8383a589a00529148abe19952a38f63c0b927e8 192.168.3.51:7005
slots: (0 slots) slave
replicates b573aa9fc5f51bd9f5fe49d162f145a89c6015a9
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 5000
What is the receiving node ID? 59bdd7ab4c2a9a76163d2770ff59ef25c6da3d85
Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1:all

Ready to move 5000 slots.
Source nodes:
M: b150c4ba51d4a3293ba97ba1ba260951daab7817 192.168.3.51:7001
slots:0-5460 (5461 slots) master
1 additional replica(s)
M: 4ffeb5a4209456520787dcbe3e929a8d2ea5f19b 192.168.3.51:7002
slots:5461-10922 (5462 slots) master
1 additional replica(s)
M: b573aa9fc5f51bd9f5fe49d162f145a89c6015a9 192.168.3.51:7003
slots:10923-16383 (5461 slots) master
1 additional replica(s)
Destination node:
M: 59bdd7ab4c2a9a76163d2770ff59ef25c6da3d85 192.168.3.51:7011
slots: (0 slots) master
1 additional replica(s)
Resharding plan:
Moving slot 5461 from 4ffeb5a4209456520787dcbe3e929a8d2ea5f19b
Moving slot 5462 from 4ffeb5a4209456520787dcbe3e929a8d2ea5f19b
............
Do you want to proceed with the proposed reshard plan (yes/no)? yes

查看resharding之后的数据
[root@dbw21ac redis]# bin/redis-cli -c -h 192.168.3.51 -p 7001 keys "*"
1) "foo7"
2) "foo3"
[root@dbw21ac redis]# bin/redis-cli -c -h 192.168.3.51 -p 7002 keys "*"
1) "foo8"
2) "foo4"
3) "foo0"
[root@dbw21ac redis]# bin/redis-cli -c -h 192.168.3.51 -p 7003 keys "*"
1) "foo5"
2) "foo1"
3) "foo9"
[root@dbw21ac redis]# bin/redis-cli -c -h 192.168.3.51 -p 7004 keys "*"
1) "foo0"
2) "foo8"
3) "foo4"
[root@dbw21ac redis]# bin/redis-cli -c -h 192.168.3.51 -p 7005 keys "*"
1) "foo9"
2) "foo5"
3) "foo1"
[root@dbw21ac redis]# bin/redis-cli -c -h 192.168.3.51 -p 7006 keys "*"
1) "foo3"
2) "foo7"
[root@dbw21ac redis]# bin/redis-cli -c -h 192.168.3.51 -p 7011 keys "*"
1) "foo2"
2) "foo6"
[root@dbw21ac redis]# bin/redis-cli -c -h 192.168.3.51 -p 7012 keys "*"
1) "foo6"
2) "foo2"

故障转移
当一个节点down之后,redis cluster可以自动切换,将slave提升为master
查看所有节点
[root@dbw21ac redis]# bin/redis-cli -c -h 192.168.3.51 -p 7001 cluster nodes
00daf29162d146958ad861fa36db697480dc3c69 192.168.3.51:7012@17012 slave 59bdd7ab4c2a9a76163d2770ff59ef25c6da3d85 0 1531205400105 8 connected
4dfb77acd69d783c44f894e29b5205c6e4e0ece7 192.168.3.51:7006@17006 slave b150c4ba51d4a3293ba97ba1ba260951daab7817 0 1531205399000 6 connected
59bdd7ab4c2a9a76163d2770ff59ef25c6da3d85 192.168.3.51:7011@17011 master - 0 1531205399000 8 connected 0-1665 5461-7127 10923-12588
4ffeb5a4209456520787dcbe3e929a8d2ea5f19b 192.168.3.51:7002@17002 master - 0 1531205401110 2 connected 7128-10922
b573aa9fc5f51bd9f5fe49d162f145a89c6015a9 192.168.3.51:7003@17003 master - 0 1531205399103 3 connected 12589-16383
c4c44644e6f9564d97b627bf4f912bc883824b1a 192.168.3.51:7004@17004 slave 4ffeb5a4209456520787dcbe3e929a8d2ea5f19b 0 1531205398099 4 connected
b150c4ba51d4a3293ba97ba1ba260951daab7817 192.168.3.51:7001@17001 myself,master - 0 1531205396000 1 connected 1666-5460
c8383a589a00529148abe19952a38f63c0b927e8 192.168.3.51:7005@17005 slave b573aa9fc5f51bd9f5fe49d162f145a89c6015a9 0 1531205396093 5 connected

[root@dbw21ac redis]# ps -ef | grep redis
root 8121 1 0 Jul09 ? 00:00:41 ../../bin/redis-server 192.168.3.51:7001 [cluster]
root 8126 1 0 Jul09 ? 00:00:41 ../../bin/redis-server 192.168.3.51:7002 [cluster]
root 8131 1 0 Jul09 ? 00:00:42 ../../bin/redis-server 192.168.3.51:7003 [cluster]
root 8136 1 0 Jul09 ? 00:00:34 ../../bin/redis-server 192.168.3.51:7004 [cluster]
root 8141 1 0 Jul09 ? 00:00:34 ../../bin/redis-server 192.168.3.51:7005 [cluster]
root 8146 1 0 Jul09 ? 00:00:35 ../../bin/redis-server 192.168.3.51:7006 [cluster]
root 23753 1 0 02:17 ? 00:00:13 ../../bin/redis-server 192.168.3.51:7011 [cluster]
root 23766 1 0 02:17 ? 00:00:04 ../../bin/redis-server 192.168.3.51:7012 [cluster]
[root@dbw21ac redis]# kill -9 8126
[root@dbw21ac redis]# ps -ef | grep redis
root 8121 1 0 Jul09 ? 00:00:41 ../../bin/redis-server 192.168.3.51:7001 [cluster]
root 8131 1 0 Jul09 ? 00:00:42 ../../bin/redis-server 192.168.3.51:7003 [cluster]
root 8136 1 0 Jul09 ? 00:00:34 ../../bin/redis-server 192.168.3.51:7004 [cluster]
root 8141 1 0 Jul09 ? 00:00:34 ../../bin/redis-server 192.168.3.51:7005 [cluster]
root 8146 1 0 Jul09 ? 00:00:35 ../../bin/redis-server 192.168.3.51:7006 [cluster]
root 23753 1 0 02:17 ? 00:00:14 ../../bin/redis-server 192.168.3.51:7011 [cluster]
root 23766 1 0 02:17 ? 00:00:04 ../../bin/redis-server 192.168.3.51:7012 [cluster]
查看cluster nodes状态,7004已经被当做master
[root@dbw21ac redis]# bin/redis-cli -c -h 192.168.3.51 -p 7001 cluster nodes
00daf29162d146958ad861fa36db697480dc3c69 192.168.3.51:7012@17012 slave 59bdd7ab4c2a9a76163d2770ff59ef25c6da3d85 0 1531205535000 8 connected
4dfb77acd69d783c44f894e29b5205c6e4e0ece7 192.168.3.51:7006@17006 slave b150c4ba51d4a3293ba97ba1ba260951daab7817 0 1531205532000 6 connected
59bdd7ab4c2a9a76163d2770ff59ef25c6da3d85 192.168.3.51:7011@17011 master - 0 1531205533669 8 connected 0-1665 5461-7127 10923-12588
4ffeb5a4209456520787dcbe3e929a8d2ea5f19b 192.168.3.51:7002@17002 master,fail - 1531205480012 1531205478407 2 disconnected
b573aa9fc5f51bd9f5fe49d162f145a89c6015a9 192.168.3.51:7003@17003 master - 0 1531205535680 3 connected 12589-16383
c4c44644e6f9564d97b627bf4f912bc883824b1a 192.168.3.51:7004@17004 master - 0 1531205534674 9 connected 7128-10922
b150c4ba51d4a3293ba97ba1ba260951daab7817 192.168.3.51:7001@17001 myself,master - 0 1531205534000 1 connected 1666-5460
c8383a589a00529148abe19952a38f63c0b927e8 192.168.3.51:7005@17005 slave b573aa9fc5f51bd9f5fe49d162f145a89c6015a9 0 1531205533000 5 connected

相关推荐
最新评论 (0)
返回
发表文章
chuckchen1222
文章数
76
评论数
0
注册排名
1305974