云计算基础架构（二）redis群集

环境:
redis版本: redis-3.2.12

实现: 3主 3从
   node01: 1主1从 (主从千万不要部署在一起单机单服务 (nginx php mysql python tomcat)
   node02: 1主1从
   node03: 1主1从

一. 基础环境的安装
1.安装redis 所有节点(node01 node02 node03)
mkdir -pv /etc/redis-cluster
yum install redis -y # gw extra

2.安装集群相关软件包
yum install ruby ruby-devel rubygems rpm-build -y

# 3.修改系统参数 (PXE装机时已经优化)
# echo "session required /lib64/security/pam_limits.so" >> /etc/pam.d/login
# wget http://10.15.200.8/sa_utils/21-nofile.conf -O /etc/security/limits.d/21-nofile.conf

# Increased maximum number of open files to 100032 (it was originally set to 1024)
# 内核优化: vm.overcommit_memory = 1' to /etc/sysctl.conf
# 'echo never > /sys/kernel/mm/transparent_hugepage/enabled'

# 4.修改配置文件将node01的配置文件分别scp到node02 node03
# /etc/redis-cluster/nodes_3000.conf /etc/redis-cluster/nodes_3001.conf
# 然后修改配置文件:
# bind 10.15.200.101 127.0.0.1 # 只修改 10.15.200.101 将此ip修改为当前node节点对应的ip地址
# [root@node01 ~]# scp /etc/redis-cluster/{nodes_3000.conf,nodes_3001.conf} node02:/etc/redis-cluster/
# [root@node01 ~]# scp /etc/redis-cluster/{nodes_3000.conf,nodes_3001.conf} node03:/etc/redis-cluster/
# [root@node02 ~]# sed -i 's/10.15.200.101/10.15.200.102/g' /etc/redis-cluster/*.conf
# [root@node03 ~]# sed -i 's/10.15.200.101/10.15.200.103/g' /etc/redis-cluster/*.conf

node01:
wget http://10.15.200.8/files/redis_cluster/node01_nodes_3000.conf -O /etc/redis-cluster/nodes_3000.conf
wget http://10.15.200.8/files/redis_cluster/node01_nodes_3001.conf -O /etc/redis-cluster/nodes_3001.conf

node02:
wget http://10.15.200.8/files/redis_cluster/node02_nodes_3000.conf -O /etc/redis-cluster/nodes_3000.conf
wget http://10.15.200.8/files/redis_cluster/node02_nodes_3001.conf -O /etc/redis-cluster/nodes_3001.conf

node03:
wget http://10.15.200.8/files/redis_cluster/node03_nodes_3000.conf -O /etc/redis-cluster/nodes_3000.conf
wget http://10.15.200.8/files/redis_cluster/node03_nodes_3001.conf -O /etc/redis-cluster/nodes_3001.conf

日志文件: tailf /var/log/redis/3000.log
如果不修改IP地址报如下的错误:
17491:M 28 Feb 08:46:11.156 # Creating Server TCP listening socket 10.15.200.101:3000: bind: Cannot assign requested address

正常的日志, 有如下的提示:
17608:M 28 Feb 08:48:19.287 * The server is now ready to accept connections on port 3000

5.启动服务 (三个节点: node01 node02 node03 都要运行如下两个命令启动3000 3001的服务:)
/usr/bin/redis-server /etc/redis-cluster/nodes_3000.conf
# tailf /var/log/redis/3000.log

/usr/bin/redis-server /etc/redis-cluster/nodes_3001.conf
# tailf /var/log/redis/3001.log

6.查看日志或查看进程 (以3000端口号为例其它同理)
1) [root@node01 ~]# tailf /var/log/redis/3000.log
2) [root@node01 ~]# ps -ef | grep 3000

7.服务全部重启后再次查看端口号 (校验所有的服务器的redis是否全部启动)
[root@node01 ~]# netstat -tulanp | grep :300
tcp 0 0 127.0.0.1:3000 0.0.0.0:* LISTEN 19899/redis-server
tcp 0 0 10.15.200.101:3000 0.0.0.0:* LISTEN 19899/redis-server
tcp 0 0 127.0.0.1:3001 0.0.0.0:* LISTEN 20282/redis-server
tcp 0 0 10.15.200.101:3001 0.0.0.0:* LISTEN 20282/redis-server

# 端口号 3000 13000 3001 13001 (含义)
8.
[root@node01 ~]# cat /etc/redis-cluster/node_3000.conf
47a9332d84fbe0ba44942e47722b3cf6fa1f2c75 :0 myself,master - 0 0 0 connected
vars currentEpoch 0 lastVoteEpoch 0
[root@node01 ~]# cat /etc/redis-cluster/node_3001.conf
947d6460c205fdb3a12445f5c442e57330b0560c :0 myself,master - 0 0 0 connected
vars currentEpoch 0 lastVoteEpoch 0

二. 配置集群Cluster
# [root@node01 ~]# gem install redis
# Fetching: redis-4.2.5.gem (100%)
# ERROR: Error installing redis:
redis requires Ruby version >= 2.3.0.

# https://rubygems.org/gems/redis/versions/3.2.2
# wget https://rubygems.org/downloads/redis-3.2.2.gem /usr/local/src

wget http://10.15.200.8/sa_utils/redis-3.2.2.gem -O /usr/local/src/redis-3.2.2.gem

[root@node01 ~]# gem install -l /usr/local/src/redis-3.2.2.gem
Successfully installed redis-3.2.2
Parsing documentation for redis-3.2.2
Installing ri documentation for redis-3.2.2
1 gem installed

node02 node03
wget http://10.15.200.8/sa_utils/redis-3.2.2.gem -O /usr/local/src/redis-3.2.2.gem

[root@node02 ~]# gem install -l /usr/local/src/redis-3.2.2.gem
Successfully installed redis-3.2.2
Parsing documentation for redis-3.2.2
Installing ri documentation for redis-3.2.2
1 gem installed

[root@node03 ~]# gem install -l /usr/local/src/redis-3.2.2.gem
Successfully installed redis-3.2.2
Parsing documentation for redis-3.2.2
Installing ri documentation for redis-3.2.2
1 gem installed

[root@node01 ~]# wget http://10.15.200.8/packages/redis-3.2.12.tar.gz -O /usr/local/src/redis-3.2.12.tar.gz
[root@node01 ~]# cd /usr/local/src/ && tar -zxvf redis-3.2.12.tar.gz && cp redis-3.2.12/src/redis-trib.rb /usr/local/sbin
[root@node01 src]# /usr/local/sbin/redis-trib.rb
Usage: redis-trib <command> <options> <arguments ...>

创建集群: replicas表示每个master需要有几个slave

[root@node01 ~]# /usr/local/sbin/redis-trib.rb create --replicas 1 10.15.200.101:3000 10.15.200.101:3001 10.15.200.102:3000 10.15.200.102:3001 10.15.200.103:3000 10.15.200.103:3001

--replicas 1: 为集群中的每个主节点创建一个从节点
集群正常工作至少需要3个主节点: 以上是三主三从

>>> Creating cluster
[ERR] Sorry, can't connect to node 10.15.200.101:3000

[root@node01 ~]# vim /usr/local/share/gems/gems/redis-3.2.2/lib/redis/client.rb +16
8 DEFAULTS = {
9 :url => lambda { ENV["REDIS_URL"] },
10 :scheme => "redis",
11 :host => "127.0.0.1",
12 :port => 6379,
13 :path => nil,
14 :timeout => 5.0,
15 :connect_timeout => 5.0,
16 :password => 'admin', # 修改此处的密码 (一定要加引号)

[root@node01 ~]# /usr/local/sbin/redis-trib.rb create --replicas 1 10.15.200.101:3000 10.15.200.101:3001 10.15.200.102:3000 10.15.200.102:3001 10.15.200.103:3000 10.15.200.103:3001

>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
10.15.200.101:3000
10.15.200.102:3000
10.15.200.103:3000
Adding replica 10.15.200.102:3001 to 10.15.200.101:3000
Adding replica 10.15.200.101:3001 to 10.15.200.102:3000
Adding replica 10.15.200.103:3001 to 10.15.200.103:3000
M: 47a9332d84fbe0ba44942e47722b3cf6fa1f2c75 10.15.200.101:3000
slots:0-5460 (5461 slots) master
S: 947d6460c205fdb3a12445f5c442e57330b0560c 10.15.200.101:3001
replicates 24f84465fdd543dd8f6a9e50375afd793db7941b
M: 24f84465fdd543dd8f6a9e50375afd793db7941b 10.15.200.102:3000
slots:5461-10922 (5462 slots) master
S: 08e742f053d23c8f1d2ed8d5dd01d6dca80c9d6e 10.15.200.102:3001
replicates 47a9332d84fbe0ba44942e47722b3cf6fa1f2c75
M: d5bbda1bfeb49d708a1c415c8e0e06fd3969ff11 10.15.200.103:3000
slots:10923-16383 (5461 slots) master
S: d5f8e89d6ecd483751991d201e1ed689cc04a28a 10.15.200.103:3001
replicates d5bbda1bfeb49d708a1c415c8e0e06fd3969ff11
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join.....
>>> Performing Cluster Check (using node 10.15.200.101:3000)
M: 47a9332d84fbe0ba44942e47722b3cf6fa1f2c75 10.15.200.101:3000
slots:0-5460 (5461 slots) master
1 additional replica(s)
M: 24f84465fdd543dd8f6a9e50375afd793db7941b 10.15.200.102:3000
slots:5461-10922 (5462 slots) master
1 additional replica(s)
S: 947d6460c205fdb3a12445f5c442e57330b0560c 10.15.200.101:3001
slots: (0 slots) slave
replicates 24f84465fdd543dd8f6a9e50375afd793db7941b
S: 08e742f053d23c8f1d2ed8d5dd01d6dca80c9d6e 10.15.200.102:3001
slots: (0 slots) slave
replicates 47a9332d84fbe0ba44942e47722b3cf6fa1f2c75
S: d5f8e89d6ecd483751991d201e1ed689cc04a28a 10.15.200.103:3001
slots: (0 slots) slave
replicates d5bbda1bfeb49d708a1c415c8e0e06fd3969ff11
M: d5bbda1bfeb49d708a1c415c8e0e06fd3969ff11 10.15.200.103:3000
slots:10923-16383 (5461 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

[root@node01 ~]# /usr/bin/redis-cli -c -p 3000 -h 10.15.200.101 -a 'admin'
10.15.200.103:3000> cluster nodes
08e742f053d23c8f1d2ed8d5dd01d6dca80c9d6e 10.15.200.102:3001 slave 47a9332d84fbe0ba44942e47722b3cf6fa1f2c75 0 1614518465535 4 connected
947d6460c205fdb3a12445f5c442e57330b0560c 10.15.200.101:3001 slave 24f84465fdd543dd8f6a9e50375afd793db7941b 0 1614518464528 3 connected
47a9332d84fbe0ba44942e47722b3cf6fa1f2c75 10.15.200.101:3000 master - 0 1614518464023 1 connected 0-5460
d5bbda1bfeb49d708a1c415c8e0e06fd3969ff11 10.15.200.103:3000 myself,master - 0 0 5 connected 10923-16383
d5f8e89d6ecd483751991d201e1ed689cc04a28a 10.15.200.103:3001 slave d5bbda1bfeb49d708a1c415c8e0e06fd3969ff11 0 1614518461507 6 connected
24f84465fdd543dd8f6a9e50375afd793db7941b 10.15.200.102:3000 master - 0 1614518463520 3 connected 5461-10922

10.15.200.103:3000> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:5
cluster_stats_messages_sent:7209
cluster_stats_messages_received:7209

三. 测试

1）get 和 set数据
[root@node01 ~]# echo 'hello_world' | /usr/bin/redis-cli -c -p 3000 -h 10.15.200.101 -a 'admin' -x set mykey
OK

[root@node01 ~]# /usr/bin/redis-cli -c -p 3000 -h 10.15.200.101 -a 'admin'
10.15.200.101:3000> get mykey
-> Redirected to slot [14687] located at 10.15.200.103:3000
"hello_world\n" # 直接根据hash匹配切换到相应的slot的节点上

# 将 10.15.200.103 3000 端口对应的服务停止之后会重分布
[root@node01 ~]# /usr/bin/redis-cli -c -p 3000 -h 10.15.200.101 -a 'admin'
10.15.200.101:3000> get mykey
-> Redirected to slot [14687] located at 10.15.200.103:3001
"hello_world\n"

写入一定数量的数据: # node01 直接命令行粘贴
for i in `seq 1 10000`
do
echo "hello_world_$i" | /usr/bin/redis-cli -c -p 3000 -h 10.15.200.101 -a 'admin' -x set "mykey_$i"
done

查看现集群分布情况:
[root@node01 ~]# /usr/local/sbin/redis-trib.rb info 10.15.200.101:3000
10.15.200.103:3001 (bbf0844e...) -> 3351 keys | 5461 slots | 1 slaves.
10.15.200.102:3000 (462e7cf4...) -> 3316 keys | 5462 slots | 1 slaves.
10.15.200.102:3001 (6ac1063d...) -> 3335 keys | 5461 slots | 1 slaves.
[OK] 10002 keys in 3 masters.
0.61 keys per slot on average.

增加一个主节点:
注意: 最后面一段地址+端口号: 10.15.200.101:3000 是现集群的任意一个正常的IP:PORT都可以

[root@node01 ~]# /usr/local/sbin/redis-trib.rb add-node 10.15.200.104:3000 10.15.200.101:3000
>>> Adding node 10.15.200.104:3000 to cluster 10.15.200.101:3000
>>> Performing Cluster Check (using node 10.15.200.101:3000)
S: 0d0c41cfa78c4372dc64da8ac957d5344ef69424 10.15.200.101:3000
slots: (0 slots) slave
replicates 6ac1063d0e014e89606a84c0cece3ddf4ee6b68e
M: bbf0844ee9b5c25635dc8f42f674e4ce631a64f9 10.15.200.103:3001
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: 86b188db102f856a129cafda55c7f6da57982881 10.15.200.101:3001
slots: (0 slots) slave
replicates 462e7cf49b2e9e1159ffe94439cd0afc231e9f4c
M: 462e7cf49b2e9e1159ffe94439cd0afc231e9f4c 10.15.200.102:3000
slots:5461-10922 (5462 slots) master
1 additional replica(s)
S: 107b83d08c63eed219242f7a32585d626ed47dba 10.15.200.103:3000
slots: (0 slots) slave
replicates bbf0844ee9b5c25635dc8f42f674e4ce631a64f9
M: 6ac1063d0e014e89606a84c0cece3ddf4ee6b68e 10.15.200.102:3001
slots:0-5460 (5461 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 10.15.200.104:3000 to make it join the cluster.
[OK] New node added correctly. # 添加成功

[root@node01 ~]# /usr/local/sbin/redis-trib.rb check 10.15.200.104:3000 # 检查集群状态
>>> Performing Cluster Check (using node 10.15.200.104:3000)
M: e140c57f1eb4d1694c14f05b2bafc998e00fbc94 10.15.200.104:3000
slots: (0 slots) master # 默认刚刚新增的主节点是没有slots的, 这样在存取数据的时候不会被选择
0 additional replica(s)

[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

# 查看key的分部情况
[root@node01 ~]# /usr/local/sbin/redis-trib.rb info 10.15.200.101:3000
10.15.200.104:3000 (e140c57f...) -> 0 keys | 0 slots | 0 slaves.
10.15.200.103:3001 (bbf0844e...) -> 3351 keys | 5461 slots | 1 slaves.
10.15.200.102:3000 (462e7cf4...) -> 3316 keys | 5462 slots | 1 slaves.
10.15.200.102:3001 (6ac1063d...) -> 3335 keys | 5461 slots | 1 slaves.
[OK] 10002 keys in 4 masters.
0.61 keys per slot on average.

[root@node01 ~]# /usr/local/sbin/redis-trib.rb reshard 10.15.200.104:3000
>>> Performing Cluster Check (using node 10.15.200.104:3000)
M: e140c57f1eb4d1694c14f05b2bafc998e00fbc94 10.15.200.104:3000
slots: (0 slots) master
0 additional replica(s)

删除节点:
[root@node01 ~]# /usr/local/sbin/redis-trib.rb check 10.15.200.104:3000
>>> Performing Cluster Check (using node 10.15.200.104:3000)
M: 4319c3a09b885fc0c818c7db946293f46ec8a0b9 10.15.200.104:3000 # 注意此处的 ID号后面删除时会用到

[root@node01 ~]# /usr/local/sbin/redis-trib.rb del-node 10.15.200.104:3000 4319c3a09b885fc0c818c7db946293f46ec8a0b9
>>> Removing node 4319c3a09b885fc0c818c7db946293f46ec8a0b9 from cluster 10.15.200.104:3000
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.

报错:
[ERR] Node 10.15.200.104:3000 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.

由于服务器重启原因, 导致该节点中默认生成的配置或历史存储数据不一致导致启动集群报错, 需要清除该节点的文件, 必要时清除数据库, 步骤如下:
[root@node04 ~]# rm -fr /var/lib/redis/*
[root@node04 ~]# redis-cli -h 127.0.0.1 -a 'admin' -p 3000

3.关闭redis服务
ps -ef
kill -9
4.启动服务
/usr/bin/redis-server /etc/redis-cluster/nodes_3000.conf

[ERR] Calling MIGRATE: ERR Target instance replied with error: NOAUTH Authentication required.

云计算基础架构（二）redis群集相关推荐

云计算基础架构(一)
一.云计算的定义一种计算模式:把IT资源.数据.应用作为服务通过网络提供给用户(IBM) 一种基础架构管理方法论:把大量的高度虚拟化的资源管理起来,组成一个大的资源池,用来统一提供服务(IBM) 以 ...
视频教程-JAVAEE真实课堂系列之javaScript全讲-云计算基础架构
JAVAEE真实课堂系列之javaScript全讲刘志远,北京邮电大学硕士研究生, 北京育华志远科技有限公司创始人, 育华志远教育品牌负责人,育华志远课程体系打造者. 率领团队为互联网行业培训千余名 ...
视频教程-JAVAEE真实课堂之MySQL数据库-云计算基础架构
JAVAEE真实课堂之MySQL数据库刘志远,北京邮电大学硕士研究生, 北京育华志远科技有限公司创始人, 育华志远教育品牌负责人,育华志远课程体系打造者. 率领团队为互联网行业培训千余名学员,打造最 ...
视频教程-JAVAEE真实课堂系列之XML全讲-云计算基础架构
JAVAEE真实课堂系列之XML全讲刘志远,北京邮电大学硕士研究生, 北京育华志远科技有限公司创始人, 育华志远教育品牌负责人,育华志远课程体系打造者. 率领团队为互联网行业培训千余名学员,打造最高 ...
IBM IaaS：云计算基础架构明星产品解决方案
IBM IaaS:云计算基础架构明星产品解决方案 IBM基础设施即服务IaaS云计算架构明星产品包括PureSystem .System Z.Power Cloud.System X.Storage ...
【合作】云计算基础架构【图】
云计算不仅是技术,更是服务模式的创新.云计算之所以能够为用户带来更高的效率.灵活性和可扩展性,是基于对整个IT领域的变革,其技术和应用涉及硬件系统.软件系统.应用系统.运维管理.服务模式等各个方面. ...
云计算基础（二）—— 虚拟化
虚拟化: 一台PC机的组成包括:Keyboard(键盘).Monitor(显示器).CPU.RAM.I/O(Disk,Network),这是基本的五大部件. 虚拟化就是在这些基础物理设备上运行多个OS ...
云计算基础架构（三）Mongodb
configure: 配置文件模板 (主从配置分片集群) *.conf 全为模板配置文件 mongodb.tgz 所有完整的配置文件包含测试用的. ...
字节后端开发工程师-基础架构二面面经
二面 1.自我介绍 2.qemu-kvm的虚拟化过程讲一讲? 3.虚拟机上硬件设备如何模拟的? 4.virtio中的VM Exit状态了解吗?什么时候会出现这个状态? 5.调度系统中的 sched_e ...

云计算基础架构（二）redis群集

云计算基础架构（二）redis群集相关推荐

最新文章

热门文章

云计算基础架构 （二）redis群集

云计算基础架构 （二）redis群集相关推荐

最新文章

热门文章

云计算基础架构（二）redis群集

云计算基础架构（二）redis群集相关推荐