数据库系列之MongoDB副本集集群环境

MongoDB集群环境有3种模式：主从复制、副本集和分片，本文详细介绍副本集模式的部署和配置，并进行测试验证。

数据库系列之MongoDB主从复制集群环境
数据库系列之MongoDB副本集集群环境
数据库系列之MongoDB分片集群环境

3、副本集

副本集是mongod实例的集合，它们维护相同的数据。Replica sets中包含一些data nodes，有些也会包含一个artiber node。在data nodes中，有且只有一个node会被选择为主节点，其它节点为从节点。

主节点接受所有写操作，replica sets中只有一个主节点设置为 { w: “majority” }
主节点会记录所有对datasets的操作到operation log中
从节点会复制主节点的oplog并将这些操作应用到datasets上，以和主节点数据同步
如果主节点不可用，从节点会选择自己为主节点

下图为一个主节点加上2个从节点：

也可以增加额外的mongod实例作为artiber，artiber不会保存任何数据，只作为心跳的回应和主节点选举的裁判。

3.1 配置服务器

1）配置Master服务器

#MongoDB config - 2018.05.15
#Replication Set Configuration#config database path
dbpath = /usr/local/mongodb/data#config log path
logpath = /usr/local/mongodb/logs/mongodb.logdirectoryperdb = true
logappend = truereplSet = rs-tango-01#config port
port = 27017
bind_ip = 192.168.112.101#config fork
fork = trueoplogSize = 2000
noprealloc = true

2）配置slave服务器1机

#MongoDB config - 2018.05.15
#Replication Set Configuration#config database path
dbpath = /usr/local/mongodb/data#config log path
logpath = /usr/local/mongodb/logs/mongodb.log#
directoryperdb = true
logappend = true#
replSet = rs-tango-01#config port
port = 27017
bind_ip = 192.168.112.102#config fork
fork = true#
oplogSize = 2000
noprealloc = true

3）配置slave服务器2机

#MongoDB config - 2018.05.15
#Replication Set Configuration#config database path
dbpath = /usr/local/mongodb/data#config log path
logpath = /usr/local/mongodb/logs/mongodb.log#
directoryperdb = true
logappend = true#
replSet = rs-tango-01#config port
port = 27017
bind_ip = 192.168.112.103#config fork
fork = true#
oplogSize = 2000
noprealloc = true

4）参数解释

dbpath：数据存放目录
logpath：日志存放路径
pidfilepath：进程文件，方便停止mongodb
directoryperdb：为每一个数据库按照数据库名建立文件夹存放
logappend：以追加的方式记录日志
replSet：replica set的名字
bind_ip：mongodb所绑定的ip地址
port：mongodb进程所使用的端口号，默认为27017
oplogSize：mongodb操作日志文件的最大大小。单位为Mb，默认为硬盘剩余空间的5%
fork：以后台方式运行进程
noprealloc：不预先分配存储

5）节点和初始化高级参数

standard 常规节点:参与投票有可能成为活跃节点
passive 副本节点:参与投票,但是不能成为活跃节点
arbiter 仲裁节点:只是参与投票不复制节点也不能成为活跃节点
Priority：0到1000之间 ,0代表是副本节点 ,1到1000是常规节点
arbiterOnly：true 仲裁节点

3.2 启动MongoDB

1）在3台服务器分别启动MongoDB

[root@tango-centos01 mongodb-linux-x86_64-rhel70-3.6.3]#./bin/mongod -f ./config/master.conf
[root@tango-centos02 mongodb-linux-x86_64-rhel70-3.6.3]#./bin/mongod -f ./config/slave.conf
[root@tango-centos03 mongodb-linux-x86_64-rhel70-3.6.3]#./bin/mongod -f ./config/slave.conf

2）查看端口

[root@tango-centos01 mongodb-linux-x86_64-rhel70-3.6.3]# netstat -tlnp | grep mongod
tcp        0      0 192.168.112.101:27017   0.0.0.0:*  LISTEN      1217/./bin/mongod
[root@tango-centos02 mongodb-linux-x86_64-rhel70-3.6.3]# netstat -tlnp | grep mongod
tcp        0      0 192.168.112.102:27017   0.0.0.0:*   LISTEN      1096/./bin/mongod
[root@tango-centos03 mongodb-linux-x86_64-rhel70-3.6.3]# netstat -tlnp | grep mongod
tcp        0      0 192.168.112.103:27017   0.0.0.0:*   LISTEN      1097/./bin/mongod

3.3 配置主备节点

1）选择3台节点中的任意一台连接到MongoDB，主备节点的配置需要使用到admin database，具体如下：

[[root@tango-centos01 mongodb-linux-x86_64-rhel70-3.6.3]# ./bin/mongo 192.168.112.101:27017
MongoDB shell version v3.6.3
connecting to: mongodb://192.168.112.101:27017/test
MongoDB server version: 3.6.3
Welcome to the MongoDB shell.
…
> use admin
switched to db admin
> cfg={_id:"rs-tango-01",members:[{_id:0,host:'192.168.112.101:27017'},{_id:1,host:'192.168.112.102:27017'},{_id:2,host:'192.168.112.103:27017'}]}
{"_id" : "rs-tango-01","members" : [{"_id" : 0,"host" : "192.168.112.101:27017"},{"_id" : 1,"host" : "192.168.112.102:27017"},{"_id" : 2,"host" : "192.168.112.103:27017"}]
}

2）初始化副本集的配置

> rs.initiate(cfg)rs.initiate(cfg)
{"ok" : 1,"operationTime" : Timestamp(1526623743, 1),"$clusterTime" : {"clusterTime" : Timestamp(1526623743, 1),"signature" : {"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),"keyId" : NumberLong(0)}}
}

3）查看副本集状态

rs-tango-01:SECONDARY> rs.status()rs.status()
{"set" : "rs-tango-01","date" : ISODate("2018-05-18T06:09:31.892Z"),"myState" : 1,"term" : NumberLong(1),"heartbeatIntervalMillis" : NumberLong(2000),"optimes" : {"lastCommittedOpTime" : {"ts" : Timestamp(1526623760, 1),"t" : NumberLong(1)},"readConcernMajorityOpTime" : {"ts" : Timestamp(1526623760, 1),"t" : NumberLong(1)},"appliedOpTime" : {"ts" : Timestamp(1526623760, 1),"t" : NumberLong(1)},"durableOpTime" : {"ts" : Timestamp(1526623760, 1),"t" : NumberLong(1)}},"members" : [{"_id" : 0,"name" : "192.168.112.101:27017","health" : 1,"state" : 1,"stateStr" : "PRIMARY","uptime" : 747,"optime" : {"ts" : Timestamp(1526623760, 1),"t" : NumberLong(1)},"optimeDate" : ISODate("2018-05-18T06:09:20Z"),"infoMessage" : "could not find member to sync from","electionTime" : Timestamp(1526623754, 1),"electionDate" : ISODate("2018-05-18T06:09:14Z"),"configVersion" : 1,"self" : true},{"_id" : 1,"name" : "192.168.112.102:27017","health" : 1,"state" : 2,"stateStr" : "SECONDARY","uptime" : 28,"optime" : {"ts" : Timestamp(1526623760, 1),"t" : NumberLong(1)},"optimeDurable" : {"ts" : Timestamp(1526623760, 1),"t" : NumberLong(1)},"optimeDate" : ISODate("2018-05-18T06:09:20Z"),"optimeDurableDate" : ISODate("2018-05-18T06:09:20Z"),"lastHeartbeat" : ISODate("2018-05-18T06:09:30.420Z"),"lastHeartbeatRecv" : ISODate("2018-05-18T06:09:31.335Z"),"pingMs" : NumberLong(0),"syncingTo" : "192.168.112.101:27017","configVersion" : 1},{"_id" : 2,"name" : "192.168.112.103:27017","health" : 1,"state" : 2,"stateStr" : "SECONDARY","uptime" : 28,"optime" : {"ts" : Timestamp(1526623760, 1),"t" : NumberLong(1)},"optimeDurable" : {"ts" : Timestamp(1526623760, 1),"t" : NumberLong(1)},"optimeDate" : ISODate("2018-05-18T06:09:20Z"),"optimeDurableDate" : ISODate("2018-05-18T06:09:20Z"),"lastHeartbeat" : ISODate("2018-05-18T06:09:30.421Z"),"lastHeartbeatRecv" : ISODate("2018-05-18T06:09:31.331Z"),"pingMs" : NumberLong(0),"syncingTo" : "192.168.112.101:27017","configVersion" : 1}],"ok" : 1,"operationTime" : Timestamp(1526623760, 1),"$clusterTime" : {"clusterTime" : Timestamp(1526623760, 1),"signature" : {"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),"keyId" : NumberLong(0)}}
}
rs-tango-01:PRIMARY>

可以看到，192.168.112.101这台服务器配置为主节点，其它2台服务器配置为备节点，并且通过mongo shell登录可以看到前缀名称主节点的变成rs-tango-01:PRIMARY、备节点变成rs-tango-01:SECONDARY。

3.4 主备环境测试验证

场景一：主节点写数据，查看从节点是否能看到数据

1）主节点写入数据

rs-tango-01:PRIMARY> use rs_testuse rs_test
switched to db rs_test
rs-tango-01:PRIMARY> function add(){function add(){
... var i=0;var i=0;
... for(;i<20;i++){for(;i<20;i++){
... db.person.insert({"name":"tang"+i})db.person.insert({"name":"tang"+i})
... }}
... }}
rs-tango-01:PRIMARY> add()add()
rs-tango-01:PRIMARY> db.person.find()db.person.find()
{ "_id" : ObjectId("5afe70379a8bdfec900dfd1f"), "name" : "tang0" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd20"), "name" : "tang1" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd21"), "name" : "tang2" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd22"), "name" : "tang3" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd23"), "name" : "tang4" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd24"), "name" : "tang5" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd25"), "name" : "tang6" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd26"), "name" : "tang7" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd27"), "name" : "tang8" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd28"), "name" : "tang9" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd29"), "name" : "tang10" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd2a"), "name" : "tang11" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd2b"), "name" : "tang12" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd2c"), "name" : "tang13" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd2d"), "name" : "tang14" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd2e"), "name" : "tang15" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd2f"), "name" : "tang16" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd30"), "name" : "tang17" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd31"), "name" : "tang18" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd32"), "name" : "tang19" }
rs-tango-01:PRIMARY> show collectionsshow collections
person

2）默认情况下从节点是不可读的，报错如下：

rs-tango-01:SECONDARY> shwshow collectionsshow collections
2018-05-18T14:20:50.512+0800 E QUERY    [thread1] Error: listCollections failed: {"operationTime" : Timestamp(1526624436, 1),"ok" : 0,"errmsg" : "not master and slaveOk=false","code" : 13435,"codeName" : "NotMasterNoSlaveOk","$clusterTime" : {"clusterTime" : Timestamp(1526624436, 1),"signature" : {"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),"keyId" : NumberLong(0)}}
} :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
DB.prototype._getCollectionInfosCommand@src/mongo/shell/db.js:941:1
DB.prototype.getCollectionInfos@src/mongo/shell/db.js:953:19
DB.prototype.getCollectionNames@src/mongo/shell/db.js:964:16
shellHelper.show@src/mongo/shell/utils.js:809:9
shellHelper@src/mongo/shell/utils.js:706:15
@(shellhelp2):1:1
rs-tango-01:SECONDARY>

3）因为从节点默认不允许读写的，在==从节点执行rs.slaveOk()==可解决

rs-tango-01:SECONDARY> rs.slaveOk();rs.slaveOk();
rs-tango-01:SECONDARY> show collectionsshow collections
person
rs-tango-01:SECONDARY> db.person.find()db.person.find()
{ "_id" : ObjectId("5afe70379a8bdfec900dfd1f"), "name" : "tang0" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd21"), "name" : "tang2" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd20"), "name" : "tang1" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd22"), "name" : "tang3" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd23"), "name" : "tang4" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd24"), "name" : "tang5" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd25"), "name" : "tang6" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd26"), "name" : "tang7" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd27"), "name" : "tang8" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd28"), "name" : "tang9" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd29"), "name" : "tang10" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd2a"), "name" : "tang11" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd2b"), "name" : "tang12" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd2c"), "name" : "tang13" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd2e"), "name" : "tang15" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd2d"), "name" : "tang14" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd2f"), "name" : "tang16" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd30"), "name" : "tang17" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd31"), "name" : "tang18" }
{ "_id" : ObjectId("5afe70389a8bdfec900dfd32"), "name" : "tang19" }

场景二：主节点异常，查看集群状态

1）强制kill掉主节点MongoDB进程

[root@tango-centos01 mongodb-linux-x86_64-rhel70-3.6.3]# ps -ef|grep mongo
root       1235      1  0 13:57 ?        00:00:16 /usr/local/mongodb/mongodb-linux-x86_64-rhel70-3.6.3/bin/mongod --config /usr/local/mongodb/mongodb-linux-x86_64-rhel70-3.6.3/config/master.conf
root       1479   1209  0 14:33 pts/0    00:00:00 grep --color=auto mongo
[root@tango-centos01 mongodb-linux-x86_64-rhel70-3.6.3]# kill -9 1235
[root@tango-centos01 mongodb-linux-x86_64-rhel70-3.6.3]# ps -ef|grep mongo
root       1487   1209  0 14:33 pts/0    00:00:00 grep --color=auto mongo

2）查看MongoDB服务状态，备节点192.168.112.103已经变成主节点

rs-tango-01:SECONDARY> rs.status()rs.status()
{"set" : "rs-tango-01","date" : ISODate("2018-05-18T06:34:43.512Z"),"myState" : 2,"term" : NumberLong(2),"syncingTo" : "192.168.112.103:27017","heartbeatIntervalMillis" : NumberLong(2000),"optimes" : {"lastCommittedOpTime" : {"ts" : Timestamp(1526625282, 1),"t" : NumberLong(2)},"readConcernMajorityOpTime" : {"ts" : Timestamp(1526625282, 1),"t" : NumberLong(2)},"appliedOpTime" : {"ts" : Timestamp(1526625282, 1),"t" : NumberLong(2)},"durableOpTime" : {"ts" : Timestamp(1526625282, 1),"t" : NumberLong(2)}},"members" : [{"_id" : 0,"name" : "192.168.112.101:27017","health" : 0,"state" : 8,"stateStr" : "(not reachable/healthy)","uptime" : 0,"optime" : {"ts" : Timestamp(0, 0),"t" : NumberLong(-1)},"optimeDurable" : {"ts" : Timestamp(0, 0),"t" : NumberLong(-1)},"optimeDate" : ISODate("1970-01-01T00:00:00Z"),"optimeDurableDate" : ISODate("1970-01-01T00:00:00Z"),"lastHeartbeat" : ISODate("2018-05-18T06:34:42.373Z"),"lastHeartbeatRecv" : ISODate("2018-05-18T06:33:49.341Z"),"pingMs" : NumberLong(0),"lastHeartbeatMessage" : "Connection refused","configVersion" : -1},{"_id" : 1,"name" : "192.168.112.102:27017","health" : 1,"state" : 2,"stateStr" : "SECONDARY","uptime" : 1579,"optime" : {"ts" : Timestamp(1526625282, 1),"t" : NumberLong(2)},"optimeDate" : ISODate("2018-05-18T06:34:42Z"),"syncingTo" : "192.168.112.103:27017","configVersion" : 1,"self" : true},{"_id" : 2,"name" : "192.168.112.103:27017","health" : 1,"state" : 1,"stateStr" : "PRIMARY","uptime" : 1531,"optime" : {"ts" : Timestamp(1526625282, 1),"t" : NumberLong(2)},"optimeDurable" : {"ts" : Timestamp(1526625282, 1),"t" : NumberLong(2)},"optimeDate" : ISODate("2018-05-18T06:34:42Z"),"optimeDurableDate" : ISODate("2018-05-18T06:34:42Z"),"lastHeartbeat" : ISODate("2018-05-18T06:34:42.357Z"),"lastHeartbeatRecv" : ISODate("2018-05-18T06:34:41.908Z"),"pingMs" : NumberLong(0),"electionTime" : Timestamp(1526625240, 1),"electionDate" : ISODate("2018-05-18T06:34:00Z"),"configVersion" : 1}],"ok" : 1,"operationTime" : Timestamp(1526625282, 1),"$clusterTime" : {"clusterTime" : Timestamp(1526625282, 1),"signature" : {"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),"keyId" : NumberLong(0)}}
}
rs-tango-01:SECONDARY>

3）重新启动主节点的MongoDB服务，通过log可以看到已经变成备节点

2018-05-18T14:39:25.030+0800 I REPL     [replexec-10] Member 192.168.112.101:27017 is now in state SECONDARY

通过rs.status()也可以看到服务器的状态。

参考资料

MongoDB Manual
高可用的MongoDB集群

转载请注明原文地址：https://blog.csdn.net/solihawk/article/details/116203851
文章会同步在公众号“牧羊人的方向”更新，感兴趣的可以关注公众号，谢谢！