Centos7配置TiDB集群
一:各模块属性
模块名称 | 状态 | 建议实例数 | 功能 | 负载均衡组件 |
TiDB | 无状态 | 2 | 接收SQL请求,处理SQL相关逻辑,并通过PB找到存储数据的TiKV地址 | LVS、HAProxy、F5 |
PB | 集群 | 3 奇数个节点,推荐>3 | 整个集群的管理模块,存储元信息、对TiKV集群进行调度和负载均衡、分配全局事务ID | Raft |
TiKV | 集群 | 3 | 负责存储数据 | Raft |
二:环境要求
2.1:开发测试环境:
组件 | CPU | 内存 | 本地存储 | 网络 | 实例数量(最低要求) |
---|---|---|---|---|---|
TiDB | 8核+ | 16 GB+ | 无特殊要求 | 千兆网卡 | 1(可与 PD 同机器) |
PD | 4核+ | 8 GB+ | SAS, 200 GB+ | 千兆网卡 | 1(可与 TiDB 同机器) |
TiKV | 8核+ | 32 GB+ | SSD, 200 GB+ | 千兆网卡 | 3 |
2.2:生产环境:
组件 | CPU | 内存 | 硬盘类型 | 网络 | 实例数量(最低要求) |
---|---|---|---|---|---|
TiDB | 16核+ | 32 GB+ | SAS | 万兆网卡(2块最佳) | 2 |
PD | 4核+ | 8 GB+ | SSD | 万兆网卡(2块最佳) | 3 |
TiKV | 16核+ | 32 GB+ | SSD | 万兆网卡(2块最佳) | 3 |
监控 | 8核+ | 16 GB+ | SAS | 千兆网卡 | 1 |
2.3:端口说明:
组件 | 默认端口 | 说明 |
---|---|---|
TiDB | 4000 | 应用及 DBA 工具访问通信端口 |
TiDB | 10080 | TiDB 状态信息上报通信端口 |
TiKV | 20160 | TiKV 通信端口 |
PD | 2379 | 提供 TiDB 和 PD 通信端口 |
PD | 2380 | PD 集群节点间通信端口 |
Pump | 8250 | Pump 通信端口 |
Drainer | 8249 | Drainer 通信端口 |
Prometheus | 9090 | Prometheus 服务通信端口 |
Pushgateway | 9091 | TiDB,TiKV,PD 监控聚合和上报端口 |
Node_exporter | 9100 | TiDB 集群每个节点的系统信息上报通信端口 |
Blackbox_exporter | 9115 | Blackbox_exporter 通信端口,用于 TiDB 集群端口监控 |
Grafana | 3000 | Web 监控服务对外服务和客户端(浏览器)访问端口 |
Grafana | 8686 | grafana_collector 通信端口,用于将 Dashboard 导出为 PDF 格式 |
Kafka_exporter | 9308 | Kafka_exporter 通信端口,用于监控 binlog kafka 集群 |
三:环境部署:
3.1:群架构
xm-tidb-01 192.168.1.1
xm-pd-01 192.168.1.2
xm-tikv-01 192.168.1.3
xm-tikv-02 192.168.1.4
xm-tikv-03 192.168.1.5
3.2:创建ext4盘:(所有节点)
参照:https://www.cnblogs.com/jackyzm/p/10402275.html
vim mount-ext4.sh
#!/bin/sh
#https://www.cnblogs.com/jackyzm/p/10402275.html
##缩小home空间到5G
mkdir /homebak
sleep 1
cp -r /home /homebak
sleep 5
umount /home
lvremove /dev/mapper/centos-home -y
sleep 3
lvcreate -L 5G -n home centos -y
sleep 3
mkfs.xfs /dev/mapper/centos-home
sleep 15
mount /dev/mapper/centos-home /home
##新建ext4分区
lvcreate -L 20G -n ext4 centos -y
sleep 3
mkfs.ext4 /dev/mapper/centos-ext4
sleep 15
lsblk -f
./mount-ext4.sh
vim /etc/fstab 添加
UUID=a45530a9-8b07-4b9e-b78b-d6480e239dea /ext4 ext4 defaults,nodelalloc,noatime 0 2
mkdir /ext4
mount -a
mount -t ext4
3.3:安装依赖包(在主控机操作xm-tidb-01 192.168.1.1)
yum -y install epel-release git curl sshpass
yum install -y python-pip
升级pip
pip install --upgrade pip
[root@zz-01 /]# pip -V
pip 19.0.2 from /usr/lib/python2.7/site-packages/pip (python 2.7)
3.4:创建tidb用户
useradd -m -d /home/tidb tidb
passwd tidb
3.5:配置tidb用户sudo免密码
vim /etc/sudoers
添加 tidb ALL=(ALL) NOPASSWD:ALL 到末尾
用:wq!保持并推出
3.6:切换用户
su - tidb
3.6:创建tidb用户ssh key
ssh-keygen -t rsa
[tidb@xm-tidb-01 ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/tidb/.ssh/id_rsa):
Created directory '/home/tidb/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/tidb/.ssh/id_rsa.
Your public key has been saved in /home/tidb/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:7oWdyfPJIGh10V8oJR3FB2BbSFJp9W3ZX4G/8leAl/4 tidb@xm-tidb-01
The key's randomart image is:
+---[RSA 2048]----+
| .o**B*.|
| ++*.oB|
| ..+.ooO|
| ..o++o|
| S . o..o|
| + = o ....|
| o + O o..|
| . . o = . .E|
| . + .|
+----[SHA256]-----+
四:在中控机部署TiDB-Ansible
4.1:各版本对应关系
tidb-ansible 分支 | TiDB 版本 | 备注 |
---|---|---|
release-2.0 | 2.0 版本 | 最新 2.0 稳定版本,可用于生产环境。 |
release-2.1 | 2.1 版本 | 最新 2.1 稳定版本,可用于生产环境(建议)。 |
master | master 版本 | 包含最新特性,每日更新。 |
以tidb用户进入/home/tidb
4.2:下载对应tidb-ansible版本:(本例下载2.1版本)
release-2.0版本:git clone -b release-2.0 https://github.com/pingcap/tidb-ansible.git
release-2.1版本:git clone -b release-2.1 https://github.com/pingcap/tidb-ansible.git
master版本: git clone https://github.com/pingcap/tidb-ansible.git
4.3:安装ansible及其依赖:
cd /home/tidb/tidb-ansible/
确定pip版本为19.0.2以上pip -V
[tidb@xm-tidb-01 tidb-ansible]$ pip -V
pip 19.0.2 from /usr/lib/python2.7/site-packages/pip (python 2.7)
sudo pip install -r ./requirements.txt
ansible --version
[tidb@xm-tidb-01 tidb-ansible]$ ansible --version
ansible 2.6.13config file = /home/tidb/tidb-ansible/ansible.cfgconfigured module search path = [u'/home/tidb/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']ansible python module location = /usr/lib/python2.7/site-packages/ansibleexecutable location = /bin/ansiblepython version = 2.7.5 (default, Oct 30 2018, 23:45:53) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]
五:部署集群ssh互信及sudo规则:(tidb用户在中控机上操作)
5.1:添加列表
cd /home/tidb/tidb-ansible/
vim hosts.ini
[servers]
192.168.10.221
192.168.10.222
192.168.10.223
192.168.10.224
192.168.10.225[all:vars]
username = tidb
ntp_server = pool.ntp.org
5.2: 执行以下命令,按提示输入部署目标机器 root
用户密码。该步骤将在部署目标机器上创建 tidb
用户,并配置 sudo 规则,配置中控机与部署目标机器之间的 ssh 互信。
ansible-playbook -i hosts.ini create_users.yml -u root -k
[tidb@xm-tidb-01 tidb-ansible]$ ansible-playbook -i hosts.ini create_users.yml -u root -k
SSH password: PLAY [all] *************************************************************************************************TASK [create user] *****************************************************************************************
changed: [192.168.10.225]
changed: [192.168.10.224]
changed: [192.168.10.223]
changed: [192.168.10.222]
ok: [192.168.10.221]TASK [set authorized key] **********************************************************************************
changed: [192.168.10.221]
changed: [192.168.10.225]
changed: [192.168.10.224]
changed: [192.168.10.222]
changed: [192.168.10.223]TASK [update sudoers file] *********************************************************************************
changed: [192.168.10.221]
changed: [192.168.10.223]
changed: [192.168.10.224]
changed: [192.168.10.222]
changed: [192.168.10.225]PLAY RECAP *************************************************************************************************
192.168.10.221 : ok=3 changed=2 unreachable=0 failed=0
192.168.10.222 : ok=3 changed=3 unreachable=0 failed=0
192.168.10.223 : ok=3 changed=3 unreachable=0 failed=0
192.168.10.224 : ok=3 changed=3 unreachable=0 failed=0
192.168.10.225 : ok=3 changed=3 unreachable=0 failed=0 Congrats! All goes well. :-)
六:在目标机上安装NTP服务
6.1:cd /home/tidb/tidb-ansible
6.2:ansible-playbook -i hosts.ini deploy_ntp.yml -u tidb -b
如机器未装ntp服务,脚本会自动安装
[tidb@xm-tidb-01 tidb-ansible]$ ansible-playbook -i hosts.ini deploy_ntp.yml -u tidb -bPLAY [all] *************************************************************************************************TASK [get facts] *******************************************************************************************
ok: [192.168.10.221]
ok: [192.168.10.225]
ok: [192.168.10.224]
ok: [192.168.10.222]
ok: [192.168.10.223]TASK [RedHat family Linux distribution - make sure ntp, ntpstat have been installed] ***********************
changed: [192.168.10.221] => (item=[u'ntp'])
changed: [192.168.10.223] => (item=[u'ntp'])
changed: [192.168.10.224] => (item=[u'ntp'])
changed: [192.168.10.225] => (item=[u'ntp'])
changed: [192.168.10.222] => (item=[u'ntp'])TASK [RedHat family Linux distribution - make sure ntpdate have been installed] ****************************
ok: [192.168.10.221] => (item=[u'ntpdate'])
ok: [192.168.10.222] => (item=[u'ntpdate'])
ok: [192.168.10.223] => (item=[u'ntpdate'])
ok: [192.168.10.224] => (item=[u'ntpdate'])
ok: [192.168.10.225] => (item=[u'ntpdate'])TASK [Debian family Linux distribution - make sure ntp, ntpstat have been installed] ***********************TASK [Debian family Linux distribution - make sure ntpdate have been installed] ****************************TASK [RedHat family Linux distribution - make sure ntpd service has been stopped] **************************
ok: [192.168.10.221]
ok: [192.168.10.225]
ok: [192.168.10.222]
ok: [192.168.10.223]
ok: [192.168.10.224]TASK [Debian family Linux distribution - make sure ntp service has been stopped] ***************************TASK [Adjust Time | start to adjust time with pool.ntp.org] ************************************************
changed: [192.168.10.222]
changed: [192.168.10.224]
changed: [192.168.10.223]
changed: [192.168.10.221]
changed: [192.168.10.225]TASK [RedHat family Linux distribution - make sure ntpd service has been started] **************************
changed: [192.168.10.221]
changed: [192.168.10.222]
changed: [192.168.10.223]
changed: [192.168.10.224]
changed: [192.168.10.225]TASK [Debian family Linux distribution - Make sure ntp service has been started] ***************************PLAY RECAP *************************************************************************************************
192.168.10.221 : ok=6 changed=3 unreachable=0 failed=0
192.168.10.222 : ok=6 changed=3 unreachable=0 failed=0
192.168.10.223 : ok=6 changed=3 unreachable=0 failed=0
192.168.10.224 : ok=6 changed=3 unreachable=0 failed=0
192.168.10.225 : ok=6 changed=3 unreachable=0 failed=0 Congrats! All goes well. :-)
七:在部署目标机配置cpufreq调节器模式:
7.1查看调节器模式
cpupower frequency-info --governors
[tidb@xm-tidb-01 tidb-ansible]$ cpupower frequency-info --governors
analyzing CPU 0:available cpufreq governors: Not Available
返回 “Not Available”,表示当前系统不支持配置 CPUfreq,跳过该步骤即可。
八:分配机器资源,编辑inventory.ini文件:
8.1:vim /home/tidb/tidb-ansible/inventory.ini
## TiDB Cluster Part
[tidb_servers]
192.168.10.221[tikv_servers]
192.168.10.223
192.168.10.224
192.168.10.225[pd_servers]
192.168.10.222[spark_master][spark_slaves][lightning_server][importer_server]## Monitoring Part
# prometheus and pushgateway servers
[monitoring_servers]
192.168.10.221[grafana_servers]
192.168.10.221# node_exporter and blackbox_exporter servers
[monitored_servers]
192.168.10.221
192.168.10.222
192.168.10.223
192.168.10.224
192.168.10.225[alertmanager_servers][kafka_exporter_servers]## Binlog Part
[pump_servers][drainer_servers]
8.2:inventory.ini变量调整:
部署目录调整:
## Global variables
[all:vars]
#deploy_dir = /home/tidb/deploy
deploy_dir = /ext4/deploy
如为某一服务单独设置部署目录,可在配置服务主机列表时配置主机变量,以 TiKV 节点为例,其他服务类推,请务必添加第一列别名,以免服务混布时混淆
TiKV1-1 ansible_host=172.16.10.4 deploy_dir=/data1/deploy
8.3:其他变量调整:True、False首字母要大写
变量 | 含义 |
---|---|
cluster_name | 集群名称,可调整 |
tidb_version | TiDB 版本,TiDB-Ansible 各分支默认已配置 |
process_supervision | 进程监管方式,默认为 systemd,可选 supervise |
timezone |
新安装 TiDB 集群第一次启动 bootstrap(初始化)时,将 TiDB 全局默认时区设置为该值。TiDB 使用的时区后续可通过 time_zone 全局变量和 session 变量来修改,参考时区支持。 默认为 Asia/Shanghai ,可选值参考 timzone 列表。
|
enable_firewalld | 开启防火墙,默认不开启,如需开启,请将部署建议-网络要求 中的端口加入白名单 |
enable_ntpd | 检测部署目标机器 NTP 服务,默认为 True,请勿关闭 |
set_hostname | 根据 IP 修改部署目标机器主机名,默认为 False |
enable_binlog |
是否部署 pump 并开启 binlog,默认为 False,依赖 Kafka 集群,参见 zookeeper_addrs 变量
|
zookeeper_addrs | binlog Kafka 集群的 zookeeper 地址 |
enable_slow_query_log | TiDB 慢查询日志记录到单独文件({{ deploy_dir }}/log/tidb_slow_query.log),默认为 False,记录到 tidb 日志 |
deploy_without_tidb |
KV 模式,不部署 TiDB 服务,仅部署 PD、TiKV 及监控服务,请将 inventory.ini 文件中 tidb_servers 主机组 IP 设置为空。
|
alertmanager_target | 可选:如果你已单独部署 alertmanager,可配置该变量,格式:alertmanager_host:alertmanager_port |
grafana_admin_user | Grafana 管理员帐号用户名,默认为 admin |
grafana_admin_password | Grafana 管理员帐号密码,默认为 admin,用于 Ansible 导入 Dashboard 和创建 API Key,如后期通过 grafana web 修改了密码,请更新此变量 |
collect_log_recent_hours | 采集日志时,采集最近几个小时的日志,默认为 2 小时 |
enable_bandwidth_limit | 在中控机上从部署目标机器拉取诊断数据时,是否限速,默认为 True,与 collect_bandwidth_limit 变量结合使用 |
collect_bandwidth_limit | 在中控机上从部署目标机器拉取诊断数据时限速多少,单位: Kbit/s,默认 10000,即 10Mb/s,如果是单机多 TiKV 实例部署方式,需除以单机实例个数 |
九:部署任务:
9.1:确认inventory.ini中ansible_user = tidb
## Connection
# ssh via normal user
ansible_user = tidb
9.2:测试ssh互信
ansible -i inventory.ini all -m shell -a 'whoami'
[tidb@xm-tidb-01 tidb-ansible]$ ansible -i inventory.ini all -m shell -a 'whoami'
192.168.10.224 | SUCCESS | rc=0 >>
tidb192.168.10.223 | SUCCESS | rc=0 >>
tidb192.168.10.221 | SUCCESS | rc=0 >>
tidb192.168.10.222 | SUCCESS | rc=0 >>
tidb192.168.10.225 | SUCCESS | rc=0 >>
tidb
9.3:测试sudo免密
ansible -i inventory.ini all -m shell -a 'whoami' -b
[tidb@xm-tidb-01 tidb-ansible]$ ansible -i inventory.ini all -m shell -a 'whoami' -b
192.168.10.224 | SUCCESS | rc=0 >>
root192.168.10.223 | SUCCESS | rc=0 >>
root192.168.10.221 | SUCCESS | rc=0 >>
root192.168.10.222 | SUCCESS | rc=0 >>
root192.168.10.225 | SUCCESS | rc=0 >>
root
9.4:下载TiDB binary到中控机:
ansible-playbook local_prepare.yml
[tidb@xm-tidb-01 tidb-ansible]$ ansible-playbook local_prepare.ymlPLAY [do local preparation] ********************************************************************************TASK [local : Stop if ansible version is too low, make sure that the Ansible version is Ansible 2.4.2 or later, otherwise a compatibility issue occurs.] ***
ok: [localhost] => {"changed": false, "msg": "All assertions passed"
}TASK [local : create downloads and resources directories] **************************************************
changed: [localhost] => (item=/home/tidb/tidb-ansible/downloads)
changed: [localhost] => (item=/home/tidb/tidb-ansible/resources)
changed: [localhost] => (item=/home/tidb/tidb-ansible/resources/bin)TASK [local : create cert directory] ***********************************************************************TASK [local : create packages.yml] *************************************************************************
changed: [localhost]TASK [local : create specific deployment method packages.yml] **********************************************
changed: [localhost]TASK [local : include_vars] ********************************************************************************
ok: [localhost]TASK [local : include_vars] ********************************************************************************
ok: [localhost]TASK [local : detect outbound network] *********************************************************************
ok: [localhost]TASK [local : set outbound network fact] *******************************************************************
ok: [localhost]TASK [local : fail] ****************************************************************************************TASK [local : detect GFW] **********************************************************************************
ok: [localhost]TASK [local : set GFW fact] ********************************************************************************
ok: [localhost]TASK [local : download tidb binary] ************************************************************************
FAILED - RETRYING: download tidb binary (4 retries left).
changed: [localhost] => (item={u'url': u'http://download.pingcap.org/tidb-v2.1.4-linux-amd64.tar.gz', u'version': u'v2.1.4', u'name': u'tidb'})TASK [local : download common binary] **********************************************************************
changed: [localhost] => (item={u'url': u'http://download.pingcap.org/fio-3.8.tar.gz', u'checksum': u'sha256:15739abde7e74b59ac59df57f129b14fc5cd59e1e2eca2ce37b41f8c289c3d58', u'version': 3.8, u'name': u'fio'})
changed: [localhost] => (item={u'url': u'http://download.pingcap.org/grafana_collector-latest-linux-amd64.tar.gz', u'version': u'latest', u'name': u'grafana_collector'})
changed: [localhost] => (item={u'url': u'http://download.pingcap.org/kafka_exporter-1.1.0.linux-amd64.tar.gz', u'version': u'1.1.0', u'name': u'kafka_exporter'})TASK [local : download diagnosis tools] ********************************************************************
changed: [localhost] => (item={u'url': u'http://download.pingcap.org/tidb-insight-v0.2.5-1-g99b8fea.tar.gz', u'version': u'v0.2.5-1-g99b8fea', u'name': u'tidb-insight'})TASK [local : download cfssl binary] ***********************************************************************TASK [local : download cfssljson binary] *******************************************************************TASK [local : include_tasks] *******************************************************************************
included: /home/tidb/tidb-ansible/roles/local/tasks/binary_deployment.yml for localhostTASK [local : download other binary] ***********************************************************************TASK [local : download other binary under gfw] *************************************************************
changed: [localhost] => (item={u'url': u'http://download.pingcap.org/prometheus-2.2.1.linux-amd64.tar.gz', u'version': u'2.2.1', u'name': u'prometheus'})
changed: [localhost] => (item={u'url': u'http://download.pingcap.org/alertmanager-0.14.0.linux-amd64.tar.gz', u'version': u'0.14.0', u'name': u'alertmanager'})
changed: [localhost] => (item={u'url': u'http://download.pingcap.org/node_exporter-0.15.2.linux-amd64.tar.gz', u'version': u'0.15.2', u'name': u'node_exporter'})
changed: [localhost] => (item={u'url': u'http://download.pingcap.org/pushgateway-0.4.0.linux-amd64.tar.gz', u'version': u'0.4.0', u'name': u'pushgateway'})
changed: [localhost] => (item={u'url': u'http://download.pingcap.org/grafana-4.6.3.linux-x64.tar.gz', u'version': u'4.6.3', u'name': u'grafana'})
changed: [localhost] => (item={u'url': u'http://download.pingcap.org/blackbox_exporter-0.12.0.linux-amd64.tar.gz', u'version': u'0.12.0', u'name': u'blackbox_exporter'})TASK [local : download TiSpark packages] *******************************************************************
changed: [localhost] => (item={u'url': u'http://download.pingcap.org/spark-2.3.2-bin-hadoop2.7.tgz', u'checksum': u'sha256:6246b20d95c7596a29fb26d5b50a3ae3163a35915bec6c515a8e183383bedc43', u'version': u'2.3.2', u'name': u'spark-2.3.2-bin-hadoop2.7.tgz'})
changed: [localhost] => (item={u'url': u'http://download.pingcap.org/tispark-latest-linux-amd64.tar.gz', u'version': u'latest', u'name': u'tispark-latest.tar.gz'})
changed: [localhost] => (item={u'url': u'http://download.pingcap.org/tispark-sample-data.tar.gz', u'version': u'latest', u'name': u'tispark-sample-data.tar.gz'})TASK [local : unarchive third party binary] ****************************************************************
changed: [localhost] => (item={u'url': u'https://github.com/prometheus/prometheus/releases/download/v2.2.1/prometheus-2.2.1.linux-amd64.tar.gz', u'version': u'2.2.1', u'name': u'prometheus'})
changed: [localhost] => (item={u'url': u'https://github.com/prometheus/alertmanager/releases/download/v0.14.0/alertmanager-0.14.0.linux-amd64.tar.gz', u'version': u'0.14.0', u'name': u'alertmanager'})
changed: [localhost] => (item={u'url': u'https://github.com/prometheus/node_exporter/releases/download/v0.15.2/node_exporter-0.15.2.linux-amd64.tar.gz', u'version': u'0.15.2', u'name': u'node_exporter'})
changed: [localhost] => (item={u'url': u'https://github.com/prometheus/blackbox_exporter/releases/download/v0.12.0/blackbox_exporter-0.12.0.linux-amd64.tar.gz', u'version': u'0.12.0', u'name': u'blackbox_exporter'})
changed: [localhost] => (item={u'url': u'https://github.com/prometheus/pushgateway/releases/download/v0.4.0/pushgateway-0.4.0.linux-amd64.tar.gz', u'version': u'0.4.0', u'name': u'pushgateway'})
changed: [localhost] => (item={u'url': u'https://s3-us-west-2.amazonaws.com/grafana-releases/release/grafana-4.6.3.linux-x64.tar.gz', u'version': u'4.6.3', u'name': u'grafana'})TASK [local : unarchive tispark] ***************************************************************************
changed: [localhost]TASK [local : unarchive tispark-sample-data] ***************************************************************
changed: [localhost]TASK [local : cp monitoring binary] ************************************************************************
changed: [localhost] => (item=alertmanager)
changed: [localhost] => (item=prometheus)
changed: [localhost] => (item=node_exporter)
changed: [localhost] => (item=pushgateway)
changed: [localhost] => (item=blackbox_exporter)TASK [local : cp tispark] **********************************************************************************
changed: [localhost]TASK [local : cp tispark-sample-data] **********************************************************************
changed: [localhost]TASK [local : unarchive tidb binary] ***********************************************************************
changed: [localhost] => (item={u'url': u'http://download.pingcap.org/tidb-v2.1.4-linux-amd64.tar.gz', u'version': u'v2.1.4', u'name': u'tidb'})TASK [local : unarchive common binary] *********************************************************************
changed: [localhost] => (item={u'url': u'http://download.pingcap.org/fio-3.8.tar.gz', u'checksum': u'sha256:15739abde7e74b59ac59df57f129b14fc5cd59e1e2eca2ce37b41f8c289c3d58', u'version': 3.8, u'name': u'fio'})
changed: [localhost] => (item={u'url': u'http://download.pingcap.org/grafana_collector-latest-linux-amd64.tar.gz', u'version': u'latest', u'name': u'grafana_collector'})
changed: [localhost] => (item={u'url': u'http://download.pingcap.org/kafka_exporter-1.1.0.linux-amd64.tar.gz', u'version': u'1.1.0', u'name': u'kafka_exporter'})TASK [local : cp tidb binary] ******************************************************************************
changed: [localhost] => (item={u'url': u'http://download.pingcap.org/tidb-v2.1.4-linux-amd64.tar.gz', u'version': u'v2.1.4', u'name': u'tidb'})TASK [local : cp fio binary] *******************************************************************************
changed: [localhost] => (item=fio)TASK [local : cp grafana_collector binary and fonts] *******************************************************
changed: [localhost]TASK [local : cp kafka_exporter binary] ********************************************************************
changed: [localhost] => (item=kafka_exporter)TASK [local : cp daemontools binary] ***********************************************************************TASK [local : cp tidb-insight tarball] *********************************************************************
changed: [localhost]TASK [local : clean up download dir] ***********************************************************************
changed: [localhost]PLAY RECAP *************************************************************************************************
localhost : ok=30 changed=22 unreachable=0 failed=0
Congrats! All goes well. :-)[tidb@xm-tidb-01 tidb-ansible]$
9.5:初始化系统环境
ansible-playbook bootstrap.yml
报错:内存不足
TASK [check_system_optional : Preflight check - Check TiDB server's RAM] ***********************************
fatal: [192.168.10.221]: FAILED! => {"changed": false, "msg": "This machine does not have sufficient RAM to run TiDB, at least 16000 MB."}NO MORE HOSTS LEFT *****************************************************************************************to retry, use: --limit @/home/tidb/tidb-ansible/retry_files/bootstrap.retryPLAY RECAP *************************************************************************************************
192.168.10.221 : ok=29 changed=10 unreachable=0 failed=1
192.168.10.222 : ok=29 changed=10 unreachable=0 failed=0
192.168.10.223 : ok=29 changed=10 unreachable=0 failed=0
192.168.10.224 : ok=29 changed=10 unreachable=0 failed=0
192.168.10.225 : ok=29 changed=10 unreachable=0 failed=0
localhost : ok=1 changed=0 unreachable=0 failed=0 ERROR MESSAGE SUMMARY **************************************************************************************
[192.168.10.221]: Ansible FAILED! => playbook: bootstrap.yml; TASK: check_system_optional : Preflight check - Check TiDB server's RAM; message: {"changed": false, "msg": "This machine does not have sufficient RAM to run TiDB, at least 16000 MB."}Ask for help:
Contact us: support@pingcap.com
It seems that you encounter some problems. You can send an email to the above email address, attached with the tidb-ansible/inventory.ini and tidb-ansible/log/ansible.log files and the error message, or new issue on https://github.com/pingcap/tidb-ansible/issues. We'll try our best to help you deploy a TiDB cluster. Thanks. :-)
[tidb@xm-tidb-01 tidb-ansible]$
修改文件:
vim bootstrap.yml
注销掉:
- { role: check_system_optional, when: not dev_mode|default(false) }
- { role: machine_benchmark, when: not dev_mode|default(false) }
- name: check systemhosts: allany_errors_fatal: trueroles:- check_system_static
# - { role: check_system_optional, when: not dev_mode|default(false) }- name: tikv_servers machine benchmarkhosts: tikv_serversgather_facts: falseroles:
# - { role: machine_benchmark, when: not dev_mode|default(false) }
再次运行:
ansible-playbook bootstrap.yml
[tidb@xm-tidb-01 tidb-ansible]$ ansible-playbook bootstrap.ymlPLAY [initializing deployment target] **********************************************************************TASK [check_config_static : Ensure only one monitoring host exists] ****************************************TASK [check_config_static : Ensure monitored_servers exists] ***********************************************TASK [check_config_static : Ensure TiDB host exists] *******************************************************TASK [check_config_static : Ensure PD host exists] *********************************************************TASK [check_config_static : Ensure TiKV host exists] *******************************************************TASK [check_config_static : Check ansible_user variable] ***************************************************TASK [check_config_static : Ensure timezone variable is set] ***********************************************TASK [check_config_static : Close old SSH control master processes] ****************************************
ok: [localhost]PLAY [check node config] ***********************************************************************************TASK [pre-ansible : disk space check - fail when disk is full] *********************************************
ok: [192.168.10.221]
ok: [192.168.10.222]
ok: [192.168.10.223]
ok: [192.168.10.224]
ok: [192.168.10.225]TASK [pre-ansible : Get distro name from /etc/os-release] **************************************************
ok: [192.168.10.221]
ok: [192.168.10.222]
ok: [192.168.10.223]
ok: [192.168.10.224]
ok: [192.168.10.225]TASK [pre-ansible : set distro facts] **********************************************************************
ok: [192.168.10.221]
ok: [192.168.10.222]
ok: [192.168.10.223]
ok: [192.168.10.224]
ok: [192.168.10.225]TASK [pre-ansible : python check] **************************************************************************
ok: [192.168.10.221]
ok: [192.168.10.222]
ok: [192.168.10.223]
ok: [192.168.10.224]
ok: [192.168.10.225]TASK [pre-ansible : set has_python facts] ******************************************************************
ok: [192.168.10.221]
ok: [192.168.10.222]
ok: [192.168.10.223]
ok: [192.168.10.224]
ok: [192.168.10.225]TASK [pre-ansible : set has_python facts] ******************************************************************TASK [pre-ansible : include_tasks] *************************************************************************TASK [pre-ansible : include_tasks] *************************************************************************
included: /home/tidb/tidb-ansible/roles/pre-ansible/tasks/root_tasks.yml for 192.168.10.221, 192.168.10.222, 192.168.10.223, 192.168.10.224, 192.168.10.225TASK [pre-ansible : Debian/Ubuntu - install python] ********************************************************TASK [pre-ansible : Redhat/CentOS - install python] ********************************************************TASK [pre-ansible : Redhat/CentOS - Make sure ntp, ntpstat have been installed] ****************************
ok: [192.168.10.221] => (item=[u'ntp'])
ok: [192.168.10.222] => (item=[u'ntp'])
ok: [192.168.10.224] => (item=[u'ntp'])
ok: [192.168.10.223] => (item=[u'ntp'])
ok: [192.168.10.225] => (item=[u'ntp'])TASK [pre-ansible : Debian/Ubuntu - Make sure ntp, ntpstat have been installed] ****************************TASK [bootstrap : gather facts] ****************************************************************************
ok: [192.168.10.225]
ok: [192.168.10.222]
ok: [192.168.10.221]
ok: [192.168.10.223]
ok: [192.168.10.224]TASK [bootstrap : group hosts by distribution] *************************************************************
ok: [192.168.10.221]
ok: [192.168.10.222]
ok: [192.168.10.223]
ok: [192.168.10.224]
ok: [192.168.10.225]TASK [bootstrap : Set deploy_dir if not presented] *********************************************************TASK [bootstrap : include_tasks] ***************************************************************************
included: /home/tidb/tidb-ansible/roles/bootstrap/tasks/root_tasks.yml for 192.168.10.221, 192.168.10.222, 192.168.10.223, 192.168.10.224, 192.168.10.225TASK [bootstrap : setting absent kernel params] ************************************************************
ok: [192.168.10.222] => (item={u'name': u'net.ipv4.tcp_tw_recycle', u'value': 0})
ok: [192.168.10.225] => (item={u'name': u'net.ipv4.tcp_tw_recycle', u'value': 0})
ok: [192.168.10.223] => (item={u'name': u'net.ipv4.tcp_tw_recycle', u'value': 0})
ok: [192.168.10.221] => (item={u'name': u'net.ipv4.tcp_tw_recycle', u'value': 0})
ok: [192.168.10.224] => (item={u'name': u'net.ipv4.tcp_tw_recycle', u'value': 0})TASK [bootstrap : setting present kernel params] ***********************************************************
ok: [192.168.10.221] => (item={u'name': u'net.core.somaxconn', u'value': 32768})
ok: [192.168.10.223] => (item={u'name': u'net.core.somaxconn', u'value': 32768})
ok: [192.168.10.224] => (item={u'name': u'net.core.somaxconn', u'value': 32768})
ok: [192.168.10.225] => (item={u'name': u'net.core.somaxconn', u'value': 32768})
ok: [192.168.10.222] => (item={u'name': u'net.core.somaxconn', u'value': 32768})
ok: [192.168.10.221] => (item={u'name': u'vm.swappiness', u'value': 0})
ok: [192.168.10.223] => (item={u'name': u'vm.swappiness', u'value': 0})
ok: [192.168.10.225] => (item={u'name': u'vm.swappiness', u'value': 0})
ok: [192.168.10.224] => (item={u'name': u'vm.swappiness', u'value': 0})
ok: [192.168.10.222] => (item={u'name': u'vm.swappiness', u'value': 0})
ok: [192.168.10.221] => (item={u'name': u'net.ipv4.tcp_syncookies', u'value': 0})
ok: [192.168.10.223] => (item={u'name': u'net.ipv4.tcp_syncookies', u'value': 0})
ok: [192.168.10.225] => (item={u'name': u'net.ipv4.tcp_syncookies', u'value': 0})
ok: [192.168.10.224] => (item={u'name': u'net.ipv4.tcp_syncookies', u'value': 0})
ok: [192.168.10.222] => (item={u'name': u'net.ipv4.tcp_syncookies', u'value': 0})
ok: [192.168.10.221] => (item={u'name': u'fs.file-max', u'value': 1000000})
ok: [192.168.10.223] => (item={u'name': u'fs.file-max', u'value': 1000000})
ok: [192.168.10.224] => (item={u'name': u'fs.file-max', u'value': 1000000})
ok: [192.168.10.222] => (item={u'name': u'fs.file-max', u'value': 1000000})
ok: [192.168.10.225] => (item={u'name': u'fs.file-max', u'value': 1000000})TASK [bootstrap : update /etc/security/limits.conf] ********************************************************
ok: [192.168.10.221]
ok: [192.168.10.224]
ok: [192.168.10.223]
ok: [192.168.10.225]
ok: [192.168.10.222]TASK [bootstrap : disable swap] ****************************************************************************TASK [bootstrap : create group] ****************************************************************************
ok: [192.168.10.221]
ok: [192.168.10.222]
ok: [192.168.10.224]
ok: [192.168.10.225]
ok: [192.168.10.223]TASK [bootstrap : create account] **************************************************************************
ok: [192.168.10.222]
ok: [192.168.10.221]
ok: [192.168.10.224]
ok: [192.168.10.225]
ok: [192.168.10.223]TASK [bootstrap : create top deploy dir when under root] ***************************************************
ok: [192.168.10.221]
ok: [192.168.10.224]
ok: [192.168.10.223]
ok: [192.168.10.222]
ok: [192.168.10.225]TASK [bootstrap : create wal_dir deploy dir when under root] ***********************************************TASK [bootstrap : create raftdb_path deploy dir when under root] *******************************************TASK [bootstrap : set hostname if hostname is not distinguishable] *****************************************TASK [bootstrap : set hostname in hosts file] **************************************************************TASK [bootstrap : determine if firewalld is running] *******************************************************
ok: [192.168.10.223]
ok: [192.168.10.221]
ok: [192.168.10.225]
ok: [192.168.10.224]
ok: [192.168.10.222]TASK [bootstrap : disable firewalld] ***********************************************************************TASK [bootstrap : or to enable firewalld] ******************************************************************TASK [bootstrap : check centos configuration file exists] **************************************************
ok: [192.168.10.222]
ok: [192.168.10.221]
ok: [192.168.10.225]
ok: [192.168.10.224]
ok: [192.168.10.223]TASK [bootstrap : check debian configuration file exists] **************************************************
ok: [192.168.10.221]
ok: [192.168.10.222]
ok: [192.168.10.223]
ok: [192.168.10.224]
ok: [192.168.10.225]TASK [bootstrap : modify centos irqbalance configuration file] *********************************************
ok: [192.168.10.221]
ok: [192.168.10.224]
ok: [192.168.10.223]
ok: [192.168.10.225]
ok: [192.168.10.222]TASK [bootstrap : modify debian irqbalance configuration file] *********************************************TASK [bootstrap : start irqbalance service] ****************************************************************
ok: [192.168.10.221]
ok: [192.168.10.222]
ok: [192.168.10.224]
ok: [192.168.10.225]
ok: [192.168.10.223]PLAY [check system] ****************************************************************************************TASK [check_system_static : Disk space check - Fail task when disk is full] ********************************
ok: [192.168.10.221]
ok: [192.168.10.222]
ok: [192.168.10.223]
ok: [192.168.10.224]
ok: [192.168.10.225]TASK [check_system_static : get facts] *********************************************************************
ok: [192.168.10.221]
ok: [192.168.10.225]
ok: [192.168.10.224]
ok: [192.168.10.222]
ok: [192.168.10.223]TASK [check_system_static : Preflight check - Linux OS family and distribution version] ********************TASK [check_system_static : Deploy check_cpufreq script] ***************************************************
changed: [192.168.10.221]
changed: [192.168.10.224]
changed: [192.168.10.223]
changed: [192.168.10.225]
changed: [192.168.10.222]TASK [check_system_static : Preflight check - Check CPUfreq governors available in the kernel] *************
changed: [192.168.10.221]
changed: [192.168.10.222]
changed: [192.168.10.223]
changed: [192.168.10.224]
changed: [192.168.10.225]TASK [check_system_static : Preflight check - Check the currently active governor] *************************
changed: [192.168.10.224]
changed: [192.168.10.223]
changed: [192.168.10.222]
changed: [192.168.10.225]
changed: [192.168.10.221]TASK [check_system_static : Preflight check - Fail when CPU frequency governor is not set to performance mode] ***TASK [check_system_static : Clean check_cpufreq script] ****************************************************
changed: [192.168.10.221]
changed: [192.168.10.222]
changed: [192.168.10.223]
changed: [192.168.10.224]
changed: [192.168.10.225]TASK [check_system_static : Preflight check - Check Linux kernel overcommit_memory parameter] **************
changed: [192.168.10.221]
changed: [192.168.10.223]
changed: [192.168.10.222]
changed: [192.168.10.225]
changed: [192.168.10.224]TASK [check_system_static : Preflight check - Fail when Linux kernel vm.overcommit_memory parameter is set to 2] ***PLAY [tikv_servers machine benchmark] **********************************************************************PLAY [create ops scripts] **********************************************************************************TASK [ops : create check_tikv.sh script] *******************************************************************
changed: [localhost]TASK [ops : create pd-ctl.sh script] ***********************************************************************
changed: [localhost]PLAY RECAP *************************************************************************************************
192.168.10.221 : ok=28 changed=5 unreachable=0 failed=0
192.168.10.222 : ok=28 changed=5 unreachable=0 failed=0
192.168.10.223 : ok=28 changed=5 unreachable=0 failed=0
192.168.10.224 : ok=28 changed=5 unreachable=0 failed=0
192.168.10.225 : ok=28 changed=5 unreachable=0 failed=0
localhost : ok=3 changed=2 unreachable=0 failed=0 Congrats! All goes well. :-)
9.6:部署TiDB集群软件:
ansible-playbook deploy.yml
过程比较多,请耐心等待
PLAY RECAP *************************************************************************************************
192.168.10.221 : ok=115 changed=60 unreachable=0 failed=0
192.168.10.222 : ok=52 changed=23 unreachable=0 failed=0
192.168.10.223 : ok=60 changed=24 unreachable=0 failed=0
192.168.10.224 : ok=60 changed=24 unreachable=0 failed=0
192.168.10.225 : ok=63 changed=26 unreachable=0 failed=0
localhost : ok=1 changed=0 unreachable=0 failed=0 Congrats! All goes well. :-)
[tidb@xm-tidb-01 tidb-ansible]$
注:Grafana Dashboard 上的 Report 按钮可用来生成 PDF 文件,此功能依赖 fontconfig
包和英文字体。如需使用该功能,登录 grafana_servers 机器,用以下命令安装:
sudo yum install fontconfig open-sans-fonts
9.7:启动集群:
!!!先切换到tidb用户!!!
su tidb
cd /home/tidb/tidb-ansible
ansible-playbook start.yml
PLAY RECAP *************************************************************************************************
192.168.10.221 : ok=33 changed=12 unreachable=0 failed=0
192.168.10.222 : ok=12 changed=3 unreachable=0 failed=0
192.168.10.223 : ok=14 changed=3 unreachable=0 failed=0
192.168.10.224 : ok=14 changed=3 unreachable=0 failed=0
192.168.10.225 : ok=14 changed=3 unreachable=0 failed=0
localhost : ok=1 changed=0 unreachable=0 failed=0 Congrats! All goes well. :-)
[tidb@xm-tidb-01 tidb-ansible]$
9.8: 测试集群:
在安装有mysql的其他服务器做连接测试
mysql -u root -h 192.168.10.221 -P 4000
-u和-h后的参数间不可有空格,否则会报错
mysql -uroot -pxxx -h 192.168.10.221 -P 4000
[root@zabbix ~]# mysql -u root -h 192.168.10.221 -P 4000
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MySQL connection id is 51
Server version: 5.7.10-TiDB-v2.1.4 MySQL Community Server (Apache License 2.0)Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.MySQL [(none)]>
http://192.168.10.221:3000/
admin admin
感谢:
官方文档:
https://pingcap.com/docs-cn/overview/
https://blog.csdn.net/xujiamin0022016/article/details/83507038
转载于:https://www.cnblogs.com/jackyzm/p/10394365.html
Centos7配置TiDB集群相关推荐
- Centos7 安装Docker(v2021 version 20.10.5) 并通过docker-compose运行TiDB集群
Centos7 安装Docker(v2021 version 20.10.5) 卸载docker旧版 注意:新系统不用卸载 yum remove docker \docker-client \dock ...
- Centos7+Elasticsearch7.4.2+Kibana+IK分词+ElasticHD 安装配置以及集群搭建
Centos7+Elasticsearch7.4.2+Kibana+IK分词+ElasticHD 安装配置以及集群搭建 1. elasticSearch 7.4.2 安装 1.1 关于jdk 1.2 ...
- 通过 TiUP 部署 TiDB 集群的拓扑文件配置
通过 TiUP 部署或扩容 TiDB 集群时,需要提供一份拓扑文件(示例)来描述集群拓扑. 同样,修改集群配置也是通过编辑拓扑文件来实现的,区别在于修改配置时仅允许修改部分字段.本文档介绍拓扑文件的各 ...
- 在线安装TIDB集群
在线安装TIDB集群 在线安装TiDB集群 服务器准备 说明:TiDB8需要能够连接外网,以便下载各类安装包 TiDB4非必须,但最好是有一台,因为后续测试Mysql数据同步或者进行性能比较时,都要 ...
- CentOS7下Elasticsearch集群部署记录
Elasticsearch是一个分布式搜索服务,提供Restful API,底层基于Lucene,采用多shard的方式保证数据安全,并且提供自动resharding的功能,github等大型的站点也 ...
- MariaDB配置、集群
MariaDB在centos 7.3的安装,配置和集群搭配 阿里云最新选配系统中,只有centos7.3可选,因此,基于centos 7的MariaDB的安装,配置... 全部删除MySQL/Mari ...
- TiDB集群大规模删除实践
女主宣言 TiDB在写入一段时间后,会存在一些可清理的数据.对这些数据进行删除,可以释放集群的空间.本文描述了在清理数据过程中,遇到的问题和具体的排查步骤,希望对有相同需求的同学有所帮助. PS:丰富 ...
- insert带来的TiDB集群性能瓶颈排障
女主宣言 本文通过分析TiDB线上集群的一次业务问题,整理了故障排查的思路,结合这个案例也可以让有类似潜在威胁的同学提前规避. PS:丰富的一线技术.多元化的表现形式,尽在"360云计算&q ...
- 华为云Centos7搭建hadoop集群二:yum源替换,ssh免密处理,hadoop用户sudo
华为云Centos7搭建hadoop集群二:yum源替换,ssh免密处理,hadoop用户sudo 新建hadoop用户并设置密码,密码不要设置为简单的123456等,我这里已经创建过了hadoop用 ...
最新文章
- Ubuntu修改终端@前面的用户名
- 大数据笔记(三十二)——SparkStreaming集成Kafka与Flume
- Lesson 4.5 梯度下降优化基础:数据归一化与学习率调度
- boost::range模块reversed相关的测试程序
- 如何使用TCP套接字的端口来区分是哪个客户端发起的连接
- mysql设置作业_MySQL-作业1
- 前端 - token 是什么?为什么每次请求头(HEADS)里要携带它?___请求时,为什么要携带token?
- 脚本实现补丁安装自动化
- 1.零基础Java基础教程9天入门
- python中将str转成数字_python初探: 数据类型与变量
- 完全总结bash中的条件判断test [ [[ 使用
- 全栈工程师进阶路线图
- android 获取设备的serialNumber和Mac地址
- node mysql菜鸟教程_Node.js 路由
- 函数接口Inside COM读书笔记-----调度接口与自动化
- 使电动机反转的matlab仿真图,基于MATLAB的电机仿真研究
- 通用软件体系结构风格总结为五个大类
- python DataFrame的shift()方法
- narwal机器人_Narwal云鲸扫地机器人自清洗加压拖,省心省力省时
- Flink on Yarn的两种模式及HA