mysql数据库relay_MySQL 数据库主从复制小知识
1.主从状态查看
mysql> SHOW SLAVE STATUS\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: localhost
Master_User: repl
Master_Port: 13000
Connect_Retry: 60
Master_Log_File: master-bin.000002
Read_Master_Log_Pos: 1307
Relay_Log_File: slave-relay-bin.000003
Relay_Log_Pos: 1508
Relay_Master_Log_File: master-bin.000002
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 1307
Relay_Log_Space: 1858
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 1
Master_UUID: 3e11fa47-71ca-11e1-9e33-c80aa9429562
Master_Info_File: /var/mysqld.2/data/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Reading event from the relay log
Master_Retry_Count: 10
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set: 3e11fa47-71ca-11e1-9e33-c80aa9429562:1-5
Executed_Gtid_Set: 3e11fa47-71ca-11e1-9e33-c80aa9429562:1-5
Auto_Position: 1
Replicate_Rewrite_DB:
Channel_name:
Master_TLS_Version: TLSv1.2
2.主从复制相关的表
Performance Schema 中有些关于主从同步的表,这些表里的数据 和 通过 SHOW SLAVE STATUS 查看的数据部分相同。
类别
表名
表数据描述
connection
replication_connection_configuration
Configuration parameters for connecting to the master.
replication_connection_status
Current status of the connection to the master.
transaction applier
replication_applier_configuration
Configuration parameters for the transaction applier on the slave.
replication_applier_status
Current status of the transaction applier on the slave.
replication_applier_status_by_coordinator
Status of the coordinator thread (empty unless the slave is multithreaded).
replication_applier_status_by_worker
Status of the applier thread or worker threads if the slave is multithreaded.
group members
replication_group_members
Provides network and status information for group members.
replication_group_member_stats
Provides statistical information about group members and transaction in which they participate.
3.SHOW SLAVE STATUS Information In the Other Tables
下面这两种表在数据库MySQL中。
3.1 slave_relay_log_info 表
Line
Table Column
约等于SHOW SLAVE STATUS Column
Description
1
Number_of_lines
[None]
Number of lines in the file or columns in the table
2
Relay_log_name
Relay_Log_File
The name of the current relay log file
3
Relay_log_pos
Relay_Log_Pos
The current position within the relay log file; events up to this position have been executed on the slave database
4
Master_log_name
Relay_Master_Log_File
The name of the master binary log file from which the events in the relay log file were read
5
Master_log_pos
Exec_Master_Log_Pos
The equivalent position within the master's binary log file of events that have already been executed
6
Sql_delay
SQL_Delay
The number of seconds that the slave must lag the master
7
Number_of_workers
[None]
The number of slave worker threads for executing replication events (transactions) in parallel
8
Id
[None]
ID used for internal purposes; currently this is always 1
9
Channel_name
Channel_name
The name of the replication channel
注意:
(1)slave_relay_log_info 表存储的是持久化的状态,而 show slave status 输出的是内存中的状态,所以二者之间的一些指标数据可能不一致;
(2)stop slave 或者正常关闭 mysqld,都会将内存中的状态持久化到磁盘上(slave_relay_log_info表中);启动 mysqld 时会读取磁盘状态,初始化给内存状态;start slave 时也会初始化内存状态。
(3)在从库启动的时候时,读取 slave_relay_log_info 表中存储的位置,并把值传给 “show slave status” 中的 Relay_Log_File、Relay_Log_Pos,下次 “start slave” 是从这个位置开始继续回放 relay log;
(4)slave sql thread 按照 Relay_Log_Name、Relay_Log_Pos 位置进行 realy log 的回放.
3.2 slave_master_info 表
Line
Table Column
约等于SHOW SLAVE STATUS Column
Description
1
Number_of_lines
[None]
Number of lines in the file, or columns in the table
2
Master_log_name
Master_Log_File
The name of the master binary log currently being read from the master
3
Master_log_pos
Read_Master_Log_Pos
The current position within the master binary log that have been read from the master
4
Host
Master_Host
The host name of the master
5
User_name
Master_User
The user name used to connect to the master
6
User_password
Password (not shown by SHOW SLAVE STATUS)
The password used to connect to the master
7
Port
Master_Port
The network port used to connect to the master
8
Connect_retry
Connect_Retry
The period (in seconds) that the slave will wait before trying to reconnect to the master
9
Enabled_ssl
Master_SSL_Allowed
Indicates whether the server supports SSL connections
10
Ssl_ca
Master_SSL_CA_File
The file used for the Certificate Authority (CA) certificate
11
Ssl_capath
Master_SSL_CA_Path
The path to the Certificate Authority (CA) certificates
12
Ssl_cert
Master_SSL_Cert
The name of the SSL certificate file
13
Ssl_cipher
Master_SSL_Cipher
The list of possible ciphers used in the handshake for the SSL connection
14
Ssl_key
Master_SSL_Key
The name of the SSL key file
15
Ssl_verify_server_cert
Master_SSL_Verify_Server_Cert
Whether to verify the server certificate
16
Heartbeat
[None]
Interval between replication heartbeats, in seconds
17
Bind
Master_Bind
Which of the slave's network interfaces should be used for connecting to the master
18
Ignored_server_ids
Replicate_Ignore_Server_Ids
The list of server IDs to be ignored. Note that forIgnored_server_idsthe list of server IDs is preceded by the total number of server IDs to ignore.
19
Uuid
Master_UUID
The master's unique ID
20
Retry_count
Master_Retry_Count
Maximum number of reconnection attempts permitted
21
Ssl_crl
[None]
Path to an SSL certificate revocation-list file
22
Ssl_crlpath
[None]
Path to a directory containing SSL certificate revocation-list files
23
Enabled_auto_position
Auto_position
If autopositioning is in use or not
24
Channel_name
Channel_name
The name of the replication channel
25
Tls_version
Master_TLS_Version
TLS version on master
注意:
slave io thread 按照 Master_Log_File、Read_Master_Log_Pos 位置读取主库的 binlog,并写入到本地 relay log。
4.slave_parallel_workers 对 复制延迟的影响
MySQL 5.7版本已经支持“真正”的并行复制功能,官方称为为enhanced multi-threaded slave(简称MTS),因此复制延迟问题已经得到了极大的改进。
在MySQL 5.7中,设置参数slave_parallel_workers>0并且global.slave_parallel_type=‘LOGICAL_CLOCK’,即可支持一个schema下,slave_parallel_workers个的worker线程并发执行relay log中主库提交的事务。其核心思想:一个组提交的事务都是可以并行回放(配合binary log group commit)。slave机器的relay log中last_committed相同的事务(sequence_num不同)可以并发执行。
注意:
(1)开启MTS功能后,推荐将参数master_info_repostitory设置为TABLE,这样性能可以有50%~80%的提升。这是因为并行复制开启后对于元master.info这个文件的更新将会大幅提升,资源的竞争也会变大。推荐将master_info_repository设置为TABLE,来减小这部分的开销。
(2)Coordinator thread on slave dispatches work across several worker threads。Each worker thread commits trx in isolation。
(3)推荐开启参数 slave_preserve_commit_order 。这是因为仅仅设置LOGICAL_CLOCK会存在问题--此时在slave上应用事务的顺序是无序的,和relay log中记录的事务顺序不一样,这样数据一致性是无法保证的,为了保证事务是按照relay log中记录的顺序来回放,就需要开启参数slave_preserve_commit_order。(The transactions that the slave applies in parallel may commit out of order, unless slave_preserve_commit_order=1.)
(4)设置 slave_parallel_workers 后,需要重启slave 进程
Setting slave_parallel_workers has no immediate effect. The state of the variable applies on all subsequent START SLAVE statements.
(5)实践中遇到的一个例子,我们将 slave_parallel_workers 由值4调整为16,延迟的速率得到了明显的改善。(具体值要操作硬件配置和从库的应用。注意此图是Delay的速率,值越大说明延迟趋势越严重)
5.XtraBackup 恢复过程,对/mysql/data数据文件赋权的时机
应该在 XXX/innobackupex --copy-back /XXX 过程之后再进行数据文件所有者变更,变更主要命令如下:
chown -R mysql:mysql /XXXX/mysql/data/
否则mysql 服务无法正常启动,报错信息如下:
.. ERROR! The server quit without updating PID file (/XXX/mysql/data/tcvm255.pid).
ERROR! MySQL is not running, but lock file (/var/lock/subsys/mysql) exists
查看mysql log 主要信息如下:
2017-06-19T10:48:18.112683+08:00 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for moredetails).2017-06-19T10:48:18.763038+08:00 0 [ERROR] InnoDB: The innodb_system data file 'ibdata1'must be writable2017-06-19T10:48:18.763074+08:00 0 [ERROR] InnoDB: The innodb_system data file 'ibdata1'must be writable2017-06-19T10:48:18.763080+08:00 0[ERROR] InnoDB: Plugin initialization aborted with error Generic error2017-06-19T10:48:19.364031+08:00 0 [ERROR] Plugin 'InnoDB' init functionreturned error.2017-06-19T10:48:19.364063+08:00 0 [ERROR] Plugin 'InnoDB'registration as a STORAGE ENGINE failed.2017-06-19T10:48:19.364075+08:00 0[ERROR] Failed to initialize builtin plugins.2017-06-19T10:48:19.364080+08:00 0 [ERROR] Aborting
6.主从复制如何设置指定的数据库
编辑从服务器上/etc/my.cnf的配置参数,通过参数(replicate-do-db)指定那个数据库可以进行复制;通过参数(replicate-ignore-db),指定那个数据库可以进行复制。
replicate-do-db: Tells the slave SQL thread to restrict replication to the specified database.
replicate-ignore-db: Tells the slave SQL thread not to replicate to the specified database.
7.备份还原,启动实例报错
我们会遇到一种情况,原实例上有多个数据库,但是我们只需要备份还原一个库或部分库的情况。通过innobackupex工具备份指定库,例如 --databases="mysql XXXdb performance_schema",然后再还原,可能报错。
2019-09-24T19:47:09.084293+08:00 0 [Note] /qqweixin/mysql/bin/mysqld (mysqld 5.7.21-log) starting as process 51778...2019-09-24T19:47:09.091489+08:00 0[Note] InnoDB: PUNCH HOLE support available2019-09-24T19:47:09.091559+08:00 0[Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins2019-09-24T19:47:09.091572+08:00 0[Note] InnoDB: Uses event mutexes2019-09-24T19:47:09.091582+08:00 0 [Note] InnoDB: GCC builtin __sync_synchronize() is used formemory barrier2019-09-24T19:47:09.091592+08:00 0 [Note] InnoDB: Compressed tables use zlib 1.2.3
2019-09-24T19:47:09.091601+08:00 0[Note] InnoDB: Using Linux native AIO2019-09-24T19:47:09.096199+08:00 0 [Note] InnoDB: Number of pools: 1
2019-09-24T19:47:09.096473+08:00 0[Note] InnoDB: Using CPU crc32 instructions2019-09-24T19:47:09.101033+08:00 0 [Note] InnoDB: Initializing buffer pool, total size = 220G, instances = 48, chunk size =512M2019-09-24T19:47:10.422726+08:00 0[Note] InnoDB: Completed initialization of buffer pool2019-09-24T19:47:10.650079+08:00 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the manpage of setpriority().2019-09-24T19:47:10.660276+08:00 0 [ERROR] InnoDB: The Auto-extending innodb_system data file './ibdata1' is of a different size 21248 pages (rounded down to MB) than specified in the .cnf file: initial 65536 pages, max 0 (relevant if non-zero) pages!
2019-09-24T19:47:10.660303+08:00 0[ERROR] InnoDB: Plugin initialization aborted with error Generic error2019-09-24T19:47:11.260892+08:00 0 [ERROR] Plugin 'InnoDB' init functionreturned error.2019-09-24T19:47:11.260934+08:00 0 [ERROR] Plugin 'InnoDB'registration as a STORAGE ENGINE failed.2019-09-24T19:47:11.260948+08:00 0[ERROR] Failed to initialize builtin plugins.2019-09-24T19:47:11.260963+08:00 0[ERROR] Aborting2019-09-24T19:47:11.260982+08:00 0[Note] Binlog end2019-09-24T19:47:11.262087+08:00 0 [Note] /qqweixin/mysql/bin/mysqld: Shutdown complete
解决方案
在数据目录下,删除ibdata1、ib_buffer_pool、ib_logfile0、ib_logfile1、ib_logfile2、ibtmp1文件。
执行的命令如下:
rm -rf ib*
此时,实例可以启动,但是搭建主从时,会报错:
2019-09-25T09:50:26.973154+08:00 0 [Warning] Info table is not ready to be used. Table 'mysql.slave_master_info'cannot be opened.2019-09-25T09:50:26.973165+08:00 0 [ERROR] Error in checking mysql.slave_master_info repository infotype of TABLE.2019-09-25T09:50:26.973171+08:00 0 [ERROR] Error creating master info: Error checking repositories.2019-09-25T09:50:26.973175+08:00 0 [ERROR] Slave: Failed to initialize the master info structure for channel ''; its record may still be present in 'mysql.slave_master_info'table, consider deleting it.2019-09-25T09:50:26.973182+08:00 0 [ERROR] Failed to create or recover replication inforepositories.2019-09-25T09:50:26.973186+08:00 0 [Note] Some of the channels are not created/initialized properly. Check for additional messages above. You will not be able to start replication on those channels until the issue is resolved and the server restarted.
如果数据库不是很大的话,建议考虑使用mysqldump 。
指定库备份
/weixinqq/mysql/bin/mysqldump --master-data=2 -uUID -pPWD --databases XXXdb --single-transaction -R --triggers > /backdbs/XXXdb.sql
导入还原
mysql -uUID -pPWD < /data/QQbackupDB/XXXdb.sql
-------------------------------------
上面这种情况的出现,常见错误原因是:主从mysql实例的配置文件(my.cnf)关于参数 innodb_data_file_path 的值设置的不相同,请留意。
8.通过XtraBackup 搭建主从
1.在GITD 模式下,通过XtraBackup恢复从实例,搭建主从,
change master to
master_host='XXX.XXX.XXX.XXX',
master_port=3306,
master_user='user_test',
master_password='pwd',
master_auto_position=1;
此时,主从搭建失败,一般会提示主键冲突,进一步分析原因是主从开始的【位点】 不对,例如,正确的同步关系应该是从330000位点开始同步,可以它科能从310000位点开始了同步。
解决方案:
如果备份是从主节点开始的,则
step 1:
reset master;
step 2:
set global gtid_purged=’开始真正需要同步的GTID′;
step 3
change master to
master_host='XXX.XXX.XXX.XXX',
master_port=3306,
master_user='user_test',
master_password='pwd',
master_auto_position=1;
step 4
start salve;
9.dump导入导出报错
再GTID模式下,导入时报错,信息如下
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR1840 (HY000) at line 24: @@GLOBAL.GTID_PURGED can only be set when @@GLOBAL.GTID_EXECUTED is empty.
在dump导出时,添加--set-gtid-purged=off参数,避免将gtid信息导出。
10.通过xtrabackup备份,找不到xtrabackup_binlog_info文件
通过xtrabackup备份,而在搭建主从时却找不到xtrabackup_binlog_info文件。异常原因是,没有开启 log-bin 参数
解决方案: 开启MySQL的binlog日志
在配置文件中my.cnf 添加以下参数
log-bin = mysql-bin
资料学习:
mysql数据库relay_MySQL 数据库主从复制小知识相关推荐
- MySQL完美卸载-奇怪的小知识
1.双击安装包,点击下一步,然后点击remove.卸载. 2.手动删除Program Files中的MySQL目录. 3.手动删除ProgramData目录(这个目录是隐藏的.)中的MySQL.
- 机构数据分析培训之关于Mysql定义及数据库相关小知识
机构数据分析培训之关于Mysql定义及数据库相关小知识 一.关于Mysql定义 MySQL是一种关联数据库管理系统,关联数据库将数据保存在不同的表中,而不是将所有数据放在一个大仓库内,这样就增加了速度 ...
- MySQL+Amoeba实现数据库主从复制和读写分离
MySQL读写分离是在主从复制的基础上进一步通过在master上执行写操作,在slave上执行读操作来实现的.通过主从复制,master上的数据改动能够同步到slave上,从而保持了数据的一致性.实现 ...
- Mysql(五)Mysql架构、数据库优化、主从复制
文章目录 一.Mysql架构 1.1 查询语句的执行过程 1.1.1 连接器 1.1.2 查询缓存 1.1.3 分析器 1.1.4 优化器 1.1.5 执行器 1.1.6 存储引擎 1.1.7 执行引 ...
- 小知识系列:数据库的主键和外键
文章目录 简介 创建主键 MySQL Oracle SQL Server 创建外键 mysql Oracle SQL Server 总结 简介 数据库是我们所有应用程序的基础,没有数据库的程序不是一个 ...
- 微信小程序订票选座PHP后台网站MySQL数据库和无前端小程序版本两个程序
微信小程序订票选座PHP后台网站MySQL数据库和无前端小程 下载点此去 两个相同功能 差别在一个有前端微信小程序 一个没有 (1)登录功能:注册普通账号登录:也可以直接使用微信登录:登录后可以修改用 ...
- MYSQL (关系型数据库管理系统)的基础知识详解
一.什么是数据库? 数据库,简而言之可视为电子化的文件柜--存储电子文件的处所,用户可以对文件中的数据进行新增.查询.更新.删除等操作 所谓"数据库"是以一定方式储存在一起.能与多 ...
- 聊聊图数据库和图数据库的小知识
图数据库 - 维基百科:在计算机科学中,图数据库(英语:graph database,GDB)是一个使用图结构进行语义查询的数据库,它使用节点.边和属性来表示和存储数据.该系统的关键概念是图,它直接将 ...
- mt5 mysql数据库_MySQL数据库配置主从复制
数据库主从复制 先本机模拟实现数据库主从复制.(本次使用MySql数据库,数据库版本5.7.25) 实现步骤: 1.安装第二个MySql数据库 2.配置数据库 3.在SpringBoot里实现数 ...
最新文章
- Unity IOC + MVC 实例需要手动释放吗?
- [云炬创业基础笔记]第七章创业资源测试3
- hive臨時udf與永久udf詳細操作流程
- linux如何查看git安装路径,Git - 如何验证Ubuntu上是否安装Git以及在何处安装Git - Ubuntu问答...
- 剑指Offer之数组中重复的数字
- Hadoop入门到精通(不断更新中。。。。)
- Linux 的文件与目录(软连接和硬连接)
- 贪心科技NLP训练营成果展示(附视频回放)
- 友盟新浪微博授权提示“找不到文件C8998”的解决办法
- gif怎么做成html,gif动态图片制作方法 附带电脑版和手机版制作gif动图教程
- Firewalld的区域(zone)
- 数据的存储,大端存储和小端小端存储
- DataTable列自定义渲染切页无效重绘
- 漫画丨CTO不得不面对的9大困境
- Failed to initialize NVML: Driver/library version mismatch
- Java实现windows,linux服务器word,excel转为PDF;aspose-words,Documents4j
- 爆款app们是如何做主动拉新和被动拉新的?
- 每一题-95(苹果和桔子)
- SQL Server数据库登陆失败,无法连接数据库
- php parse url 反向,php parse_url()函数
热门文章
- 计算机word缩小表格,word排版 文件等比例缩小 word表格等比例缩小
- 四级英语作文:拯救动物
- 留住前端面试官目光简历到底是怎样?4个方面把你简历问题一一突破
- pat basic 1100 校庆
- leetcode-种花问题
- java小游戏贪吃蛇
- 比较电路中的正相反馈电阻作用
- 元宇宙的运行之“DAO”:在元宇宙中创作、分享,付出的劳动如何获得回报?...
- c# 解决:Panel 添加自定义控件后滚动条跳动问题
- vue 上次登录时间_Vue设置长时间未操作登录自动到期返回登录页