MySQL示例DTID主从原理解析
1.gtid基本概念
mysql 5.6.5开始支持的,全局事务标识符(gtid(global transaction id))是创建的唯一标识符,并与在源(主)服务器上提交的每个事务相关联。
此标识符不但是唯一的,而且在给定复制设置中的所有服务器上都是唯一的。
所有交易和所有gtid之间都有一对一的映射关系 。
它由服务器id以及事务id组合而成。
这个全局事务id不仅仅在原始服务器上唯一,在所有存在主从关系 的mysql服务器上也是唯一的。
正是因为这样一个特性使得mysql的主从复制变得更加简单,以及数据库一致性更可靠。
一个gtid在一个服务器上只执行一次,避免重复执行导致数据混乱或者主从不一致。
2.gtid优点
保证同一个事务在某slave上绝对只执行一次,没有执行过的gtid事务总是会被执行。
不用像传统复制那样保证binlog的坐标准确,因为根本不需要binlog以及坐标。
故障转移到新的master的时候很方便,简化了很多任务。
很容易判断master和slave的数据是否一致。只要master上提交的事务在slave上也提交了,那么一定是一致的。
3.gtid的工作原理
1.当一个事务在主库端执行并提交时,产生gtid,一同记录到binlog日志中。
2.binlog传输到slave,并存储到slave的relaylog后,读取这个gtid的这个值设置gtid_next变量,即告诉slave,下一个要执行的gtid值。
3、sql线程从relay log中获取gtid,然后对比slave端的binlog是否有该gtid。
4、如果有记录,说明该gtid的事务已经执行,slave会忽略。
5、如果没有记录,slave就会执行该gtid事务,并记录该gtid到自身的binlog,在读取执行事务前会先检查其他session持有该gtid,确保不被重复执行。
6、在解析过程中会判断是否有主键,如果有就用二级索引,如果没有就用全部扫描。
4.gtid比传统复制的优势
1.更简单的实现故障转移(failover),不需要找log_file,log_pos
2.更简单的搭建主从复制
3.更加安全
4.gtid是连续没有空洞的,因此主数据库发生冲突时,可以添加空事件的方式进行跳过
5.启动的方法
- 方法一:如果是新搭建的服务器,直接启动即可
- 方法二:如果是以及跑的服务器,需要重启一下mysql server
启动前,先关闭master的写入,保证master端和slave端数据保持同步,所有slave需要加上skip_slave_start=1的配置参数,避免启动后还是使用之前的复制协议
6.gtid(一主一从)配置
6.1环境:
centos8.0 ip:192.168.136.239 有数据 hostname:mysql01
centos8.0 ip:192.168.136.219 无数据 hostname:mysql02
#二进制安装以及mysql自启动服务略
6.2在主库上给从库授权:
mysql> grant replication slave on *.* to 'slave'@'192.168.136.219' identified by 'slave'; query ok, 0 rows affected, 1 warning (0.00 sec) mysql> flush privileges; query ok, 0 rows affected (0.00 sec) #俩服务器均关闭防火墙 [root@mysql01 ~]# systemctl stop firewalld [root@mysql01 ~]# setenforce 0 [root@mysql02 ~]# systemctl stop firewalld [root@mysql02 ~]# setenforce 0 从库测试连接: [root@mysql02 ~]# mysql -u slave -p'slave' -h192.168.136.239 type 'help;' or '\h' for help. type '\c' to clear the current input statement. mysql>
6.3确保数据一致操作
1.对主库进行锁表 mysql> flush tables with read lock; 2.对主库进行全备 [root@mysql01 ~]# mysqldump -uroot -a > /clq/all-databases-20210519.sql 3.拷贝到从库主机上去 [root@mysql01 ~]# scp /clq/all-databases-20210519.sql root@192.168.136.219:/backup/ [root@mysql02 backup]# ll -rw-r--r--. 1 root root 873527 5月 19 16:40 all-databases-20210519.sql 4.从库上进行主库的恢复 [root@mysql02 backup]# mysql -uroot -phuawei0917@ < all-databases-20210519.sql
6.4配置主库
[mysqld] basedir = /usr/local/mysql datadir = /opt/data socket = /tmp/mysql.sock port = 3306 user = mysql pid-file = /opt/data/mysql.pid skip-name-resolve #skip-grant-tables log-bin = master_bin #开启主库日志 server-id = 10 #服务唯一标识id gtid-mode = on #gtid模式开启 enforce_gtid_consistency = on #强制gtid模式一致性 log-slave-updates = 1 #从库允许更新日志,同步操作日志 binlog_format = row #binlog日志格式为行格式, 默认是mixed混合模式 skip_slave_start = 1 #跳过从库开启,以主库开始开启 #重启 systemctl restart mysqld
6.5配置从库
[root@mysql02 data]# cat /etc/my.cnf [mysqld] basedir = /usr/local/mysql datadir = /opt/data socket = /tmp/mysql.sock port = 3306 user = mysql pid-file = /opt/data/mysql.pid skip-name-resolve #skip-grant-tables gtid_mode=on enforce_gtid_consistency=on server-id=20 log-bin=slave_binlog #开启从库日志 log_slave-updates=1 #从库允许更新 binlog_format=row #格式为行 skip-slave_start=1 #重启 systemctl restart mysqld
查看gtid状态情况
mysql> show variables like '%gtid%'; +----------------------------------+-----------+ | variable_name | value | +----------------------------------+-----------+ | binlog_gtid_simple_recovery | on | | enforce_gtid_consistency | on | | gtid_executed_compression_period | 1000 | | gtid_mode | on | | gtid_next | automatic | | gtid_owned | | | gtid_purged | | | session_track_gtids | off | +----------------------------------+-----------+ 8 rows in set (0.00 sec)
6.6配置主从复制
#从库上root登录配置 #help change master to 可以查看帮助文档实例 mysql> change master to -> master_host='192.168.136.239', -> master_user='slave', -> master_password='slave', -> master_port=3306, #主库端口 -> master_auto_position=1; #位置 #master_use_gtid = current_pos query ok, 0 rows affected, 2 warnings (0.01 sec) mysql> start slave; query ok, 0 rows affected (0.00 sec) mysql> show slave status\g; slave_io_running: connecting slave_io_running: yes slave_sql_running: yes 保证系统一致性 授权一致性
(一主一从gtid)测试:
主库创建一个数据库test,进行测试查看
从库创建一个数据库test02,进行测试查看
#主库创建一个test数据库 mysql> create database test; mysql> show databases; +--------------------+ | database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | | test | +--------------------+ #从库上查看同步情况 mysql> show databases; +--------------------+ | database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | | test | +--------------------+ 6 rows in set (0.00 sec) #从库创建test02库 mysql> create database test02; query ok, 1 row affected (0.00 sec) #主库上查看 mysql> show databases; +--------------------+ | database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | #是没有test02库的 | test | +--------------------+ 5 rows in set (0.00 sec)
小结:主库上的数据操作会同步到从库上面去,而从库上的数据操作与主库没联系
7.gtid(一主俩从)
第三台mysql连接的话,相应配置
第3台mysql ,版本:centos8 ip:192.168.136.230 主机名:mysql03
[root@mysql03 ~]# cat /etc/my.cnf [mysqld] basedir = /usr/local/mysql datadir = /opt/data socket = /tmp/mysql.sock port = 3306 user = mysql pid-file = /opt/data/mysql.pid skip-name-resolve #skip-grant-tables # replication config log-bin = master_bin server-id = 21 #id必须与之前不同 gtid-mode = on enforce-gtid-consistency = on log-slave-updates = 1 binlog-format = row skip-slave-start = 1 #查看gtid情况 mysql> show variables like '%gtid%'; +----------------------------------+-----------+ | variable_name | value | +----------------------------------+-----------+ | binlog_gtid_simple_recovery | on | | enforce_gtid_consistency | on | | gtid_executed_compression_period | 1000 | | gtid_mode | on | | gtid_next | automatic | | gtid_owned | | | gtid_purged | | | session_track_gtids | off | +----------------------------------+-----------+ #由于之前只权限了一个ip,此刻在mysql01主数据库上再授权一个ip mysql> grant replication slave on *.* to 'slave'@'192.168.136.230' identified by 'slave'; query ok, 0 rows affected, 1 warning (0.00 sec) mysql> flush privileges; query ok, 0 rows affected (0.00 sec) #测试连接 [root@mysql ~]# mysql -uslave -pslave -h192.168.136.239 mysql: [warning] using a password on the command line interface can be insecure. welcome to the mysql monitor. commands end with ; or \g. your mysql connection id is 17 server version: 5.7.33-log mysql community server (gpl) copyright (c) 2000, 2021, oracle and/or its affiliates. oracle is a registered trademark of oracle corporation and/or its affiliates. other names may be trademarks of their respective owners. type 'help;' or '\h' for help. type '\c' to clear the current input statement. mysql> #mysql03从库上root用户连接进行相应配置 [root@mysql03 ~]# mysql -uroot -p1 mysql> change master to -> master_host='192.168.136.239', #主库ip -> master_user='slave', #主库授权的普通用户 -> master_password='slave', -> master_port=3306, #主库端口 -> master_auto_position=1; #位置从1开始同步 #也可以查看帮助进行配置 mysql> help change master to; change master to master_host='source2.example.com', master_user='replication', master_password='password', master_port=3306, master_log_file='source2-bin.001', master_log_pos=4, master_connect_retry=10; url: https://dev.mysql.com/doc/refman/5.7/en/change-master-to.html #开启 mysql> start slave; mysql> show slave status\g; *************************** 1. row *************************** slave_io_state: waiting for master to send event master_host: 192.168.136.239 master_user: slave master_port: 3306 connect_retry: 60 master_log_file: master_bin.000002 read_master_log_pos: 2172 relay_log_file: mysql-relay-bin.000002 relay_log_pos: 2387 relay_master_log_file: master_bin.000002 slave_io_running: yes slave_sql_running: yes #显示俩个yes则运行成功! #mysql03查看数据库,数据库内容也同步成功 mysql> show databases; +--------------------+ | database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | | test | +--------------------+ 5 rows in set (0.00 sec)
8.gtid(俩主一从)
1.最新环境
版本 | ip | 主机名 | 身份 |
---|---|---|---|
centos8 | 192.168.136.239 | master01 | 主库 |
centos8 | 192.168.136.219 | master02 | 主库 |
centos8 | 192.168.136.230 | slave | 从库 |
2.所有服务器均关闭防火墙或者放行防火墙
[root@master01 ~]# systemctl stop firewalld [root@master01 ~]# systemctl disable firewalld [root@master02 ~]# systemctl stop firewalld [root@master02 ~]# systemctl disable firewalld [root@slave ~]# systemctl stop firewalld [root@slave ~]# systemctl disable firewalld
3.授权连接
master01库授权普通用户
mysql> grant replication slave on *.* to 'user'@'192.168.136.%' identified by 'user';
slave进行连接
[root@slave ~]# mysql -uuser -p'user' -h192.168.136.239 mysql: [warning] using a password on the command line interface can be insecure. welcome to the mysql monitor. commands end with ; or \g. your mysql connection id is 5 server version: 5.7.33 mysql community server (gpl) copyright (c) 2000, 2021, oracle and/or its affiliates. oracle is a registered trademark of oracle corporation and/or its affiliates. other names may be trademarks of their respective owners. type 'help;' or '\h' for help. type '\c' to clear the current input statement.
master02授权普通用户
mysql> grant replication slave on *.* to 'app'@'192.168.136.%' identified by 'app'; query ok, 0 rows affected, 1 warning (0.01 sec) mysql> flush privileges; query ok, 0 rows affected (0.00 sec)
slave进行连接
[root@slave ~]# mysql -uapp -papp -h192.168.136.219 mysql: [warning] using a password on the command line interface can be insecure. welcome to the mysql monitor. commands end with ; or \g. your mysql connection id is 3 server version: 5.7.33 mysql community server (gpl) copyright (c) 2000, 2021, oracle and/or its affiliates. oracle is a registered trademark of oracle corporation and/or its affiliates. other names may be trademarks of their respective owners. type 'help;' or '\h' for help. type '\c' to clear the current input statement. mysql>
4.分别进行配置文件修改
#master01主机: [root@master01 ~]# cat /etc/my.cnf [mysqld] basedir = /usr/local/mysql datadir = /opt/data socket = /tmp/mysql.sock port = 3306 user = mysql pid-file = /opt/data/mysql.pid skip-name-resolve skip-grant-tables log-bin = master_bin server-id = 10 gtid-mode = on enforce-gtid-consistency = on log-slave-updates = 1 binlog-format = row skip-slave-start = 1 #master02主机 [mysqld] basedir = /usr/local/mysql datadir = /opt/data socket = /tmp/mysql.sock port = 3306 user = mysql pid-file = /opt/data/mysql.pid skip-name-resolve #replication config log-bin = master_bin server-id = 11 gtid-mode = on enforce-gtid-consistency = on log-slave-updates = 1 binlog-format = row skip-slave-start = 1 #slave主机 [mysqld] basedir = /usr/local/mysql datadir = /opt/data socket = /tmp/mysql.sock port = 3306 user = mysql pid-file = /opt/data/mysql.pid skip-name-resolve log-bin = slave_bin server-id = 13 gtid-mode = on enforce-gtid-consistency = on log-slave-updates = 1 binlog-format = row skip-slave-start = 1
5.分别重启
[root@master01 ~]# systemctl restart mysqld [root@master02 ~]# systemctl restart mysqld [root@slave ~]# systemctl restart mysqld
6.在进行gtid多主一从配置前,先引入一个概念
channel
(频道):每一个channel都是一个独立的slave服务,都有一个io_thread和sql_thread,原理和普通复制一样,只是需要在change master to语句后面使用for channel来进行区分slave
在使用channel时需要将从库的master-info-repository、relay-log-info-repository设置为table,否则会报错。
将信息存储库设置为table格式
方式一(mysql内设置): set global master_info_repository='table'; set global relay_log_info_repository='table'; 方式二(/etc/my.cnf内设置): 3.在my.cnf中设置 master_info_repository = table relay_log_info_repository = table #检查是否更改成功 mysql> show variables where variable_name in ('relay_log_info_repository','master_info_repository'); +---------------------------+-------+ | variable_name | value | +---------------------------+-------+ | master_info_repository | table | | relay_log_info_repository | table | +---------------------------+-------+
7.slave从库以root用户登录进行gtid配置
#slave从库上配置俩个主库gtid复制 mysql> change master to -> master_host='192.168.136.219', #mysql02主库ip -> master_user='app', #mysql02主库授权的普通用户 -> master_password='app', #mysql02主库授权的普通用户密码 -> master_port=3306, #主库端口 -> master_auto_position=1 for channel 'master01'; #位置从1开始同步,并且第一个slave取名master01 mysql> change master to -> master_host='192.168.136.239', #mysql01主库ip -> master_user='user', -> master_password='user', -> master_port=3306, #主库端口 -> master_auto_position=1 for channel 'master02'; #位置从1开始同步,并且第一个slave取名master01 #查看俩个slave状态 mysql> show slave status\g; *************************** 1. row *************************** slave_io_state: master_host: 192.168.136.219 master_user: app master_port: 3306 connect_retry: 60 master_log_file: read_master_log_pos: 4 relay_log_file: slave02-relay-bin-master1.000001 relay_log_pos: 4 relay_master_log_file: slave_io_running: no slave_sql_running: no #都是关闭的 replicate_do_db: replicate_ignore_db: replicate_do_table: replicate_ignore_table: replicate_wild_do_table: replicate_wild_ignore_table: last_errno: 0 last_error: skip_counter: 0 exec_master_log_pos: 0 relay_log_space: 154 until_condition: none until_log_file: until_log_pos: 0 master_ssl_allowed: no master_ssl_ca_file: master_ssl_ca_path: master_ssl_cert: master_ssl_cipher: master_ssl_key: seconds_behind_master: null master_ssl_verify_server_cert: no last_io_errno: 0 last_io_error: last_sql_errno: 0 last_sql_error: replicate_ignore_server_ids: master_server_id: 0 master_uuid: master_info_file: mysql.slave_master_info sql_delay: 0 sql_remaining_delay: null slave_sql_running_state: master_retry_count: 86400 master_bind: last_io_error_timestamp: last_sql_error_timestamp: master_ssl_crl: master_ssl_crlpath: retrieved_gtid_set: executed_gtid_set: b4326a77-0a31-11ec-a991-000c298d3571:1-2, d68b404d-0a35-11ec-9df1-000c29581959:1 auto_position: 1 replicate_rewrite_db: channel_name: master1 master_tls_version: *************************** 2. row *************************** slave_io_state: master_host: 192.168.136.239 master_user: user master_port: 3306 connect_retry: 60 master_log_file: read_master_log_pos: 4 relay_log_file: slave02-relay-bin-master2.000001 relay_log_pos: 4 relay_master_log_file: slave_io_running: no slave_sql_running: no replicate_do_db: replicate_ignore_db: replicate_do_table: replicate_ignore_table: replicate_wild_do_table: replicate_wild_ignore_table: last_errno: 0 last_error: skip_counter: 0 exec_master_log_pos: 0 relay_log_space: 154 until_condition: none until_log_file: until_log_pos: 0 master_ssl_allowed: no master_ssl_ca_file: master_ssl_ca_path: master_ssl_cert: master_ssl_cipher: master_ssl_key: seconds_behind_master: null master_ssl_verify_server_cert: no last_io_errno: 0 last_io_error: last_sql_errno: 0 last_sql_error: replicate_ignore_server_ids: master_server_id: 0 master_uuid: master_info_file: mysql.slave_master_info sql_delay: 0 sql_remaining_delay: null slave_sql_running_state: master_retry_count: 86400 master_bind: last_io_error_timestamp: last_sql_error_timestamp: master_ssl_crl: master_ssl_crlpath: retrieved_gtid_set: executed_gtid_set: b4326a77-0a31-11ec-a991-000c298d3571:1-2, d68b404d-0a35-11ec-9df1-000c29581959:1 auto_position: 1 replicate_rewrite_db: channel_name: master2 master_tls_version: 2 rows in set (0.00 sec) #开启俩个slave mysql> start slave; #再次查看状态
gtid(俩主一从)测试:
#master01主库创建一个test数据库 mysql> create database test; query ok, 1 row affected (0.00 sec) mysql> show databases; +--------------------+ | database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | | test | +--------------------+ 5 rows in set (0.00 sec) #master02主库上查看 mysql> show databases; +--------------------+ | database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | #没有内容 +--------------------+ 4 rows in set (0.00 sec) #slave从库查看 mysql> show databases; +--------------------+ | database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | | test | #已经同步了test库 +--------------------+ 5 rows in set (0.00 sec) #mysql02主库创建一个rhca数据库 mysql> create database rhca; query ok, 1 row affected (0.01 sec) mysql> show databases; +--------------------+ | database | +--------------------+ | information_schema | | rhca | | mysql | | performance_schema | | sys | +--------------------+ 5 rows in set (0.00 sec) #slave从库 mysql> show databases; +--------------------+ | database | +--------------------+ | information_schema | | rhca | | mysql | | performance_schema | | sys | #有了mysql01主库的test库和mysql02的rhca的库 | test | +--------------------+ 6 rows in set (0.00 sec)
slave相关命令:
show slave status; //查看全部slave状态
show slave status for channel ‘naem'; //查看单个slave状态
reset slave; #重置全部slave
reset slave for channel ‘master1'; #重置单个slave
stop slave for channel ‘master1'; #暂停单个slave
start slave for channel ‘master1'; #开启单个slave
虽然我在做的过程没有遇到错误,但是下面这个是最最容易出现的错误
配置完开启slave出现报错
mysql> start slave; error 1872 (hy000): slave failed to initialize relay log info structure from the repository
解决问题
由于mysql.slave_relay_log_info表中保留了以前的复制信息,导致新从库启动时无法找到对应文件,那么我们清理掉该表中的记录即可
mysql> reset slave; query ok, 0 rows affected (0.00 sec)
以上就是mysql示例dtid主从原理解析的详细内容,更多关于mysql示例dtid主从原理的资料请关注其它相关文章!
上一篇: 原生JS实现旋转轮播图+文字内容切换效果【附源码】
下一篇: 浅谈开发eslint规则