MySQL示例DTID主从原理解析

1.gtid基本概念

mysql 5.6.5开始支持的，全局事务标识符（gtid（global transaction id））是创建的唯一标识符，并与在源（主）服务器上提交的每个事务相关联。
此标识符不但是唯一的，而且在给定复制设置中的所有服务器上都是唯一的。
所有交易和所有gtid之间都有一对一的映射关系 。
它由服务器id以及事务id组合而成。
这个全局事务id不仅仅在原始服务器上唯一，在所有存在主从关系 的mysql服务器上也是唯一的。
正是因为这样一个特性使得mysql的主从复制变得更加简单，以及数据库一致性更可靠。
一个gtid在一个服务器上只执行一次，避免重复执行导致数据混乱或者主从不一致。

2.gtid优点

保证同一个事务在某slave上绝对只执行一次，没有执行过的gtid事务总是会被执行。
不用像传统复制那样保证binlog的坐标准确，因为根本不需要binlog以及坐标。
故障转移到新的master的时候很方便，简化了很多任务。
很容易判断master和slave的数据是否一致。只要master上提交的事务在slave上也提交了，那么一定是一致的。

3.gtid的工作原理

1.当一个事务在主库端执行并提交时，产生gtid，一同记录到binlog日志中。
2.binlog传输到slave,并存储到slave的relaylog后，读取这个gtid的这个值设置gtid_next变量，即告诉slave，下一个要执行的gtid值。
3、sql线程从relay log中获取gtid，然后对比slave端的binlog是否有该gtid。
4、如果有记录，说明该gtid的事务已经执行，slave会忽略。
5、如果没有记录，slave就会执行该gtid事务，并记录该gtid到自身的binlog，在读取执行事务前会先检查其他session持有该gtid，确保不被重复执行。
6、在解析过程中会判断是否有主键，如果有就用二级索引，如果没有就用全部扫描。

4.gtid比传统复制的优势

1.更简单的实现故障转移（failover），不需要找log_file,log_pos

2.更简单的搭建主从复制

3.更加安全

4.gtid是连续没有空洞的，因此主数据库发生冲突时，可以添加空事件的方式进行跳过

5.启动的方法

• 方法一：如果是新搭建的服务器，直接启动即可
• 方法二：如果是以及跑的服务器，需要重启一下mysql server

启动前，先关闭master的写入，保证master端和slave端数据保持同步，所有slave需要加上skip_slave_start=1的配置参数，避免启动后还是使用之前的复制协议

6.gtid(一主一从)配置

6.1环境：

centos8.0 ip：192.168.136.239 有数据 hostname：mysql01

centos8.0 ip：192.168.136.219 无数据 hostname：mysql02

#二进制安装以及mysql自启动服务略

6.2在主库上给从库授权：

mysql> grant replication slave on *.* to 'slave'@'192.168.136.219' identified by 'slave';
query ok, 0 rows affected, 1 warning (0.00 sec)
mysql> flush privileges;
query ok, 0 rows affected (0.00 sec)
#俩服务器均关闭防火墙
[root@mysql01 ~]# systemctl stop firewalld
[root@mysql01 ~]# setenforce 0
[root@mysql02 ~]# systemctl stop firewalld
[root@mysql02 ~]# setenforce 0
从库测试连接：
[root@mysql02 ~]# mysql -u slave -p'slave' -h192.168.136.239
type 'help;' or '\h' for help. type '\c' to clear the current input statement.
mysql> 

6.3确保数据一致操作

1.对主库进行锁表
mysql> flush tables with read lock;
2.对主库进行全备
[root@mysql01 ~]# mysqldump -uroot -a > /clq/all-databases-20210519.sql
3.拷贝到从库主机上去
[root@mysql01 ~]# scp /clq/all-databases-20210519.sql root@192.168.136.219:/backup/
[root@mysql02 backup]# ll
-rw-r--r--. 1 root root 873527 5月  19 16:40 all-databases-20210519.sql
4.从库上进行主库的恢复
[root@mysql02 backup]# mysql -uroot -phuawei0917@ < all-databases-20210519.sql 

6.4配置主库

[mysqld]
basedir = /usr/local/mysql
datadir = /opt/data
socket = /tmp/mysql.sock
port = 3306
user = mysql
pid-file = /opt/data/mysql.pid
skip-name-resolve
#skip-grant-tables
log-bin = master_bin   #开启主库日志
server-id = 10        #服务唯一标识id
gtid-mode = on        #gtid模式开启
enforce_gtid_consistency = on #强制gtid模式一致性
log-slave-updates = 1    #从库允许更新日志，同步操作日志
binlog_format = row    #binlog日志格式为行格式, 默认是mixed混合模式
skip_slave_start = 1   #跳过从库开启，以主库开始开启
#重启
systemctl restart mysqld 

6.5配置从库

[root@mysql02 data]# cat /etc/my.cnf
[mysqld]
basedir = /usr/local/mysql 
datadir = /opt/data 
socket = /tmp/mysql.sock 
port = 3306
user = mysql
pid-file = /opt/data/mysql.pid
skip-name-resolve
#skip-grant-tables 
gtid_mode=on
enforce_gtid_consistency=on
server-id=20
log-bin=slave_binlog       #开启从库日志
log_slave-updates=1        #从库允许更新
binlog_format=row          #格式为行
skip-slave_start=1   
#重启
systemctl restart mysqld 

查看gtid状态情况

mysql> show variables like '%gtid%';
+----------------------------------+-----------+
| variable_name                    | value     |
+----------------------------------+-----------+
| binlog_gtid_simple_recovery      | on        |
| enforce_gtid_consistency         | on        |
| gtid_executed_compression_period | 1000      |
| gtid_mode                        | on        |
| gtid_next                        | automatic |
| gtid_owned                       |           |
| gtid_purged                      |           |
| session_track_gtids              | off       |
+----------------------------------+-----------+
8 rows in set (0.00 sec)

6.6配置主从复制

#从库上root登录配置      #help change master to 可以查看帮助文档实例
mysql> change master to
    -> master_host='192.168.136.239',
    -> master_user='slave',
    -> master_password='slave',
    -> master_port=3306,        #主库端口
    -> master_auto_position=1;  #位置
                                       #master_use_gtid = current_pos
query ok, 0 rows affected, 2 warnings (0.01 sec)
mysql> start slave;  
query ok, 0 rows affected (0.00 sec)
mysql> show slave status\g; 
slave_io_running: connecting
            slave_io_running: yes
            slave_sql_running: yes
保证系统一致性 
授权一致性

（一主一从gtid）测试：

主库创建一个数据库test，进行测试查看

从库创建一个数据库test02，进行测试查看

#主库创建一个test数据库
mysql> create database test;
mysql> show databases;
+--------------------+
| database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
| test               |
+--------------------+
#从库上查看同步情况
mysql> show databases;
+--------------------+
| database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
| test               |
+--------------------+
6 rows in set (0.00 sec)

#从库创建test02库
mysql> create database test02;
query ok, 1 row affected (0.00 sec)
#主库上查看
mysql> show databases;
+--------------------+
| database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |         #是没有test02库的
| test               |
+--------------------+
5 rows in set (0.00 sec)

小结：主库上的数据操作会同步到从库上面去，而从库上的数据操作与主库没联系

7.gtid（一主俩从）

第三台mysql连接的话，相应配置

第3台mysql ，版本:centos8 ip:192.168.136.230 主机名:mysql03

[root@mysql03 ~]# cat /etc/my.cnf 
[mysqld]
basedir = /usr/local/mysql
datadir = /opt/data
socket = /tmp/mysql.sock
port = 3306
user = mysql
pid-file = /opt/data/mysql.pid
skip-name-resolve
#skip-grant-tables
# replication config
log-bin = master_bin
server-id = 21               #id必须与之前不同
gtid-mode = on
enforce-gtid-consistency = on
log-slave-updates = 1
binlog-format = row
skip-slave-start = 1
#查看gtid情况
mysql> show variables like '%gtid%';
+----------------------------------+-----------+
| variable_name                    | value     |
+----------------------------------+-----------+
| binlog_gtid_simple_recovery      | on        |
| enforce_gtid_consistency         | on        |
| gtid_executed_compression_period | 1000      |
| gtid_mode                        | on        |
| gtid_next                        | automatic |
| gtid_owned                       |           |
| gtid_purged                      |           |
| session_track_gtids              | off       |
+----------------------------------+-----------+
#由于之前只权限了一个ip，此刻在mysql01主数据库上再授权一个ip
mysql> grant replication slave on *.* to 'slave'@'192.168.136.230' identified by 'slave';
query ok, 0 rows affected, 1 warning (0.00 sec)
mysql> flush privileges;
query ok, 0 rows affected (0.00 sec)
#测试连接
[root@mysql ~]#  mysql -uslave -pslave -h192.168.136.239
mysql: [warning] using a password on the command line interface can be insecure.
welcome to the mysql monitor.  commands end with ; or \g.
your mysql connection id is 17
server version: 5.7.33-log mysql community server (gpl)
copyright (c) 2000, 2021, oracle and/or its affiliates.
oracle is a registered trademark of oracle corporation and/or its
affiliates. other names may be trademarks of their respective
owners.
type 'help;' or '\h' for help. type '\c' to clear the current input statement.
mysql> 
#mysql03从库上root用户连接进行相应配置
[root@mysql03 ~]#  mysql -uroot -p1
mysql> change master to
    -> master_host='192.168.136.239',  #主库ip
    -> master_user='slave',              #主库授权的普通用户
    -> master_password='slave',
    -> master_port=3306,              #主库端口
    -> master_auto_position=1;   #位置从1开始同步
#也可以查看帮助进行配置
mysql> help change master to;
change master to
  master_host='source2.example.com',
  master_user='replication',
  master_password='password',
  master_port=3306,
  master_log_file='source2-bin.001',
  master_log_pos=4,
  master_connect_retry=10;
url: https://dev.mysql.com/doc/refman/5.7/en/change-master-to.html

#开启
mysql> start slave;
mysql> show slave status\g;
*************************** 1. row ***************************
               slave_io_state: waiting for master to send event
                  master_host: 192.168.136.239
                  master_user: slave
                  master_port: 3306
                connect_retry: 60
              master_log_file: master_bin.000002
          read_master_log_pos: 2172
               relay_log_file: mysql-relay-bin.000002
                relay_log_pos: 2387
        relay_master_log_file: master_bin.000002
             slave_io_running: yes
            slave_sql_running: yes    #显示俩个yes则运行成功！
#mysql03查看数据库，数据库内容也同步成功
mysql> show databases;
+--------------------+
| database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
| test               |
+--------------------+
5 rows in set (0.00 sec)

8.gtid（俩主一从）

1.最新环境

版本	ip	主机名	身份
centos8	192.168.136.239	master01	主库
centos8	192.168.136.219	master02	主库
centos8	192.168.136.230	slave	从库

2.所有服务器均关闭防火墙或者放行防火墙

[root@master01 ~]# systemctl stop firewalld
[root@master01 ~]# systemctl disable firewalld
[root@master02 ~]# systemctl stop firewalld
[root@master02 ~]# systemctl disable firewalld
[root@slave ~]# systemctl stop firewalld
[root@slave ~]# systemctl disable firewalld

3.授权连接

master01库授权普通用户

mysql> grant replication slave on *.* to  'user'@'192.168.136.%' identified by 'user';

slave进行连接

[root@slave ~]# mysql -uuser -p'user' -h192.168.136.239
mysql: [warning] using a password on the command line interface can be insecure.
welcome to the mysql monitor.  commands end with ; or \g.
your mysql connection id is 5
server version: 5.7.33 mysql community server (gpl)
copyright (c) 2000, 2021, oracle and/or its affiliates.
oracle is a registered trademark of oracle corporation and/or its
affiliates. other names may be trademarks of their respective
owners.
type 'help;' or '\h' for help. type '\c' to clear the current input statement.

master02授权普通用户

mysql> grant replication slave on *.* to  'app'@'192.168.136.%' identified by 'app';
query ok, 0 rows affected, 1 warning (0.01 sec)
mysql> flush privileges;
query ok, 0 rows affected (0.00 sec)

slave进行连接

[root@slave ~]# mysql -uapp -papp -h192.168.136.219
mysql: [warning] using a password on the command line interface can be insecure.
welcome to the mysql monitor.  commands end with ; or \g.
your mysql connection id is 3
server version: 5.7.33 mysql community server (gpl)
copyright (c) 2000, 2021, oracle and/or its affiliates.
oracle is a registered trademark of oracle corporation and/or its
affiliates. other names may be trademarks of their respective
owners.
type 'help;' or '\h' for help. type '\c' to clear the current input statement.
mysql> 

4.分别进行配置文件修改

#master01主机：
[root@master01 ~]# cat /etc/my.cnf 
[mysqld]
basedir = /usr/local/mysql
datadir = /opt/data
socket = /tmp/mysql.sock
port = 3306
user = mysql
pid-file = /opt/data/mysql.pid
skip-name-resolve
skip-grant-tables
log-bin = master_bin
server-id = 10
gtid-mode = on
enforce-gtid-consistency = on
log-slave-updates = 1
binlog-format = row
skip-slave-start = 1
#master02主机
[mysqld]
basedir = /usr/local/mysql
datadir = /opt/data
socket = /tmp/mysql.sock
port = 3306
user = mysql
pid-file = /opt/data/mysql.pid
skip-name-resolve
                      #replication config
log-bin = master_bin
server-id = 11
gtid-mode = on
enforce-gtid-consistency = on
log-slave-updates = 1
binlog-format = row
skip-slave-start = 1
#slave主机
[mysqld]
basedir = /usr/local/mysql
datadir = /opt/data
socket = /tmp/mysql.sock
port = 3306
user = mysql
pid-file = /opt/data/mysql.pid
skip-name-resolve
log-bin = slave_bin
server-id = 13
gtid-mode = on
enforce-gtid-consistency = on
log-slave-updates = 1
binlog-format = row
skip-slave-start = 1

5.分别重启

[root@master01 ~]# systemctl restart mysqld
[root@master02 ~]# systemctl restart mysqld
[root@slave ~]# systemctl restart mysqld

6.在进行gtid多主一从配置前，先引入一个概念

channel(频道)：每一个channel都是一个独立的slave服务，都有一个io_thread和sql_thread，原理和普通复制一样，只是需要在change master to语句后面使用for channel来进行区分slave

在使用channel时需要将从库的master-info-repository、relay-log-info-repository设置为table，否则会报错。

将信息存储库设置为table格式

方式一（mysql内设置）:
set global master_info_repository='table';
set global relay_log_info_repository='table';
方式二（/etc/my.cnf内设置）:
3.在my.cnf中设置
master_info_repository    = table 
relay_log_info_repository = table   
#检查是否更改成功
mysql> show variables where variable_name in  ('relay_log_info_repository','master_info_repository');
+---------------------------+-------+
| variable_name             | value |
+---------------------------+-------+
| master_info_repository    | table |
| relay_log_info_repository | table |
+---------------------------+-------+

7.slave从库以root用户登录进行gtid配置

#slave从库上配置俩个主库gtid复制
mysql> change master to
    -> master_host='192.168.136.219',  #mysql02主库ip
    -> master_user='app',              #mysql02主库授权的普通用户
    -> master_password='app',           #mysql02主库授权的普通用户密码
    -> master_port=3306,              #主库端口
    -> master_auto_position=1 for channel 'master01';   #位置从1开始同步,并且第一个slave取名master01
mysql> change master to
    -> master_host='192.168.136.239',  #mysql01主库ip
    -> master_user='user',              
    -> master_password='user',          
    -> master_port=3306,              #主库端口
    -> master_auto_position=1 for channel 'master02';   #位置从1开始同步,并且第一个slave取名master01
#查看俩个slave状态                   
mysql> show slave status\g;
*************************** 1. row ***************************
               slave_io_state: 
                  master_host: 192.168.136.219
                  master_user: app
                  master_port: 3306
                connect_retry: 60
              master_log_file: 
          read_master_log_pos: 4
               relay_log_file: slave02-relay-bin-master1.000001
                relay_log_pos: 4
        relay_master_log_file: 
             slave_io_running: no
            slave_sql_running: no         #都是关闭的
              replicate_do_db: 
          replicate_ignore_db: 
           replicate_do_table: 
       replicate_ignore_table: 
      replicate_wild_do_table: 
  replicate_wild_ignore_table: 
                   last_errno: 0
                   last_error: 
                 skip_counter: 0
          exec_master_log_pos: 0
              relay_log_space: 154
              until_condition: none
               until_log_file: 
                until_log_pos: 0
           master_ssl_allowed: no
           master_ssl_ca_file: 
           master_ssl_ca_path: 
              master_ssl_cert: 
            master_ssl_cipher: 
               master_ssl_key: 
        seconds_behind_master: null
master_ssl_verify_server_cert: no
                last_io_errno: 0
                last_io_error: 
               last_sql_errno: 0
               last_sql_error: 
  replicate_ignore_server_ids: 
             master_server_id: 0
                  master_uuid: 
             master_info_file: mysql.slave_master_info
                    sql_delay: 0
          sql_remaining_delay: null
      slave_sql_running_state: 
           master_retry_count: 86400
                  master_bind: 
      last_io_error_timestamp: 
     last_sql_error_timestamp: 
               master_ssl_crl: 
           master_ssl_crlpath: 
           retrieved_gtid_set: 
            executed_gtid_set: b4326a77-0a31-11ec-a991-000c298d3571:1-2,
d68b404d-0a35-11ec-9df1-000c29581959:1
                auto_position: 1
         replicate_rewrite_db: 
                 channel_name: master1
           master_tls_version: 
*************************** 2. row ***************************
               slave_io_state: 
                  master_host: 192.168.136.239
                  master_user: user
                  master_port: 3306
                connect_retry: 60
              master_log_file: 
          read_master_log_pos: 4
               relay_log_file: slave02-relay-bin-master2.000001
                relay_log_pos: 4
        relay_master_log_file: 
             slave_io_running: no
            slave_sql_running: no
              replicate_do_db: 
          replicate_ignore_db: 
           replicate_do_table: 
       replicate_ignore_table: 
      replicate_wild_do_table: 
  replicate_wild_ignore_table: 
                   last_errno: 0
                   last_error: 
                 skip_counter: 0
          exec_master_log_pos: 0
              relay_log_space: 154
              until_condition: none
               until_log_file: 
                until_log_pos: 0
           master_ssl_allowed: no
           master_ssl_ca_file: 
           master_ssl_ca_path: 
              master_ssl_cert: 
            master_ssl_cipher: 
               master_ssl_key: 
        seconds_behind_master: null
master_ssl_verify_server_cert: no
                last_io_errno: 0
                last_io_error: 
               last_sql_errno: 0
               last_sql_error: 
  replicate_ignore_server_ids: 
             master_server_id: 0
                  master_uuid: 
             master_info_file: mysql.slave_master_info
                    sql_delay: 0
          sql_remaining_delay: null
      slave_sql_running_state: 
           master_retry_count: 86400
                  master_bind: 
      last_io_error_timestamp: 
     last_sql_error_timestamp: 
               master_ssl_crl: 
           master_ssl_crlpath: 
           retrieved_gtid_set: 
            executed_gtid_set: b4326a77-0a31-11ec-a991-000c298d3571:1-2,
d68b404d-0a35-11ec-9df1-000c29581959:1
                auto_position: 1
         replicate_rewrite_db: 
                 channel_name: master2
           master_tls_version: 
2 rows in set (0.00 sec)
#开启俩个slave
mysql> start slave;
#再次查看状态

gtid(俩主一从)测试：

#master01主库创建一个test数据库
mysql> create database test;
query ok, 1 row affected (0.00 sec)
mysql>  show databases;
+--------------------+
| database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
| test               |
+--------------------+
5 rows in set (0.00 sec)
#master02主库上查看
mysql>  show databases;
+--------------------+
| database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |                  #没有内容
+--------------------+
4 rows in set (0.00 sec)
#slave从库查看
mysql> show databases;
+--------------------+
| database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
| test               |          #已经同步了test库
+--------------------+
5 rows in set (0.00 sec)
#mysql02主库创建一个rhca数据库
mysql> create database rhca;
query ok, 1 row affected (0.01 sec)
mysql>  show databases;
+--------------------+
| database           |
+--------------------+
| information_schema |
| rhca               |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
5 rows in set (0.00 sec)
#slave从库
mysql> show databases;
+--------------------+
| database           |
+--------------------+
| information_schema |
| rhca               |
| mysql              |
| performance_schema |
| sys                |             #有了mysql01主库的test库和mysql02的rhca的库
| test               |
+--------------------+
6 rows in set (0.00 sec)

slave相关命令：

show slave status； //查看全部slave状态

show slave status for channel ‘naem'; //查看单个slave状态

reset slave; #重置全部slave

reset slave for channel ‘master1'; #重置单个slave

stop slave for channel ‘master1'; #暂停单个slave

start slave for channel ‘master1'; #开启单个slave

虽然我在做的过程没有遇到错误，但是下面这个是最最容易出现的错误

配置完开启slave出现报错

mysql> start slave;
error 1872 (hy000): slave failed to initialize relay log info structure from the repository

解决问题

由于mysql.slave_relay_log_info表中保留了以前的复制信息，导致新从库启动时无法找到对应文件，那么我们清理掉该表中的记录即可

mysql> reset slave;
query ok, 0 rows affected (0.00 sec)

以上就是mysql示例dtid主从原理解析的详细内容，更多关于mysql示例dtid主从原理的资料请关注其它相关文章！