MySQL示例DTID主從原理解析

Posted on by

1.GTID基本概念

MySQL 5.6.5開始支持的，全局事務標識符（GTID（Global Transaction ID））是創建的唯一標識符，並與在源（主）服務器上提交的每個事務相關聯。
此標識符不但是唯一的，而且在給定復制設置中的所有服務器上都是唯一的。
所有交易和所有GTID之間都有一對一的映射關系 。
它由服務器ID以及事務ID組合而成。
這個全局事務ID不僅僅在原始服務器上唯一，在所有存在主從關系 的mysql服務器上也是唯一的。
正是因為這樣一個特性使得mysql的主從復制變得更加簡單，以及數據庫一致性更可靠。
一個GTID在一個服務器上隻執行一次，避免重復執行導致數據混亂或者主從不一致。

2.GTID優點

保證同一個事務在某slave上絕對隻執行一次，沒有執行過的gtid事務總是會被執行。
不用像傳統復制那樣保證binlog的坐標準確，因為根本不需要binlog以及坐標。
故障轉移到新的master的時候很方便，簡化瞭很多任務。
很容易判斷master和slave的數據是否一致。隻要master上提交的事務在slave上也提交瞭，那麼一定是一致的。

3.GTID的工作原理

在這裡插入圖片描述

1.當一個事務在主庫端執行並提交時，產生GTID，一同記錄到binlog日志中。
2.binlog傳輸到slave,並存儲到slave的relaylog後，讀取這個GTID的這個值設置gtid_next變量，即告訴Slave，下一個要執行的GTID值。
3、sql線程從relay log中獲取GTID，然後對比slave端的binlog是否有該GTID。
4、如果有記錄，說明該GTID的事務已經執行，slave會忽略。
5、如果沒有記錄，slave就會執行該GTID事務，並記錄該GTID到自身的binlog，在讀取執行事務前會先檢查其他session持有該GTID，確保不被重復執行。
6、在解析過程中會判斷是否有主鍵，如果有就用二級索引，如果沒有就用全部掃描。

4.GTID比傳統復制的優勢

1.更簡單的實現故障轉移（failover），不需要找log_file,log_pos

2.更簡單的搭建主從復制

3.更加安全

4.GTID是連續沒有空洞的，因此主數據庫發生沖突時，可以添加空事件的方式進行跳過

5.啟動的方法

  • 方法一：如果是新搭建的服務器，直接啟動即可
  • 方法二：如果是以及跑的服務器，需要重啟一下mysql server

啟動前，先關閉master的寫入，保證master端和slave端數據保持同步，所有slave需要加上skip_slave_start=1的配置參數，避免啟動後還是使用之前的復制協議

6.GTID(一主一從)配置

6.1環境：

centos8.0 ip：192.168.136.239 有數據 hostname：mysql01

centos8.0 ip：192.168.136.219 無數據 hostname：mysql02

#二進制安裝以及mysql自啟動服務略

6.2在主庫上給從庫授權：

mysql> grant replication slave on *.* to 'slave'@'192.168.136.219' identified by 'slave';
Query OK, 0 rows affected, 1 warning (0.00 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
#倆服務器均關閉防火墻
[root@mysql01 ~]# systemctl stop firewalld
[root@mysql01 ~]# setenforce 0
[root@mysql02 ~]# systemctl stop firewalld
[root@mysql02 ~]# setenforce 0
從庫測試連接：
[root@mysql02 ~]# mysql -u slave -p'slave' -h192.168.136.239
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>

6.3確保數據一致操作

1.對主庫進行鎖表
mysql> flush tables with read lock;
2.對主庫進行全備
[root@mysql01 ~]# mysqldump -uroot -A > /clq/all-databases-20210519.sql
3.拷貝到從庫主機上去
[root@mysql01 ~]# scp /clq/all-databases-20210519.sql [email protected]:/backup/
[root@mysql02 backup]# ll
-rw-r--r--. 1 root root 873527 5月  19 16:40 all-databases-20210519.sql
4.從庫上進行主庫的恢復
[root@mysql02 backup]# mysql -uroot -pHuawei0917@ < all-databases-20210519.sql

6.4配置主庫

[mysqld]
basedir = /usr/local/mysql
datadir = /opt/data
socket = /tmp/mysql.sock
port = 3306
user = mysql
pid-file = /opt/data/mysql.pid
skip-name-resolve
#skip-grant-tables
log-bin = master_bin   #開啟主庫日志
server-id = 10        #服務唯一標識id
gtid-mode = on        #GTID模式開啟
enforce_gtid_consistency = on #強制gtid模式一致性
log-slave-updates = 1    #從庫允許更新日志，同步操作日志
binlog_format = row    #binlog日志格式為行格式, 默認是mixed混合模式
skip_slave_start = 1   #跳過從庫開啟，以主庫開始開啟
#重啟
systemctl restart mysqld

6.5配置從庫

[root@mysql02 data]# cat /etc/my.cnf
[mysqld]
basedir = /usr/local/mysql 
datadir = /opt/data 
socket = /tmp/mysql.sock 
port = 3306
user = mysql
pid-file = /opt/data/mysql.pid
skip-name-resolve
#skip-grant-tables 
gtid_mode=on
enforce_gtid_consistency=on
server-id=20
log-bin=slave_binlog       #開啟從庫日志
log_slave-updates=1        #從庫允許更新
binlog_format=row          #格式為行
skip-slave_start=1   
#重啟
systemctl restart mysqld

查看gtid狀態情況

mysql> show variables like '%gtid%';
+----------------------------------+-----------+
| Variable_name                    | Value     |
+----------------------------------+-----------+
| binlog_gtid_simple_recovery      | ON        |
| enforce_gtid_consistency         | ON        |
| gtid_executed_compression_period | 1000      |
| gtid_mode                        | ON        |
| gtid_next                        | AUTOMATIC |
| gtid_owned                       |           |
| gtid_purged                      |           |
| session_track_gtids              | OFF       |
+----------------------------------+-----------+
8 rows in set (0.00 sec)

6.6配置主從復制

#從庫上root登錄配置      #help change master to 可以查看幫助文檔實例
mysql> change master to
    -> master_host='192.168.136.239',
    -> master_user='slave',
    -> master_password='slave',
    -> master_port=3306,        #主庫端口
    -> master_auto_position=1;  #位置
                                       #master_use_gtid = current_pos
Query OK, 0 rows affected, 2 warnings (0.01 sec)
mysql> start slave;  
Query OK, 0 rows affected (0.00 sec)
mysql> show slave status\G; 
Slave_IO_Running: Connecting
            Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
保證系統一致性 
授權一致性

（一主一從GTID）測試：

主庫創建一個數據庫test，進行測試查看

從庫創建一個數據庫test02，進行測試查看

#主庫創建一個test數據庫
mysql> create database test;
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
| test               |
+--------------------+
#從庫上查看同步情況
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
| test               |
+--------------------+
6 rows in set (0.00 sec)

#從庫創建test02庫
mysql> create database test02;
Query OK, 1 row affected (0.00 sec)
#主庫上查看
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |         #是沒有test02庫的
| test               |
+--------------------+
5 rows in set (0.00 sec)

小結：主庫上的數據操作會同步到從庫上面去，而從庫上的數據操作與主庫沒聯系

7.GTID（一主倆從）

第三臺mysql連接的話，相應配置

第3臺mysql ，版本:centos8 ip:192.168.136.230 主機名:mysql03

[root@mysql03 ~]# cat /etc/my.cnf 
[mysqld]
basedir = /usr/local/mysql
datadir = /opt/data
socket = /tmp/mysql.sock
port = 3306
user = mysql
pid-file = /opt/data/mysql.pid
skip-name-resolve
#skip-grant-tables
# replication config
log-bin = master_bin
server-id = 21               #id必須與之前不同
gtid-mode = on
enforce-gtid-consistency = on
log-slave-updates = 1
binlog-format = row
skip-slave-start = 1
#查看gtid情況
mysql> show variables like '%gtid%';
+----------------------------------+-----------+
| Variable_name                    | Value     |
+----------------------------------+-----------+
| binlog_gtid_simple_recovery      | ON        |
| enforce_gtid_consistency         | ON        |
| gtid_executed_compression_period | 1000      |
| gtid_mode                        | ON        |
| gtid_next                        | AUTOMATIC |
| gtid_owned                       |           |
| gtid_purged                      |           |
| session_track_gtids              | OFF       |
+----------------------------------+-----------+
#由於之前隻權限瞭一個ip，此刻在mysql01主數據庫上再授權一個ip
mysql> grant replication slave on *.* to 'slave'@'192.168.136.230' identified by 'slave';
Query OK, 0 rows affected, 1 warning (0.00 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
#測試連接
[root@mysql ~]#  mysql -uslave -pslave -h192.168.136.239
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 17
Server version: 5.7.33-log MySQL Community Server (GPL)
Copyright (c) 2000, 2021, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> 
#mysql03從庫上root用戶連接進行相應配置
[root@mysql03 ~]#  mysql -uroot -p1
mysql> change master to
    -> master_host='192.168.136.239',  #主庫ip
    -> master_user='slave',              #主庫授權的普通用戶
    -> master_password='slave',
    -> master_port=3306,              #主庫端口
    -> master_auto_position=1;   #位置從1開始同步
#也可以查看幫助進行配置
mysql> help change master to;
CHANGE MASTER TO
  MASTER_HOST='source2.example.com',
  MASTER_USER='replication',
  MASTER_PASSWORD='password',
  MASTER_PORT=3306,
  MASTER_LOG_FILE='source2-bin.001',
  MASTER_LOG_POS=4,
  MASTER_CONNECT_RETRY=10;
URL: https://dev.mysql.com/doc/refman/5.7/en/change-master-to.html

#開啟
mysql> start slave;
mysql> show slave status\G;
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 192.168.136.239
                  Master_User: slave
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: master_bin.000002
          Read_Master_Log_Pos: 2172
               Relay_Log_File: mysql-relay-bin.000002
                Relay_Log_Pos: 2387
        Relay_Master_Log_File: master_bin.000002
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes    #顯示倆個yes則運行成功！
#mysql03查看數據庫，數據庫內容也同步成功
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
| test               |
+--------------------+
5 rows in set (0.00 sec)

8.GTID（倆主一從）

1.最新環境

版本 ip 主機名 身份
centos8 192.168.136.239 master01 主庫
centos8 192.168.136.219 master02 主庫
centos8 192.168.136.230 slave 從庫

2.所有服務器均關閉防火墻或者放行防火墻

[root@master01 ~]# systemctl stop firewalld
[root@master01 ~]# systemctl disable firewalld
[root@master02 ~]# systemctl stop firewalld
[root@master02 ~]# systemctl disable firewalld
[root@slave ~]# systemctl stop firewalld
[root@slave ~]# systemctl disable firewalld

3.授權連接

master01庫授權普通用戶

mysql> grant replication slave on *.* to  'user'@'192.168.136.%' identified by 'user';

slave進行連接

[root@slave ~]# mysql -uuser -p'user' -h192.168.136.239
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 5
Server version: 5.7.33 MySQL Community Server (GPL)
Copyright (c) 2000, 2021, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

master02授權普通用戶

mysql> grant replication slave on *.* to  'app'@'192.168.136.%' identified by 'app';
Query OK, 0 rows affected, 1 warning (0.01 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

slave進行連接

[root@slave ~]# mysql -uapp -papp -h192.168.136.219
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.7.33 MySQL Community Server (GPL)
Copyright (c) 2000, 2021, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>

4.分別進行配置文件修改

#master01主機：
[root@master01 ~]# cat /etc/my.cnf 
[mysqld]
basedir = /usr/local/mysql
datadir = /opt/data
socket = /tmp/mysql.sock
port = 3306
user = mysql
pid-file = /opt/data/mysql.pid
skip-name-resolve
skip-grant-tables
log-bin = master_bin
server-id = 10
gtid-mode = on
enforce-gtid-consistency = on
log-slave-updates = 1
binlog-format = row
skip-slave-start = 1
#master02主機
[mysqld]
basedir = /usr/local/mysql
datadir = /opt/data
socket = /tmp/mysql.sock
port = 3306
user = mysql
pid-file = /opt/data/mysql.pid
skip-name-resolve
                      #replication config
log-bin = master_bin
server-id = 11
gtid-mode = on
enforce-gtid-consistency = on
log-slave-updates = 1
binlog-format = row
skip-slave-start = 1
#slave主機
[mysqld]
basedir = /usr/local/mysql
datadir = /opt/data
socket = /tmp/mysql.sock
port = 3306
user = mysql
pid-file = /opt/data/mysql.pid
skip-name-resolve
log-bin = slave_bin
server-id = 13
gtid-mode = on
enforce-gtid-consistency = on
log-slave-updates = 1
binlog-format = row
skip-slave-start = 1

5.分別重啟

[root@master01 ~]# systemctl restart mysqld
[root@master02 ~]# systemctl restart mysqld
[root@slave ~]# systemctl restart mysqld

6.在進行GTID多主一從配置前，先引入一個概念

channel(頻道)：每一個channel都是一個獨立的slave服務，都有一個IO_THREAD和SQL_THREAD，原理和普通復制一樣，隻是需要在change master to語句後面使用FOR Channel來進行區分slave

在使用channel時需要將從庫的master-info-repository、relay-log-info-repository設置為table，否則會報錯。

將信息存儲庫設置為table格式

方式一（mysql內設置）:
set global master_info_repository='table';
set global relay_log_info_repository='table';
方式二（/etc/my.cnf內設置）:
3.在my.cnf中設置
master_info_repository    = TABLE 
relay_log_info_repository = TABLE   
#檢查是否更改成功
mysql> show variables where variable_name in  ('relay_log_info_repository','master_info_repository');
+---------------------------+-------+
| Variable_name             | Value |
+---------------------------+-------+
| master_info_repository    | TABLE |
| relay_log_info_repository | TABLE |
+---------------------------+-------+

7.slave從庫以root用戶登錄進行GTID配置

#slave從庫上配置倆個主庫GTID復制
mysql> change master to
    -> master_host='192.168.136.219',  #mysql02主庫ip
    -> master_user='app',              #mysql02主庫授權的普通用戶
    -> master_password='app',           #mysql02主庫授權的普通用戶密碼
    -> master_port=3306,              #主庫端口
    -> master_auto_position=1 for channel 'master01';   #位置從1開始同步,並且第一個slave取名master01
mysql> change master to
    -> master_host='192.168.136.239',  #mysql01主庫ip
    -> master_user='user',              
    -> master_password='user',          
    -> master_port=3306,              #主庫端口
    -> master_auto_position=1 for channel 'master02';   #位置從1開始同步,並且第一個slave取名master01
#查看倆個slave狀態                   
mysql> show slave status\G;
*************************** 1. row ***************************
               Slave_IO_State: 
                  Master_Host: 192.168.136.219
                  Master_User: app
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: 
          Read_Master_Log_Pos: 4
               Relay_Log_File: slave02-relay-bin-master1.000001
                Relay_Log_Pos: 4
        Relay_Master_Log_File: 
             Slave_IO_Running: No
            Slave_SQL_Running: No         #都是關閉的
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 0
              Relay_Log_Space: 154
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 0
                  Master_UUID: 
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: 
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 
            Executed_Gtid_Set: b4326a77-0a31-11ec-a991-000c298d3571:1-2,
d68b404d-0a35-11ec-9df1-000c29581959:1
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: master1
           Master_TLS_Version: 
*************************** 2. row ***************************
               Slave_IO_State: 
                  Master_Host: 192.168.136.239
                  Master_User: user
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: 
          Read_Master_Log_Pos: 4
               Relay_Log_File: slave02-relay-bin-master2.000001
                Relay_Log_Pos: 4
        Relay_Master_Log_File: 
             Slave_IO_Running: No
            Slave_SQL_Running: No
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 0
              Relay_Log_Space: 154
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 0
                  Master_UUID: 
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: 
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 
            Executed_Gtid_Set: b4326a77-0a31-11ec-a991-000c298d3571:1-2,
d68b404d-0a35-11ec-9df1-000c29581959:1
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: master2
           Master_TLS_Version: 
2 rows in set (0.00 sec)
#開啟倆個slave
mysql> start slave;
#再次查看狀態

GTID(倆主一從)測試：

#master01主庫創建一個test數據庫
mysql> create database test;
Query OK, 1 row affected (0.00 sec)
mysql>  show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
| test               |
+--------------------+
5 rows in set (0.00 sec)
#master02主庫上查看
mysql>  show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |                  #沒有內容
+--------------------+
4 rows in set (0.00 sec)
#slave從庫查看
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
| test               |          #已經同步瞭test庫
+--------------------+
5 rows in set (0.00 sec)
#mysql02主庫創建一個RHCA數據庫
mysql> create database RHCA;
Query OK, 1 row affected (0.01 sec)
mysql>  show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| RHCA               |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
5 rows in set (0.00 sec)
#slave從庫
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| RHCA               |
| mysql              |
| performance_schema |
| sys                |             #有瞭mysql01主庫的test庫和mysql02的RHCA的庫
| test               |
+--------------------+
6 rows in set (0.00 sec)

slave相關命令：

show slave status； //查看全部slave狀態

show slave status for channel ‘naem’; //查看單個slave狀態

reset slave; #重置全部slave

reset slave for channel ‘master1′; #重置單個slave

stop slave for channel ‘master1′; #暫停單個slave

start slave for channel ‘master1′; #開啟單個slave

雖然我在做的過程沒有遇到錯誤，但是下面這個是最最容易出現的錯誤

配置完開啟slave出現報錯

mysql> start slave;
ERROR 1872 (HY000): Slave failed to initialize relay log info structure from the repository

解決問題

由於mysql.slave_relay_log_info表中保留瞭以前的復制信息，導致新從庫啟動時無法找到對應文件，那麼我們清理掉該表中的記錄即可

mysql> reset slave;
Query OK, 0 rows affected (0.00 sec)

以上就是MySQL示例DTID主從原理解析的詳細內容，更多關於MySQL示例DTID主從原理的資料請關註WalkonNet其它相關文章！

推薦閱讀：