Uploaded image for project: 'Percona XtraDB Cluster'
  1. Percona XtraDB Cluster
  2. PXC-816

Regression in GTID consistency when parallel applying used for async replication

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Priority: Medium
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      LP bug: https://bugs.launchpad.net/percona-xtradb-cluster/+bug/1681831

      After upgrade from PXC 5.7.16 to 5.7.17, parallel async slave leads to inconsistency in GTIDs on the cluster peers. Events replicated by the async slave node get assigned the Galera's cluster UUID instead of the original async master's UUIDs. This only happens when slave_parallel_worker>1.

      Test case is simple - two nodes PXC cluster, where one of the nodes is slave of standalone master. Do couple of updates (to a different data set) on the PXC nodes and on the async master.
      Example tests:

      node1> select @@version,@@version_comment\G
      *************************** 1. row ***************************
              @@version: 5.7.16-10-57-log
      @@version_comment: Percona XtraDB Cluster (GPL), Release rel10, Revision bec0879, WSREP version 27.19, wsrep_27.19
      1 row in set (0.00 sec)
      
      node1> show global variables like 'slave_par%';
      +------------------------+----------+
      | Variable_name | Value |
      +------------------------+----------+
      | slave_parallel_type | DATABASE |
      | slave_parallel_workers | 5 |
      +------------------------+----------+
      2 rows in set (0.00 sec)
      
      master> show master status\G
      *************************** 1. row ***************************
                   File: mysql-binlog.000001
               Position: 1399
           Binlog_Do_DB:
       Binlog_Ignore_DB:
      Executed_Gtid_Set: e893fd74-1090-11e7-9df2-0242ac150002:1-5
      1 row in set (0.00 sec)
      
      node1> show global variables like 'gtid_executed'\G
      *************************** 1. row ***************************
      Variable_name: gtid_executed
              Value: 5d8a1eab-ef88-ee18-4891-fd62d95ebf12:1-4,
      e893fd74-1090-11e7-9df2-0242ac150002:1-5
      1 row in set (0.00 sec)
      
      node2> show global variables like 'gtid_executed'\G
      *************************** 1. row ***************************
      Variable_name: gtid_executed
              Value: 5d8a1eab-ef88-ee18-4891-fd62d95ebf12:1-4,
      e893fd74-1090-11e7-9df2-0242ac150002:1-5
      1 row in set (0.00 sec)
      
      node1> show status like 'wsrep_cluster_state_uuid';
      +--------------------------+--------------------------------------+
      | Variable_name | Value |
      +--------------------------+--------------------------------------+
      | wsrep_cluster_state_uuid | a275e154-1077-11e7-b76e-029d26a140ed |
      +--------------------------+--------------------------------------+
      1 row in set (0.01 sec)
      
      -- same test in 5.7.17
      
      master> show master status\G
      

      After upgrade from PXC 5.7.16 to 5.7.17, parallel async slave leads to inconsistency in GTIDs on the cluster peers. Events replicated by the async slave node get assigned the Galera's cluster UUID instead of the original async master's UUIDs. This only happens when slave_parallel_worker>1.

      Test case is simple - two nodes PXC cluster, where one of the nodes is slave of standalone master. Do couple of updates (to a different data set) on the PXC nodes and on the async master.
      Example tests:

      node1> select @@version,@@version_comment\G
              @@version: 5.7.16-10-57-log
      @@version_comment: Percona XtraDB Cluster (GPL), Release rel10, Revision bec0879, WSREP version 27.19, wsrep_27.19
      1 row in set (0.00 sec)
      
      node1> show global variables like 'slave_par%';
      +------------------------+----------+
      | Variable_name | Value |
      +------------------------+----------+
      | slave_parallel_type | DATABASE |
      | slave_parallel_workers | 5 |
      +------------------------+----------+
      2 rows in set (0.00 sec)
      
      master> show master status\G
                   File: mysql-binlog.000001
               Position: 1399
           Binlog_Do_DB:
       Binlog_Ignore_DB:
      Executed_Gtid_Set: e893fd74-1090-11e7-9df2-0242ac150002:1-5
      1 row in set (0.00 sec)
      
      node1> show global variables like 'gtid_executed'\G
      Variable_name: gtid_executed
              Value: 5d8a1eab-ef88-ee18-4891-fd62d95ebf12:1-4,
      e893fd74-1090-11e7-9df2-0242ac150002:1-5
      1 row in set (0.00 sec)
      
      node2> show global variables like 'gtid_executed'\G
      Variable_name: gtid_executed
              Value: 5d8a1eab-ef88-ee18-4891-fd62d95ebf12:1-4,
      e893fd74-1090-11e7-9df2-0242ac150002:1-5
      1 row in set (0.00 sec)
      
      node1> show status like 'wsrep_cluster_state_uuid';
      +--------------------------+--------------------------------------+
      | Variable_name | Value |
      +--------------------------+--------------------------------------+
      | wsrep_cluster_state_uuid | a275e154-1077-11e7-b76e-029d26a140ed |
      +--------------------------+--------------------------------------+
      1 row in set (0.01 sec)
      
      -- same test in 5.7.17
      
      master> show master status\G
                   File: mysql-binlog.000001
               Position: 901
           Binlog_Do_DB:
       Binlog_Ignore_DB:
      Executed_Gtid_Set: e893fd74-1090-11e7-9df2-0242ac150002:1-3
      1 row in set (0.00 sec)
      
      node1> select @@version,@@version_comment\G
              @@version: 5.7.17-11-57-log
      @@version_comment: Percona XtraDB Cluster (GPL), Release rel11, Revision e2a7fdd, WSREP version 27.20, wsrep_27.20
      1 row in set (0.00 sec)
      
      node1> show global variables like 'gtid_executed'\G
      Variable_name: gtid_executed
              Value: 5d8a1eab-ef88-ee18-4891-fd62d95ebf12:1-4,
      e893fd74-1090-11e7-9df2-0242ac150002:1-3
      1 row in set (0.00 sec)
      
      node2> show global variables like 'gtid_executed'\G
      Variable_name: gtid_executed
              Value: 5d8a1eab-ef88-ee18-4891-fd62d95ebf12:1-7
      1 row in set (0.00 sec)             File: mysql-binlog.000001
               Position: 901
           Binlog_Do_DB:
       Binlog_Ignore_DB:
      Executed_Gtid_Set: e893fd74-1090-11e7-9df2-0242ac150002:1-3
      1 row in set (0.00 sec)
      
      node1> select @@version,@@version_comment\G
              @@version: 5.7.17-11-57-log
      @@version_comment: Percona XtraDB Cluster (GPL), Release rel11, Revision e2a7fdd, WSREP version 27.20, wsrep_27.20
      1 row in set (0.00 sec)
      
      node1> show global variables like 'gtid_executed'\G
      Variable_name: gtid_executed
              Value: 5d8a1eab-ef88-ee18-4891-fd62d95ebf12:1-4,
      e893fd74-1090-11e7-9df2-0242ac150002:1-3
      1 row in set (0.00 sec)
      
      node2> show global variables like 'gtid_executed'\G
      Variable_name: gtid_executed
              Value: 5d8a1eab-ef88-ee18-4891-fd62d95ebf12:1-7
      1 row in set (0.00 sec)
      

        Smart Checklist

          Attachments

            Activity

              People

              Assignee:
              kenn.takara Kenn Takara
              Reporter:
              kenn.takara Kenn Takara
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: