Uploaded image for project: 'Percona Toolkit'
  1. Percona Toolkit
  2. PT-1630

pt-table-checksum not working with galera cluster anymore since 3.0.11

    Details

    • Type: Bug
    • Status: Pending Release
    • Priority: Medium
    • Resolution: Fixed
    • Affects Version/s: 3.0.11, 3.0.12
    • Fix Version/s: 3.1.0
    • Component/s: pt-table-checksum
    • Environment:
      • Ubuntu 16.04
      • Percona XtraDB Cluster 5.7

      Description

      Hi,

      we have a XtraDB Cluster with Galera replication running and use pt-table-checksum to verify the sync.
      Up to version 3.0.10 all works fine but with v3.0.11 and the current 3.0.12 there are issues to check the replication.

      I checked the differences to 3.0.10 and I identified the new function wait_for_slaves as source of the issues.
      When I comment out the function call than all works fine again.

      This function seems to work with other replication types but not with galera. Please fix.

       

      # dpkg -l | grep percona
      ii percona-galera-3 3.21-0ubuntu0.16.04.1 amd64 Galera replication framework for Percona XtraDB Cluster
      ii percona-release 0.1-4.xenial all Package to install Percona gpg key and APT repo
      ii percona-toolkit 3.0.12-1.xenial amd64 Advanced MySQL and system command-line tools
      ii percona-xtrabackup-24 2.4.12-1.xenial amd64 Open source backup tool for InnoDB and XtraDB
      ii percona-xtradb-cluster-client-5.7 5.7.23-31.31-2.xenial amd64 Percona XtraDB Cluster database client binaries
      ii percona-xtradb-cluster-common-5.7 5.7.23-31.31-2.xenial amd64 Percona XtraDB Cluster database common files (e.g. /etc/mysql/my.cnf)
      ii percona-xtradb-cluster-server-5.7 5.7.23-31.31-2.xenial amd64 Percona XtraDB Cluster database server binaries
      
      # Original: 3.0.12 pt-table-checksum
      # pt-table-checksum --databases phpmyadmin --replicate=percona.checksums --recursion-method=dsn=h=${HOST},D=percona,t=dsns,u=${MYSQLUSER},p=${MYSQLPASS}
      Checking if all tables can be checksummed ...
      Starting checksum ...
      Not checking replica lag on mysql9.db.cgn.ps-intern.de because it is a cluster node.
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__bookmark: Use of uninitialized value in concatenation (.) or string at /usr/bin/pt-table-checksum line 5521.
      
      TS ERRORS DIFFS ROWS DIFF_ROWS CHUNKS SKIPPED TIME TABLE
      10-15T13:51:04 1 0 0 0 1 0 0.015 phpmyadmin.pma__bookmark
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__central_columns: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.010 phpmyadmin.pma__central_columns
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__column_info: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.010 phpmyadmin.pma__column_info
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__designer_settings: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.010 phpmyadmin.pma__designer_settings
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__export_templates: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.009 phpmyadmin.pma__export_templates
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__favorite: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.008 phpmyadmin.pma__favorite
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__history: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.008 phpmyadmin.pma__history
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__navigationhiding: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.009 phpmyadmin.pma__navigationhiding
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__pdf_pages: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.009 phpmyadmin.pma__pdf_pages
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__recent: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.009 phpmyadmin.pma__recent
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__relation: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.008 phpmyadmin.pma__relation
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__savedsearches: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.008 phpmyadmin.pma__savedsearches
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__table_coords: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.009 phpmyadmin.pma__table_coords
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__table_info: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.009 phpmyadmin.pma__table_info
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__table_uiprefs: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 1 0 1 0 0.009 phpmyadmin.pma__table_uiprefs
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__tracking: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.009 phpmyadmin.pma__tracking
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__userconfig: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 6 0 1 0 0.008 phpmyadmin.pma__userconfig
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__usergroups: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.007 phpmyadmin.pma__usergroups
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__users: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.008 phpmyadmin.pma__users
      

       

      # Modified version of 3.0.12, diff below
      # pt-table-checksum.new --databases phpmyadmin --replicate=percona.checksums --recursion-method=dsn=h=${HOST},D=percona,t=dsns,u=${MYSQLUSER},p=${MYSQLPASS}
      Checking if all tables can be checksummed ...
      Starting checksum ...
      Not checking replica lag on mysql9.db.cgn.ps-intern.de because it is a cluster node.
       TS ERRORS DIFFS ROWS DIFF_ROWS CHUNKS SKIPPED TIME TABLE
      10-15T13:52:35 0 0 0 0 1 0 0.010 phpmyadmin.pma__bookmark
      10-15T13:52:35 0 0 0 0 1 0 0.011 phpmyadmin.pma__central_columns
      10-15T13:52:35 0 0 0 0 1 0 0.011 phpmyadmin.pma__column_info
      10-15T13:52:35 0 0 0 0 1 0 0.010 phpmyadmin.pma__designer_settings
      10-15T13:52:35 0 0 0 0 1 0 0.011 phpmyadmin.pma__export_templates
      10-15T13:52:35 0 0 0 0 1 0 0.009 phpmyadmin.pma__favorite
      10-15T13:52:35 0 0 0 0 1 0 0.010 phpmyadmin.pma__history
      10-15T13:52:35 0 0 0 0 1 0 0.011 phpmyadmin.pma__navigationhiding
      10-15T13:52:35 0 0 0 0 1 0 0.009 phpmyadmin.pma__pdf_pages
      10-15T13:52:35 0 0 0 0 1 0 0.011 phpmyadmin.pma__recent
      10-15T13:52:35 0 0 0 0 1 0 0.011 phpmyadmin.pma__relation
      10-15T13:52:35 0 0 0 0 1 0 0.010 phpmyadmin.pma__savedsearches
      10-15T13:52:35 0 0 0 0 1 0 0.010 phpmyadmin.pma__table_coords
      10-15T13:52:35 0 0 0 0 1 0 0.009 phpmyadmin.pma__table_info
      10-15T13:52:35 0 0 1 0 1 0 0.010 phpmyadmin.pma__table_uiprefs
      10-15T13:52:35 0 0 0 0 1 0 0.011 phpmyadmin.pma__tracking
      10-15T13:52:35 0 0 6 0 1 0 0.010 phpmyadmin.pma__userconfig
      10-15T13:52:35 0 0 0 0 1 0 0.010 phpmyadmin.pma__usergroups
      10-15T13:52:35 0 0 0 0 1 0 0.010 phpmyadmin.pma__users
      
      # diff -ruN /usr/bin/pt-table-checksum /usr/bin/pt-table-checksum.new
      --- /usr/bin/pt-table-checksum 2018-09-13 12:29:46.000000000 +0000
      +++ /usr/bin/pt-table-checksum.new 2018-10-15 13:52:26.839469005 +0000
      @@ -11298,7 +11298,7 @@
       # MySQL 8+ replication is slower than 5.7 and the old wait_for_last_checksum alone
       # was failing. The new wait_for_slaves checks that Read_Master_Log_Pos on slaves is
       # greather or equal Position in the master
      - wait_for_slaves(master_dbh => $args\{Cxn}->dbh(), master_slave => $ms, slaves => $slaves);
      + #wait_for_slaves(master_dbh => $args\{Cxn}->dbh(), master_slave => $ms, slaves => $slaves);
       wait_for_last_checksum(
       tbl => $tbl,
       repl_table => $repl_table,
      

        Smart Checklist

          Attachments

          1. mysql.tar.gz
            4 kB
          2. phpmyadmin.sql
            20 kB
          3. pt-checksum-log.txt
            90 kB
          4. pt-table-checksum.debug
            227 kB

            Activity

              People

              • Assignee:
                carlos.salguero Carlos Salguero
                Reporter:
                mjaepel Marcel Jäpel
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - Not Specified
                  Not Specified
                  Logged:
                  Time Spent - 1 day, 7 hours
                  1d 7h