Uploaded image for project: 'Percona Toolkit'
  1. Percona Toolkit
  2. PT-1630

pt-table-checksum not working with galera cluster anymore since 3.0.11

Details

    • Bug
    • Status: Done
    • Medium
    • Resolution: Fixed
    • 3.0.11, 3.0.12
    • 3.1.0
    • pt-table-checksum
      • Ubuntu 16.04
      • Percona XtraDB Cluster 5.7

    Description

      Hi,

      we have a XtraDB Cluster with Galera replication running and use pt-table-checksum to verify the sync.
      Up to version 3.0.10 all works fine but with v3.0.11 and the current 3.0.12 there are issues to check the replication.

      I checked the differences to 3.0.10 and I identified the new function wait_for_slaves as source of the issues.
      When I comment out the function call than all works fine again.

      This function seems to work with other replication types but not with galera. Please fix.

       

      # dpkg -l | grep percona
      ii percona-galera-3 3.21-0ubuntu0.16.04.1 amd64 Galera replication framework for Percona XtraDB Cluster
      ii percona-release 0.1-4.xenial all Package to install Percona gpg key and APT repo
      ii percona-toolkit 3.0.12-1.xenial amd64 Advanced MySQL and system command-line tools
      ii percona-xtrabackup-24 2.4.12-1.xenial amd64 Open source backup tool for InnoDB and XtraDB
      ii percona-xtradb-cluster-client-5.7 5.7.23-31.31-2.xenial amd64 Percona XtraDB Cluster database client binaries
      ii percona-xtradb-cluster-common-5.7 5.7.23-31.31-2.xenial amd64 Percona XtraDB Cluster database common files (e.g. /etc/mysql/my.cnf)
      ii percona-xtradb-cluster-server-5.7 5.7.23-31.31-2.xenial amd64 Percona XtraDB Cluster database server binaries
      
      # Original: 3.0.12 pt-table-checksum
      # pt-table-checksum --databases phpmyadmin --replicate=percona.checksums --recursion-method=dsn=h=${HOST},D=percona,t=dsns,u=${MYSQLUSER},p=${MYSQLPASS}
      Checking if all tables can be checksummed ...
      Starting checksum ...
      Not checking replica lag on mysql9.db.cgn.ps-intern.de because it is a cluster node.
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__bookmark: Use of uninitialized value in concatenation (.) or string at /usr/bin/pt-table-checksum line 5521.
      
      TS ERRORS DIFFS ROWS DIFF_ROWS CHUNKS SKIPPED TIME TABLE
      10-15T13:51:04 1 0 0 0 1 0 0.015 phpmyadmin.pma__bookmark
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__central_columns: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.010 phpmyadmin.pma__central_columns
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__column_info: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.010 phpmyadmin.pma__column_info
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__designer_settings: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.010 phpmyadmin.pma__designer_settings
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__export_templates: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.009 phpmyadmin.pma__export_templates
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__favorite: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.008 phpmyadmin.pma__favorite
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__history: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.008 phpmyadmin.pma__history
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__navigationhiding: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.009 phpmyadmin.pma__navigationhiding
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__pdf_pages: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.009 phpmyadmin.pma__pdf_pages
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__recent: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.009 phpmyadmin.pma__recent
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__relation: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.008 phpmyadmin.pma__relation
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__savedsearches: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.008 phpmyadmin.pma__savedsearches
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__table_coords: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.009 phpmyadmin.pma__table_coords
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__table_info: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.009 phpmyadmin.pma__table_info
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__table_uiprefs: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 1 0 1 0 0.009 phpmyadmin.pma__table_uiprefs
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__tracking: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.009 phpmyadmin.pma__tracking
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__userconfig: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 6 0 1 0 0.008 phpmyadmin.pma__userconfig
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__usergroups: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.007 phpmyadmin.pma__usergroups
      10-15T13:51:04 Error checksumming table phpmyadmin.pma__users: I need a master_status argument at /usr/bin/pt-table-checksum line 5500.
      
      10-15T13:51:04 1 0 0 0 1 0 0.008 phpmyadmin.pma__users
      

       

      # Modified version of 3.0.12, diff below
      # pt-table-checksum.new --databases phpmyadmin --replicate=percona.checksums --recursion-method=dsn=h=${HOST},D=percona,t=dsns,u=${MYSQLUSER},p=${MYSQLPASS}
      Checking if all tables can be checksummed ...
      Starting checksum ...
      Not checking replica lag on mysql9.db.cgn.ps-intern.de because it is a cluster node.
       TS ERRORS DIFFS ROWS DIFF_ROWS CHUNKS SKIPPED TIME TABLE
      10-15T13:52:35 0 0 0 0 1 0 0.010 phpmyadmin.pma__bookmark
      10-15T13:52:35 0 0 0 0 1 0 0.011 phpmyadmin.pma__central_columns
      10-15T13:52:35 0 0 0 0 1 0 0.011 phpmyadmin.pma__column_info
      10-15T13:52:35 0 0 0 0 1 0 0.010 phpmyadmin.pma__designer_settings
      10-15T13:52:35 0 0 0 0 1 0 0.011 phpmyadmin.pma__export_templates
      10-15T13:52:35 0 0 0 0 1 0 0.009 phpmyadmin.pma__favorite
      10-15T13:52:35 0 0 0 0 1 0 0.010 phpmyadmin.pma__history
      10-15T13:52:35 0 0 0 0 1 0 0.011 phpmyadmin.pma__navigationhiding
      10-15T13:52:35 0 0 0 0 1 0 0.009 phpmyadmin.pma__pdf_pages
      10-15T13:52:35 0 0 0 0 1 0 0.011 phpmyadmin.pma__recent
      10-15T13:52:35 0 0 0 0 1 0 0.011 phpmyadmin.pma__relation
      10-15T13:52:35 0 0 0 0 1 0 0.010 phpmyadmin.pma__savedsearches
      10-15T13:52:35 0 0 0 0 1 0 0.010 phpmyadmin.pma__table_coords
      10-15T13:52:35 0 0 0 0 1 0 0.009 phpmyadmin.pma__table_info
      10-15T13:52:35 0 0 1 0 1 0 0.010 phpmyadmin.pma__table_uiprefs
      10-15T13:52:35 0 0 0 0 1 0 0.011 phpmyadmin.pma__tracking
      10-15T13:52:35 0 0 6 0 1 0 0.010 phpmyadmin.pma__userconfig
      10-15T13:52:35 0 0 0 0 1 0 0.010 phpmyadmin.pma__usergroups
      10-15T13:52:35 0 0 0 0 1 0 0.010 phpmyadmin.pma__users
      
      # diff -ruN /usr/bin/pt-table-checksum /usr/bin/pt-table-checksum.new
      --- /usr/bin/pt-table-checksum 2018-09-13 12:29:46.000000000 +0000
      +++ /usr/bin/pt-table-checksum.new 2018-10-15 13:52:26.839469005 +0000
      @@ -11298,7 +11298,7 @@
       # MySQL 8+ replication is slower than 5.7 and the old wait_for_last_checksum alone
       # was failing. The new wait_for_slaves checks that Read_Master_Log_Pos on slaves is
       # greather or equal Position in the master
      - wait_for_slaves(master_dbh => $args\{Cxn}->dbh(), master_slave => $ms, slaves => $slaves);
      + #wait_for_slaves(master_dbh => $args\{Cxn}->dbh(), master_slave => $ms, slaves => $slaves);
       wait_for_last_checksum(
       tbl => $tbl,
       repl_table => $repl_table,
      

      Attachments

        1. mysql.tar.gz
          4 kB
        2. phpmyadmin.sql
          20 kB
        3. pt-checksum-log.txt
          90 kB
        4. pt-table-checksum.debug
          227 kB

        Activity

          People

            carlos.salguero Carlos Salguero (Inactive)
            mjaepel Marcel Jäpel
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - Not Specified
                Not Specified
                Logged:
                Time Spent - 1 day, 7 hours
                1d 7h

                Smart Checklist