Uploaded image for project: 'Percona XtraDB Cluster'
  1. Percona XtraDB Cluster
  2. PXC-2543

WSREP: BF lock wait long

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Priority: High
    • Resolution: Incomplete
    • Affects Version/s: 5.7.x
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Environment:

      5.7.25-28-57-log Percona XtraDB Cluster (GPL), Release rel28, Revision a2ef85f, WSREP version 31.35, wsrep_31.35

      Linux stable-db7 3.16.0-6-amd64 #1 SMP Debian 3.16.57-2 (2018-07-14) x86_64 GNU/Linux

      Description

      We are running three node Percona XtraDB Cluster with three equal nodes. We are on version 5.7.25-28-57-log Percona XtraDB Cluster (GPL), Release rel28, Revision a2ef85f, WSREP version 31.35, wsrep_31.35.

      Since we started cluster operation we encounter the common problem of WSREP: BF lock wait long. The symptomps are commonly known and present on DB forums. The node causing the problem tries to commit transaction and it never finishes. Entire cluster stops writing changes. The only way out of this situation is hard kill the problematic node. This is happening several times a day. The hunging transactions are different from case to case. We don't have any massive TRUNCATE TABLE in the transaction causing the operation stop. The transactions being stuck are not any bigger (they are rather smaller ones) that other transactions in the system, which does not cause any problem.

      We were looking through different DB forums for solution. We were able to read several cases with exactly the same symptomps, but with no evidence of the cause, nor any glue to some fix to the problem.

      Is there anyone having some idea of how to fix this problem?

        Smart Checklist

          Attachments

          1. bf_lock_wait_logs.tgz
            23 kB
          2. crash_20190507.log
            2.74 MB
          3. my.cnf
            5 kB
          4. mysql_core_dump.gz
            173 kB
          5. var_innodb.txt
            5 kB
          6. var_wsrep.txt
            2 kB

            Activity

              People

              Assignee:
              Unassigned
              Reporter:
              hornych Michal Hornych
              Votes:
              3 Vote for this issue
              Watchers:
              11 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - Not Specified
                  Not Specified
                  Logged:
                  Time Spent - 1 day, 5 hours, 5 minutes
                  1d 5h 5m