Uploaded image for project: 'Percona XtraDB Cluster'
  1. Percona XtraDB Cluster
  2. PXC-3999

Cluster stuck on "Waiting for acl cache lock" with concurrent User DDL commands

Details

    • Bug
    • Status: Done
    • Medium
    • Resolution: Fixed
    • 8.0.28-19 (Q1 2022)
    • 8.0.29-21 (Q2 2022)
    • None
    • Yes

    Description

      This stall affects PXC 8.0.28 only. Run a single node cluster and execute this mysqlslap command:

       

      mysqlslap --user=root --password=msandbox --host=127.0.0.1 --port=28829 --concurrency=32 --iterations=100 --create-schema test --query="CREATE USER IF NOT EXISTS msandbox@'127.%' IDENTIFIED BY 'test';ALTER USER msandbox@'127.%' IDENTIFIED BY 'msandbox'; DROP USER IF EXISTS msandbox@'127.%';" --verbose

      You may need to increase concurrency to get the cluster node to stall. Once stalled, the processlist will look like this:

      +-----+-----------------+-----------------+------+---------+------+------------------------------------------+---------------------------------------------------------------------+---------+-----------+---------------+
      +-----+-----------------+-----------------+------+---------+------+------------------------------------------+---------------------------------------------------------------------+---------+-----------+---------------+
      | Id  | User            | Host            | db   | Command | Time | State                                    | Info                                                                | Time_ms | Rows_sent | Rows_examined |
      +-----+-----------------+-----------------+------+---------+------+------------------------------------------+---------------------------------------------------------------------+---------+-----------+---------------+
      |   1 | system user     |                 | NULL | Sleep   | 1304 | wsrep aborter idle                       | NULL                                                                | 1304908 |         0 |             0 |
      |   2 | system user     |                 | NULL | Sleep   | 1304 | innobase_commit_low (-1)                 | NULL                                                                | 1304908 |         0 |             0 |
      |   7 | event_scheduler | localhost       | NULL | Daemon  | 1304 | Waiting on empty queue                   | NULL                                                                | 1303942 |         0 |             0 |
      |  10 | system user     |                 | NULL | Sleep   | 1304 | wsrep: applier idle                      | NULL                                                                | 1303939 |         0 |             0 |
      |  13 | msandbox        | localhost       | NULL | Sleep   |   76 |                                          | NULL                                                                |   76630 |         1 |             1 |
      | 185 | root            | localhost:36312 | NULL | Sleep   |    6 |                                          | NULL                                                                |    5935 |         0 |             0 |
      | 248 | root            | localhost:36470 | test | Query   |    2 | wsrep: initiating TOI for write set (-1) | CREATE USER IF NOT EXISTS 'msandbox'@'127.%' IDENTIFIED BY <secret> |    2717 |         0 |             0 |
      | 251 | root            | localhost:36442 | test | Query   |    2 | Waiting for acl cache lock               | CREATE USER IF NOT EXISTS 'msandbox'@'127.%' IDENTIFIED BY <secret> |    2740 |         0 |             0 |
      | 280 | root            | localhost:36508 | NULL | Query   |    0 | init                                     | show full processlist                                               |       0 |         0 |             0 |
      +-----+-----------------+-----------------+------+---------+------+------------------------------------------+---------------------------------------------------------------------+---------+-----------+---------------+

      pt-pmp.txt

       

       

      Attachments

        Activity

          People

            kamil.holubicki Kamil Holubicki
            jaime.sicam@percona.com Jaime Sicam
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Smart Checklist