Uploaded image for project: 'Percona Server'
  1. Percona Server
  2. PS-5424

MyRocks - rocksdb_manual_wal_flush doesn't appear to change WAL flushing behavior

    Details

    • Type: Bug
    • Status: On Hold
    • Priority: Medium
    • Resolution: Unresolved
    • Affects Version/s: Not 5.6
    • Fix Version/s: None
    • Component/s: MyRocks
    • Labels:
      None
    • Environment:

      Description

      Per the reference found here...

      https://www.percona.com/doc/percona-server/5.7/myrocks/variables.html#rocksdb_manual_wal_flush

      The description for this variable is as follows...

      This variable can be used to disable automatic/timed WAL flushing and instead rely on the application to do the flushing.

      My interpretation is that if rocksdb_manual_wal_flush is enabled that flushing and syncing of the WAL via automatic processes would cease. I decided to test this in order to better understand under what circumstances I could cause the WAL to flush with this enabled.

      In my first test I set rocksdb_manual_wal_flush to 1, rocksdb_flush_log_at_trx_commit to 1, and rocksdb_wal_bytes_per_sync to 0.  I performed 100 inserts and expected 0 WAL syncs given that rocksdb_manual_wal_flush was enabled. This turned out to be false as there was 1 sync per commit.

      mysql> show global variables like 'rocksdb_manual_wal_flush';
      +--------------------------+-------+
      | Variable_name            | Value |
      +--------------------------+-------+
      | rocksdb_manual_wal_flush | ON    |
      +--------------------------+-------+
      1 row in set (0.00 sec)mysql> show global variables like 'rocksdb_flush_log_at_trx_commit';
      +---------------------------------+-------+
      | Variable_name                   | Value |
      +---------------------------------+-------+
      | rocksdb_flush_log_at_trx_commit | 1     |
      +---------------------------------+-------+
      1 row in set (0.00 sec)mysql> show global variables like 'Rocksdb_wal_bytes_per_sync';
      +----------------------------+-------+
      | Variable_name              | Value |
      +----------------------------+-------+
      | rocksdb_wal_bytes_per_sync | 0     |
      +----------------------------+-------+
      1 row in set (0.00 sec)mysql> show global status like 'rocks%wal%';
      +-------------------------+-------+
      | Variable_name           | Value |
      +-------------------------+-------+
      | rocksdb_wal_bytes       | 60    |
      | rocksdb_wal_group_syncs | 2     |
      | rocksdb_wal_synced      | 4     |
      | rocksdb_write_wal       | 4     |
      +-------------------------+-------+
      4 rows in set (0.00 sec)mysql> use localtest;
      Reading table information for completion of table and column names
      You can turn off this feature to get a quicker startup with -ADatabase changed
      mysql> select count(*) from t1;
      +----------+
      | count(*) |
      +----------+
      |     1503 |
      +----------+
      1 row in set (0.13 sec)mysql> #started write test
      mysql> select count(*) from t1;
      +----------+
      | count(*) |
      +----------+
      |     1603 |
      +----------+
      1 row in set (0.00 sec)mysql> show global status like 'rocks%wal%';
      +-------------------------+--------+
      | Variable_name           | Value  |
      +-------------------------+--------+
      | rocksdb_wal_bytes       | 235270 |
      | rocksdb_wal_group_syncs | 2      |
      | rocksdb_wal_synced      | 105    |
      | rocksdb_write_wal       | 206    |
      +-------------------------+--------+
      4 rows in set (0.00 sec)
      

      For my second test I set rocksdb_manual_wal_flush =1, rocksdb_flush_log_at_trx_commit = 2, and did not change rocksdb_wal_bytes_per_sync. As soon as I did this I checked the status of rocksdb_wal_synced and it started increment once per second.

      mysql> set global rocksdb_flush_log_at_trx_commit = 2;
      Query OK, 0 rows affected (0.00 sec)mysql> show global status like 'rocks%wal%';
      +-------------------------+--------+
      | Variable_name           | Value  |
      +-------------------------+--------+
      | rocksdb_wal_bytes       | 235270 |
      | rocksdb_wal_group_syncs | 0      |
      | rocksdb_wal_synced      | 111    |
      | rocksdb_write_wal       | 206    |
      +-------------------------+--------+
      4 rows in set (0.01 sec)mysql> show global status like 'rocks%wal%';
      +-------------------------+--------+
      | Variable_name           | Value  |
      +-------------------------+--------+
      | rocksdb_wal_bytes       | 235270 |
      | rocksdb_wal_group_syncs | 0      |
      | rocksdb_wal_synced      | 113    |
      | rocksdb_write_wal       | 206    |
      +-------------------------+--------+
      4 rows in set (0.00 sec)mysql> show global status like 'rocks%wal%';
      +-------------------------+--------+
      | Variable_name           | Value  |
      +-------------------------+--------+
      | rocksdb_wal_bytes       | 235270 |
      | rocksdb_wal_group_syncs | 0      |
      | rocksdb_wal_synced      | 114    |
      | rocksdb_write_wal       | 206    |
      +-------------------------+--------+
      4 rows in set (0.01 sec)mysql> show global status like 'rocks%wal%';
      +-------------------------+--------+
      | Variable_name           | Value  |
      +-------------------------+--------+
      | rocksdb_wal_bytes       | 235270 |
      | rocksdb_wal_group_syncs | 0      |
      | rocksdb_wal_synced      | 115    |
      | rocksdb_write_wal       | 206    |
      +-------------------------+--------+
      4 rows in set (0.00 sec)
      

      In conclusion it looks like setting rocksdb_manual_wal_flush doesn't appear to disable automatic WAL flushing or syncing based on the data available in global status.

      Also, the documentation says it's reliant on the application to do the flushing, but it doesn't really appear to say how that's done. I've tested with a few commands at the MySQL CLI and was not able to manually prompt a flush / sync without performing an transaction.

        Smart Checklist

          Attachments

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                Sylvester Peter Sylvester
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: