Uploaded image for project: 'Percona XtraBackup'
  1. Percona XtraBackup
  2. PXB-1915

MyRocks backups feature request -- renewing checkpoints

    XMLWordPrintable

    Details

    • Type: New Feature
    • Status: Done
    • Priority: Medium
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 8.0.8
    • Component/s: None
    • Labels:
      None
    • Needs QA:
      Yes
    • Needs Doc:
      Yes

      Description

      The purpose is to minimize amount of binary logs to apply after the backup was completed.

      Server keep changing the data while xtrabackup copies SST files. By the time xtrabackup completes copying, the amount of changes can become huge. If we want to setup new slave from the backup, we will need to apply a lot of binary logs.

      The idea is to copy as less data as we can after we captured binlog coordinates. For that we will create initial checkpoint and copy all SST files into a backup. Then we will create new checkpoint and copy only missing SST files. We can do several iterations until the last copying phase become short enough (should be configurable).

      Implementation will be something like:

      1. LOCK TABLES FOR BACKUP
      2. Repeat every X minutes, until reaching last round:
      2.1 Ask myrocks to create temporary checkpoint
      2.2. Copy the SST files that are not copied yet from the temporary checkpoint.
      Rmember which SST files are copied, so that it does not have to send the same files multiple times.
      2.3. Delete temporary checkpoint
      Last round of backup (same as current implementation):
      3. Ask myrocks to disable file deletions
      4. Ask myrocks to create temporary checkpoint
      5. FLUSH BINARY LOGS
      6. Query p_s.log_status for InnoDB and binlog coordinates as well as
      rocksdb wal file names and sizes
      7. Copy MySQL binary log
      8. Copy live WAL files
      9. Enable file deletions
      10. UNLOCK TABLES
      11. Copy the remaining files from the temporary checkpoint
      12. Delete temporary checkpoint

        Smart Checklist

          Attachments

            Activity

              People

              Assignee:
              Unassigned
              Reporter:
              sergei.glushchenko Sergei Glushchenko (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - Not Specified
                  Not Specified
                  Logged:
                  Time Spent - 1 week, 25 minutes
                  1w 25m