Uploaded image for project: 'Percona XtraBackup'
  1. Percona XtraBackup
  2. PXB-1704

Xtrabackup deadlocks on insert buffer merge

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Priority: Medium
    • Resolution: Fixed
    • Affects Version/s: 2.4.12
    • Fix Version/s: 2.4.13, 8.0.4
    • Component/s: None
    • Labels:
      None

      Description

      Xtrabackup version:

      /usr/bin/xtrabackup version 2.4.12 based on MySQL server 5.7.19 Linux (x86_64) (revision id: 170eb8c)

      Stacktrace of the deadlock:

       

      Thread 11 (LWP 49259):
       #01 0x00007fb0396de0b0 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread-2.19.so
       #02 0x00000000007c1916 in os_event::wait_low () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/os/os0event.cc:165
       #03 0x000000000078a753 in sync_array_wait_event () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/sync/sync0arr.cc:476
       #04 0x0000000000984bde in rw_lock_s_lock_spin () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/sync/sync0rw.cc:425
       #05 0x00000000008462ee in buf_page_get_gen () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/include/sync0rw.ic:433
       #06 0x00000000007d4777 in btr_cur_search_to_nth_level () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/btr/btr0cur.cc:1107
       #07 0x00000000009b5e7e in btr_pcur_open_on_user_rec_func () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/include/btr0pcur.ic:465
       #08 0x00000000009d6c83 in ibuf_merge_or_delete_for_page () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/ibuf/ibuf0ibuf.cc:4552
       #09 0x000000000084489e in buf_page_io_complete () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/buf/buf0buf.cc:5919
       #10 0x000000000090ca1f in fil_aio_wait () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/fil/fil0fil.cc:6150
       #11 0x000000000081b9c8 in io_handler_thread () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/srv/srv0start.cc:315
       #12 0x00007fb0396da0ca in start_thread () from /lib/x86_64-linux-gnu/libpthread-2.19.so
      Thread 10 (LWP 49260):
       #01 0x00007fb0396de0b0 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread-2.19.so
       #02 0x00000000007c1916 in os_event::wait_low () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/os/os0event.cc:165
       #03 0x000000000088a0f6 in os_aio_handler () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/os/os0file.cc:8066
       #04 0x000000000090c932 in fil_aio_wait () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/fil/fil0fil.cc:6116
       #05 0x000000000081b9c8 in io_handler_thread () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/srv/srv0start.cc:315
       #06 0x00007fb0396da0ca in start_thread () from /lib/x86_64-linux-gnu/libpthread-2.19.so
      Thread 9 (LWP 49261):
       #01 0x00007fb0396e11ad in __nanosleep_nocancel () from /lib/x86_64-linux-gnu/libpthread-2.19.so
       #02 0x0000000000869960 in os_thread_sleep () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/os/os0thread.cc:279
       #03 0x000000000096a6ad in recv_apply_hashed_log_recs () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/log/log0recv.cc:2743
       #04 0x000000000079ba78 in log_checkpoint () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/log/log0log.cc:1932
       #05 0x000000000079dcfd in log_margin_checkpoint_age () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/log/log0log.cc:335
       #06 0x00000000007b3d78 in mtr_t::Command::prepare_write () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/mtr/mtr0mtr.cc:878
       #07 0x00000000007b5709 in mtr_t::Command::execute () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/mtr/mtr0mtr.cc:958
       #08 0x00000000007b59d9 in mtr_t::commit () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/mtr/mtr0mtr.cc:576
       #09 0x00000000009d7763 in ibuf_merge_or_delete_for_page () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/include/btr0pcur.ic:386
       #10 0x000000000084489e in buf_page_io_complete () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/buf/buf0buf.cc:5919
       #11 0x000000000090ca1f in fil_aio_wait () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/fil/fil0fil.cc:6150
       #12 0x000000000081b9c8 in io_handler_thread () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/srv/srv0start.cc:315
       #13 0x00007fb0396da0ca in start_thread () from /lib/x86_64-linux-gnu/libpthread-2.19.so
       Thread 1 (LWP 49256):
       #01 0x00007fb0396e11ad in __nanosleep_nocancel () from /lib/x86_64-linux-gnu/libpthread-2.19.so
       #02 0x0000000000869960 in os_thread_sleep () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/os/os0thread.cc:279
       #03 0x0000000000989680 in rtr_page_split_and_insert () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/include/mach0data.ic:88
       #04 0x00000000008c24df in buf_read_recv_pages () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/buf/buf0rea.cc:867
       #05 0x000000000096b4fa in recv_apply_hashed_log_recs () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/log/log0recv.cc:2709
       #06 0x000000000082500c in innobase_start_or_create_for_mysql () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/srv/srv0start.cc:2287
       #07 0x00000000007198a9 in innodb_init () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/xtrabackup/src/xtrabackup.cc:2084
       #08 0x0000000000725305 in xtrabackup_prepare_func () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/xtrabackup/src/xtrabackup.cc:7746
       #09 0x000000000070337e in main () from /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/min-jessie-x64/percona-xtrabackup-24-2.4.12/storage/innobase/xtrabackup/src/xtrabackup.cc:8954
      

       
      Symptoms are the same as in: https://jira.percona.com/browse/PXB-1467

        Smart Checklist

          Attachments

            Activity

              People

              • Assignee:
                sergei.glushchenko Sergei Glushchenko
                Reporter:
                firsov Yevgeniy Firsov
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 1 day, 4 hours
                  1d 4h