Uploaded image for project: 'Percona XtraBackup'
  1. Percona XtraBackup
  2. PXB-1913

PXB crashes in incremental backup prepare when a multi valued index is added/dropped for json data

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Priority: Medium
    • Resolution: Fixed
    • Affects Version/s: 8.0.7
    • Fix Version/s: 8.0.8
    • Component/s: None
    • Labels:
      None

      Description

      Start MS8.0.17 and create innodb data in database. Use sysbench to create 10 tables with about 3000 rows in each table.
      Create another database, add data and then drop it

      echo "Create a database test1_innodb, add data and then drop it"
       ( for ((i=1; i<=3; i++)); do
       # Check if database is up otherwise exit the loop
       ${mysqldir}/bin//mysqladmin ping --user=root --socket=${mysqldir}/socket.sock 2>/dev/null 1>&2
       if [ "$?" -ne 0 ]; then
       break
       fi
       ${mysqldir}/bin/mysql -uroot -S${mysqldir}/socket.sock -e "CREATE DATABASE IF NOT EXISTS test1_innodb;" >/dev/null 2>&1
       sysbench /usr/share/sysbench/oltp_insert.lua --tables=1 --table-size=1000 --mysql-db=test1_innodb --mysql-user=root --threads=10 --db-driver=mysql --mysql-socket=${mysqldir}/socket.sock prepare >/dev/null 2>&1
       ${mysqldir}/bin/mysql -uroot -S${mysqldir}/socket.sock -e "ALTER TABLE test1_innodb.sbtest1 ADD COLUMN b JSON AS('{\"k1\": \"value\", \"k2\": [10, 20]}');" >/dev/null 2>&1
       # Create a multivalue index
       ${mysqldir}/bin/mysql -uroot -S${mysqldir}/socket.sock -e "CREATE INDEX jindex on test1_innodb.sbtest1( (CAST(b->'$.k2' AS UNSIGNED ARRAY)) );"
       ${mysqldir}/bin/mysql -uroot -S${mysqldir}/socket.sock -e "DROP INDEX jindex on test1_innodb.sbtest1;"
       ${mysqldir}/bin/mysql -uroot -S${mysqldir}/socket.sock -e "ALTER TABLE test1_innodb.sbtest1 DROP COLUMN b;" >/dev/null 2>&1
       ${mysqldir}/bin/mysql -uroot -S${mysqldir}/socket.sock -e "DROP DATABASE test1_innodb;" >/dev/null 2>&1
       done ) &

      Take full backup with lock-ddl

      ${xtrabackup_dir}/xtrabackup --user=root --password='' --backup --target-dir=${backup_dir}/full -S ${mysqldir}/socket.sock --datadir=${datadir} --lock-ddl

      Add data in database

      sysbench /usr/share/sysbench/oltp_insert.lua --tables=10 --mysql-db=test --mysql-user=root --threads=50 --db-driver=mysql --mysql-socket=${mysqldir}/socket.sock --time=20 run >/dev/null 2>&1 &
      
      sleep 10

      Take incremental backup with lock-ddl

      ${xtrabackup_dir}/xtrabackup --user=root --password='' --backup --target-dir=${backup_dir}/inc --incremental-basedir=${backup_dir}/full -S ${mysqldir}/socket.sock --datadir=${datadir} --lock-ddl

      Prepare full backup

      ${xtrabackup_dir}/xtrabackup --user=root --password='' --prepare --apply-log-only --target_dir=${backup_dir}/full

      Prepare incremental backup

      ${xtrabackup_dir}/xtrabackup --user=root --password='' --prepare --target_dir=${backup_dir}/full --incremental-dir=${backup_dir}/inc

      PXB crashes

      Allocated tablespace ID 1 for sys/sys_config, old maximum was 0
      InnoDB: Assertion failure: dict0dict.cc:2213:!field->prefix_len || field->fixed_len == field->prefix_len
      InnoDB: thread 140291076622720InnoDB: We intentionally generate a memory trap.
      InnoDB: Submit a detailed bug report to https://jira.percona.com/projects/PXB.
      InnoDB: If you get repeated assertion failures or crashes, even
      InnoDB: immediately after the mysqld startup, there may be
      InnoDB: corruption in the InnoDB tablespace. Please refer to
      InnoDB: http://dev.mysql.com/doc/refman/8.0/en/forcing-innodb-recovery.html
      InnoDB: about forcing recovery.
      11:06:33 UTC - mysqld got signal 6 ;
      Most likely, you have hit a bug, but this error can also be caused by malfunctioning hardware.
      Thread pointer: 0x98b0230
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 7fff0edfc6c8 thread_stack 0x46000
      /home/mchawla/pxb_8_0_8_debug/bin/xtrabackup(my_print_stacktrace(unsigned char*, unsigned long)+0x43) [0x45fb661]
      /home/mchawla/pxb_8_0_8_debug/bin/xtrabackup(handle_fatal_signal+0x291) [0x30a0521]
      /usr/lib64/libpthread.so.0(+0xf5d0) [0x7f980f88a5d0]
      /usr/lib64/libc.so.6(gsignal+0x37) [0x7f980d5042c7]
      /usr/lib64/libc.so.6(abort+0x148) [0x7f980d5059b8]
      /home/mchawla/pxb_8_0_8_debug/bin/xtrabackup() [0x36e58f3]
      /home/mchawla/pxb_8_0_8_debug/bin/xtrabackup() [0x32d07fe]
      /home/mchawla/pxb_8_0_8_debug/bin/xtrabackup(dict_index_add_to_cache_w_vcol(dict_table_t*, dict_index_t*, dict_add_v_col_t const*, unsigned int, unsigned long)+0x452) [0x32d1073]
      /home/mchawla/pxb_8_0_8_debug/bin/xtrabackup(dict_index_add_to_cache(dict_table_t*, dict_index_t*, unsigned int, unsigned long)+0x6f) [0x32d0abe]
      /home/mchawla/pxb_8_0_8_debug/bin/xtrabackup(dd_table_create_on_dd_obj(dd::Table const*, dd::Partition const*, std::basic_string<char, std::char_traits<char>, Stateless_allocator<char, dd::String_type_alloc, My_free_functor> > const*, bool)+0x2224) [0x32ef1c7]
      /home/mchawla/pxb_8_0_8_debug/bin/xtrabackup(dd_table_load_part(unsigned long, dd::Table const&, dd::Partition const*, dict_table_t*&, THD*, std::basic_string<char, std::char_traits<char>, Stateless_allocator<char, dd::String_type_alloc, My_free_functor> > const*, bool)+0x21a) [0x32f093a]
      /home/mchawla/pxb_8_0_8_debug/bin/xtrabackup(dd_table_load_on_dd_obj(dd::cache::Dictionary_client*, unsigned int, dd::Table const&, dict_table_t*&, THD*, std::basic_string<char, std::char_traits<char>, Stateless_allocator<char, dd::String_type_alloc, My_free_functor> > const*, bool)+0x1ad) [0x32f0d64]
      /home/mchawla/pxb_8_0_8_debug/bin/xtrabackup(dict_load_tables_from_space_id(unsigned int, THD*, trx_t*)+0x41c) [0x295e1bc]
      /home/mchawla/pxb_8_0_8_debug/bin/xtrabackup() [0x295e60f]
      /home/mchawla/pxb_8_0_8_debug/bin/xtrabackup() [0x295e80a]
      /home/mchawla/pxb_8_0_8_debug/bin/xtrabackup() [0x296ac81]
      /home/mchawla/pxb_8_0_8_debug/bin/xtrabackup(main+0x883) [0x296d958]
      /usr/lib64/libc.so.6(__libc_start_main+0xf5) [0x7f980d4f0495]
      /home/mchawla/pxb_8_0_8_debug/bin/xtrabackup() [0x2957955]

      Note: The issue is reproducible 1 out of 2 times on qaserver-02.

        Smart Checklist

          Attachments

            Activity

              People

              Assignee:
              sergei.glushchenko Sergei Glushchenko (Inactive)
              Reporter:
              manish.chawla Manish Chawla
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - Not Specified
                  Not Specified
                  Logged:
                  Time Spent - 2 days, 2 hours, 17 minutes
                  2d 2h 17m