Uploaded image for project: 'Percona XtraDB Cluster'
  1. Percona XtraDB Cluster
  2. PXC-2079

Crash during rollback of a stored procedure called in an event

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Priority: Medium
    • Resolution: Fixed
    • Affects Version/s: 5.6.x, 5.7.x
    • Fix Version/s: 5.6.40-26.25, 5.7.22-29.26
    • Component/s: None
    • Labels:
      None

      Description

      Forum bug:
      https://www.percona.com/forums/questions-discussions/percona-xtradb-cluster/51072-percona-xtradb-cluster-5-7-19-bug-or-not

      Repro:
      1) Create an event which calls a stored proc
      2) When the event runs on node1, run an operation on node2 that updates the rows used by the stored proc (force a BF on node1)

      Result:
      Crash on node1.

      2018-03-14T12:25:00.689691Z 16 [Note] WSREP: Assigned new next trx-id (3867) to Store-Procedure execution
      2018-03-14T12:25:18.956176Z 2 [Note] WSREP: wsrep: applying write-set (3760)
      2018-03-14T12:25:18.958353Z 2 [Note] WSREP: --------- CONFLICT DETECTED --------
      2018-03-14T12:25:18.958378Z 2 [Note] WSREP: cluster conflict due to high priority abort for threads:

      2018-03-14T12:25:18.958386Z 2 [Note] WSREP: Winning thread:
      THD: 2, mode: applier, state: executing, conflict: no conflict, seqno: 3760
      SQL: (null)

      2018-03-14T12:25:18.958391Z 2 [Note] WSREP: Victim thread:
      THD: 16, mode: local, state: idle, conflict: no conflict, seqno: -1
      SQL: update sbtest.sbtest1 set n = 1 where pad like '1%1'

      2018-03-14T12:25:18.958397Z 2 [Note] WSREP: BF thread 2 (with write-set: 3760) aborting Victim thread 16 with transaction (5306)
      2018-03-14T12:25:18.958401Z 2 [Note] WSREP: Aborting query: update sbtest.sbtest1 set n = 1 where pad like '1%1'
      2018-03-14T12:25:18.958407Z 2 [Note] WSREP: Killing Transaction (5306) in QUERY_IDLE state
      2018-03-14T12:25:18.958424Z 2 [Note] WSREP: Enqueue transaction for force-abort 16
      2018-03-14T12:25:18.958428Z 2 [Note] WSREP: Signal Rollback/Aborter
      2018-03-14T12:25:18.958960Z 1 [Note] WSREP: Rollback/Aborter thread wakes for signal. Has some transaction rollback work to do.
      2018-03-14T12:25:18.959118Z 16 [Note] WSREP: Initiating rollback. Transaction (query: update sbtest.sbtest1 set n = 1 where pad like '1%1') triggered by thread 16 was BF aborted
      2018-03-14 02:25:18 0x7fe22c785700 InnoDB: Assertion failure in thread 140609385420544 in file row0upd.cc line 3299
      InnoDB: Failing assertion: !thr_get_trx(thr)->in_rollback
      InnoDB: We intentionally generate a memory trap.
      InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
      InnoDB: If you get repeated assertion failures or crashes, even
      InnoDB: immediately after the mysqld startup, there may be
      InnoDB: corruption in the InnoDB tablespace. Please refer to
      InnoDB: http://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html
      InnoDB: about forcing recovery.
      12:25:18 UTC - mysqld got signal 6 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
      Attempting to collect some information that could help diagnose the problem.
      As this is a crash and something is definitely wrong, the information
      collection process might fail.
      Please help us make Percona XtraDB Cluster better by reporting any
      bugs at https://jira.percona.com/projects/PXC/issues

      key_buffer_size=8388608
      read_buffer_size=131072
      max_used_connections=4
      max_threads=152
      thread_count=6
      connection_count=2
      It is possible that mysqld could use up to
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 68414 K bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.

      Thread pointer: 0x607e0006cd00
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 7fe22c784a70 thread_stack 0x40000
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(my_print_stacktrace+0x66)[0x1e8351c]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(handle_fatal_signal+0x485)[0x7c2594]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0x10330)[0x7fe24e79b330]
      /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x37)[0x7fe24dbd8c37]
      /lib/x86_64-linux-gnu/libc.so.6(abort+0x148)[0x7fe24dbdc028]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug[0x2499b97]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(_Z7row_updP10upd_node_tP9que_thr_t+0x153)[0x236f817]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(_Z12row_upd_stepP9que_thr_t+0x46d)[0x23705c8]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug[0x22b69d5]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(_Z20row_update_for_mysqlPKhP14row_prebuilt_t+0x167b)[0x22c155e]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(_ZN11ha_innobase10update_rowEPKhPh+0x972)[0x2037d0e]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(_ZN7handler13ha_update_rowEPKhPh+0x3ce)[0x952a7c]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(Z12mysql_updateP3THDR4ListI4ItemES4_y15enum_duplicatesPyS6+0x4b54)[0x18cf1e2]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(_ZN14Sql_cmd_update23try_single_table_updateEP3THDPb+0x5ab)[0x18d6a47]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(_ZN14Sql_cmd_update7executeEP3THD+0x116)[0x18d731c]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(_Z21mysql_execute_commandP3THDb+0xd82b)[0x1709a74]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(_ZN13sp_instr_stmt9exec_coreEP3THDPj+0x12c)[0x154baa2]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(_ZN12sp_lex_instr23reset_lex_and_exec_coreEP3THDPjb+0xf46)[0x1551492]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(_ZN12sp_lex_instr29validate_lex_and_execute_coreEP3THDPjb+0x677)[0x1553777]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(_ZN13sp_instr_stmt7executeEP3THDPj+0x79c)[0x1556784]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(_ZN7sp_head7executeEP3THDb+0x2272)[0x15389c4]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(_ZN7sp_head17execute_procedureEP3THDP4ListI4ItemE+0x1294)[0x1546ab2]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(_Z21mysql_execute_commandP3THDb+0x168cd)[0x1712b16]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(_ZN13sp_instr_stmt9exec_coreEP3THDPj+0x12c)[0x154baa2]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(_ZN12sp_lex_instr23reset_lex_and_exec_coreEP3THDPjb+0xf46)[0x1551492]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(_ZN12sp_lex_instr29validate_lex_and_execute_coreEP3THDPjb+0x677)[0x1553777]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(_ZN13sp_instr_stmt7executeEP3THDPj+0x79c)[0x1556784]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(_ZN7sp_head7executeEP3THDb+0x2272)[0x15389c4]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(_ZN7sp_head17execute_procedureEP3THDP4ListI4ItemE+0x1294)[0x1546ab2]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(_ZN14Event_job_data7executeEP3THDb+0x1b68)[0x19d5030]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(_ZN19Event_worker_thread3runEP3THDP28Event_queue_element_for_exec+0x3dd)[0x1c7692d]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(event_worker_thread+0xd7)[0x1c76ce2]
      /home/kennt/dev/pxc/build-bin/bin/mysqld-debug(pfs_spawn_thread+0x45b)[0x1ec48b0]
      /usr/lib/x86_64-linux-gnu/libasan.so.0(+0x18b98)[0x7fe24fa5cb98]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0x8184)[0x7fe24e793184]
      /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fe24dca003d]

      Trying to get some variables.
      Some pointers may be invalid and cause the dump to abort.
      Query (6016007c8ca0): update sbtest.sbtest1 set n = 1 where pad like '1%1'
      Connection ID (thread ID): 16
      Status: NOT_KILLED

      You may download the Percona XtraDB Cluster operations manual by visiting
      http://www.percona.com/software/percona-xtradb-cluster/. You may find information
      in the manual which will help you identify the cause of the crash.

        Attachments

          Activity

            People

            Assignee:
            kenn.takara Kenn Takara
            Reporter:
            kenn.takara Kenn Takara
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 1 week, 2 days, 5 hours
                1w 2d 5h

                  Smart Checklist