Uploaded image for project: 'Percona Server for MySQL'
  1. Percona Server for MySQL
  2. PS-6035

Percona Server for MySQL 8.0 cr

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Priority: Medium
    • Resolution: Incomplete
    • Affects Version/s: 8.0.15-6
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      VERSION

      {{root@rc1b-servername ~ # mysqld --version
      /usr/sbin/mysqld Ver 8.0.15-6 for debian-linux-gnu on x86_64 (Percona Server (GPL), Release '6', Revision '63abd08')}}

      LOG

      {{2019-10-29T03:11:01.181910+03:00 275188 [Warning] [MY-010055] [Server] IP address '49.51.12.244' could not be resolved: Name or service not known
      2019-10-29T07:23:48.696248+03:00 4 [ERROR] [MY-010934] [Server] Out of memory (Needed 465429424 bytes)
      2019-10-29T07:23:48.720946+03:00 4 [ERROR] [MY-010934] [Server] Out of memory; check if mysqld or some other process uses all available memory; if not, you may have to use 'ulimit' to allow mysqld to use more memory or you can add more swap space
      2019-10-29T07:23:49.740330+03:00 4 [ERROR] [MY-010934] [Server] Out of memory (Needed 465429424 bytes)
      2019-10-29T07:23:49.741105+03:00 4 [ERROR] [MY-010934] [Server] Out of memory; check if mysqld or some other process uses all available memory; if not, you may have to use 'ulimit' to allow mysqld to use more memory or you can add more swap space
      04:23:49 UTC - mysqld got signal 11 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
      Attempting to collect some information that could help diagnose the problem.
      As this is a crash and something is definitely wrong, the information
      collection process might fail.
      Please help us make Percona Server better by reporting any
      bugs at https://bugs.percona.com/

      key_buffer_size=8388608
      read_buffer_size=131072
      max_used_connections=13
      max_threads=200
      thread_count=7
      connection_count=6
      It is possible that mysqld could use up to
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 87340 K bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.

      Thread pointer: 0x5626a82886a0
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 7fe058252d90 thread_stack 0x30000
      /usr/sbin/mysqld(my_print_stacktrace(unsigned char*, unsigned long)+0x3d) [0x5626a46f751d]
      /usr/sbin/mysqld(handle_fatal_signal+0x423) [0x5626a39ea653]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0x12890) [0x7fe06f3cf890]
      /usr/sbin/mysqld(lex_start(THD*)+0x11) [0x5626a389aba1]
      /usr/sbin/mysqld(dd::Update_dictionary_tables_ctx::Update_dictionary_tables_ctx(THD*)+0x2dd) [0x5626a44c07dd]
      /usr/sbin/mysqld(bool dd::cache::Storage_adapter::store<dd::Event>(THD*, dd::Event*)+0xb2) [0x5626a453bda2]
      /usr/sbin/mysqld(bool dd::cache::Dictionary_client::update<dd::Event>(dd::Event*)+0xd9) [0x5626a44eabd9]
      /usr/sbin/mysqld(Event_db_repository::update_timing_fields_for_event(THD*, MYSQL_LEX_STRING, MYSQL_LEX_STRING, long, unsigned long long)+0x1ae) [0x5626a3b622de]
      /usr/sbin/mysqld(Event_queue::get_top_for_execution_if_time(THD*, Event_queue_element_for_exec**)+0x1cf) [0x5626a3b6544f]
      /usr/sbin/mysqld(Event_scheduler::run(THD*)+0x156) [0x5626a3b67246]
      /usr/sbin/mysqld(+0x11ec534) [0x5626a3b67534]
      /usr/sbin/mysqld(+0x1e27bbf) [0x5626a47a2bbf]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fe06f3c46db]
      /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fe06d3ec88f]

      Trying to get some variables.
      Some pointers may be invalid and cause the dump to abort.
      Query (0): Connection ID (thread ID): 4
      Status: NOT_KILLED

      You may download the Percona Server operations manual by visiting
      http://www.percona.com/software/percona-server/. You may find information
      in the manual which will help you identify the cause of the crash.
      2019-10-29T07:23:52.546698+03:00 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.15-6) starting as process 533187
      2019-10-29T07:24:17.714586+03:00 1 [Warning] [MY-010005] [Server] Skip re-populating collations and character sets tables in read-only mode.
      2019-10-29T07:24:18.030617+03:00 0 [System] [MY-010229] [Server] Starting crash recovery...
      2019-10-29T07:24:18.045636+03:00 0 [System] [MY-010232] [Server] Crash recovery finished.
      2019-10-29T07:24:18.465983+03:00 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.15-6' socket: '/var/run/mysqld/mysqld.sock' port: 3306 Percona Server (GPL), Release '6', Revision '63abd08'.

      }}h3. MY.CNF
      {{[client]
      port = 3307
      socket = /var/run/mysqld/mysqld.sock

      [mysqld_safe]
      nice = 0
      socket = /var/run/mysqld/mysqld.sock
      log_error = /var/log/mysql/error.log
      timezone = Europe/Moscow
      log_timestamps = SYSTEM
      open_files_limit = 65535

      [mysqld]
      user = mysql
      port = 3306
      admin_port = 3307
      admin_address = rc1b-servername
      pid_file = /var/run/mysqld/mysqld.pid
      plugin-dir = /usr/lib/mysql/plugin
      socket = /var/run/mysqld/mysqld.sock
      log_error = /var/log/mysql/error.log
      general_log_file = /var/log/mysql/query.log
      basedir = /usr
      datadir = /var/lib/mysql
      tmpdir = /var/lib/mysql/.tmp
      lc_messages_dir = /usr/share/mysql
      max-connect-errors = 4294967295

      ssl=ON
      ssl-ca=/etc/mysql/ssl/allCAs.pem
      ssl-cert=/etc/mysql/ssl/server.crt
      ssl-key=/etc/mysql/ssl/server.key
      require-secure-transport=ON

      event_scheduler = ON
      skip_external_locking = ON
      default_storage_engine = InnoDB
      disabled_storage_engines="MyISAM,MRG_MyISAM,BLACKHOLE,FEDERATED,ARCHIVE,CSV"
      explicit_defaults_for_timestamp = ON
      log_timestamps = SYSTEM
      max_allowed_packet = 16777216
      thread_stack = 192K

      max_connections = 200
      thread_handling = one-thread-per-connection
      thread_cache_size = 5

      1. Slow log
        slow_query_log_file = /var/log/mysql/slow-query.log
      1. InnoDB settings
        innodb_buffer_pool_size = 2048M
        innodb_file_per_table = ON
        innodb_log_buffer_size = 16M
        innodb_log_file_size = 256M
        innodb_autoinc_lock_mode = 2
        innodb_use_native_aio = ON
        innodb_flush_method = O_DIRECT
        innodb_flush_log_at_trx_commit = 1
      1. SQL mode
        sql_mode = ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION,NO_DIR_IN_CREATE
      1. Replication
        server_id = 1
        binlog_format = ROW
        gtid_mode = ON
        enforce_gtid_consistency = ON
        log_bin_trust_function_creators = ON
        log_slave_updates = ON
        max_binlog_size = 100M
        binlog_expire_logs_seconds = 86400
        master_info_repository = TABLE
        relay_log_info_repository = TABLE
        relay_log_recovery = ON
        log_bin = mysql-bin-log-rc1b-servername
        log_bin_index = /var/lib/mysql/mysql-bin-log.index
        relay_log = mysql-relay-log-rc1b-servername
        relay_log_index = /var/lib/mysql/mysql-relay-log.index
      1. Semi-sync replication
        plugin_load_add = 'rpl_semi_sync_master=semisync_master.so;rpl_semi_sync_slave=semisync_slave.so'
        rpl_semi_sync_master_timeout = 31536000000
        rpl_semi_sync_master_wait_for_slave_count = 1
        rpl_semi_sync_master_wait_no_slave = ON
        rpl_semi_sync_master_wait_point = AFTER_SYNC

      read_only = ON
      super_read_only = ON
      slave_parallel_workers = 0
      report_host = rc1b-servername
      report_port = 3307

      [mysqldump]
      quick
      quote-names
      max_allowed_packet = 16M}}

      TYPICAL MEMORY STATS

      {{root@rc1b-servername ~ # free
      total used free shared buff/cache available
      Mem: 4046560 3195332 282260 4256 568968 608808
      Swap: 0 0}}

      QUERY

      It's highly possible that crash happens during handling scheduled event:

      {{CREATE DEFINER=`admin`@`localhost` EVENT `mdb_repl_mon_event` ON SCHEDULE EVERY 1 SECOND STARTS '2019-10-23 08:57:13' ON COMPLETION NOT PRESERVE ENABLE DO BEGIN
      SET SESSION innodb_lock_wait_timeout = 1;
      UPDATE mysql.mdb_repl_mon SET ts = CURRENT_TIMESTAMP(3) WHERE @@read_only = 0;
      END}}

        Smart Checklist

          Attachments

            Activity

              People

              Assignee:
              lalit.choudhary Lalit Choudhary
              Reporter:
              mialinx Dmitry Smal
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: