Uploaded image for project: 'Percona Server'
  1. Percona Server
  2. PS-5085

Starting the server with rocksdb-allow-mmap-reads and rocksdb-use-direct-io-for-flush-and-compaction enabled can lead to a crash on a debug build

    XMLWordPrintable

    Details

      Description

      startup options required:

      --plugin-load-add=RocksDB=ha_rocksdb.so --rocksdb-allow-mmap-reads=1 --rocksdb-use-direct-io-for-flush-and-compaction=1
      

      gdb:

      (gdb) bt
      +bt
      #0  0x00007fc675952a01 in __pthread_kill (threadid=<optimized out>, signo=6) at ../nptl/sysdeps/unix/sysv/linux/pthread_kill.c:61
      #1  0x000000000189ded9 in my_write_core (sig=6) at /home/hrvoje/worktable/PS-5.7.24-26_dbg/mysys/stacktrace.c:249
      #2  0x0000000001617f8b in handle_fatal_signal (sig=6) at /home/hrvoje/worktable/PS-5.7.24-26_dbg/sql/signal_handler.cc:223
      #3  <signal handler called>
      #4  0x00007fc673a56277 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
      #5  0x00007fc673a57968 in __GI_abort () at abort.c:90
      #6  0x00007fc673a4f096 in __assert_fail_base (fmt=0x7fc673baa580 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x7fc635e92f7f "!options.use_direct_reads", file=file@entry=0x7fc635e92cb8 "/home/hrvoje/worktable/PS-5.7.24-26_dbg/storage/rocksdb/rocksdb/env/io_posix.cc", line=line@entry=439, function=function@entry=0x7fc635e94ec0 <rocksdb::PosixMmapReadableFile::PosixMmapReadableFile(int, std::string const&, void*, unsigned long, rocksdb::EnvOptions const&)::__PRETTY_FUNCTION__> "rocksdb::PosixMmapReadableFile::PosixMmapReadableFile(int, const string&, void*, size_t, const rocksdb::EnvOptions&)") at assert.c:92
      #7  0x00007fc673a4f142 in __GI___assert_fail (assertion=0x7fc635e92f7f "!options.use_direct_reads", file=0x7fc635e92cb8 "/home/hrvoje/worktable/PS-5.7.24-26_dbg/storage/rocksdb/rocksdb/env/io_posix.cc", line=439, function=0x7fc635e94ec0 <rocksdb::PosixMmapReadableFile::PosixMmapReadableFile(int, std::string const&, void*, unsigned long, rocksdb::EnvOptions const&)::__PRETTY_FUNCTION__> "rocksdb::PosixMmapReadableFile::PosixMmapReadableFile(int, const string&, void*, size_t, const rocksdb::EnvOptions&)") at assert.c:101
      #8  0x00007fc635a82e54 in rocksdb::PosixMmapReadableFile::PosixMmapReadableFile (this=0x7fc65b947700, fd=21, fname="./.rocksdb/IDENTITY", base=0x7fc675f8b000, length=37, options=...) at /home/hrvoje/worktable/PS-5.7.24-26_dbg/storage/rocksdb/rocksdb/env/io_posix.cc:439
      #9  0x00007fc635a769ba in rocksdb::(anonymous namespace)::PosixEnv::NewRandomAccessFile (this=0x7fc6363fb200 <rocksdb::Env::Default()::default_env>, fname="./.rocksdb/IDENTITY", result=0x7fff894b2230, options=...) at /home/hrvoje/worktable/PS-5.7.24-26_dbg/storage/rocksdb/rocksdb/env/env_posix.cc:237
      #10 0x00007fc635950eb0 in rocksdb::DBImpl::Recover (this=0x7fc6718a2a00, column_families=std::vector of length 1, capacity 1 = {...}, read_only=false, error_if_log_file_exist=false, error_if_data_exists_in_logs=false) at /home/hrvoje/worktable/PS-5.7.24-26_dbg/storage/rocksdb/rocksdb/db/db_impl_open.cc:360
      #11 0x00007fc635954e45 in rocksdb::DBImpl::Open (db_options=..., dbname="./.rocksdb", column_families=std::vector of length 1, capacity 1 = {...}, handles=0x7fff894b36b0, dbptr=0x7fff894b3108, seq_per_batch=false) at /home/hrvoje/worktable/PS-5.7.24-26_dbg/storage/rocksdb/rocksdb/db/db_impl_open.cc:1067
      #12 0x00007fc635d268ba in rocksdb::TransactionDB::Open (db_options=..., txn_db_options=..., dbname="./.rocksdb", column_families=std::vector of length 1, capacity 1 = {...}, handles=0x7fff894b36b0, dbptr=0x7fc6363dd6c0 <myrocks::rdb>) at /home/hrvoje/worktable/PS-5.7.24-26_dbg/storage/rocksdb/rocksdb/utilities/transactions/pessimistic_transaction_db.cc:220
      #13 0x00007fc6357823e8 in myrocks::rocksdb_init_func (p=0x7fc66a7f5120) at /home/hrvoje/worktable/PS-5.7.24-26_dbg/storage/rocksdb/ha_rocksdb.cc:4205
      #14 0x0000000000eb1adb in ha_initialize_handlerton (plugin=0x7fc65ba9aa08) at /home/hrvoje/worktable/PS-5.7.24-26_dbg/sql/handler.cc:873
      #15 0x00000000014f3a52 in plugin_initialize (plugin=0x7fc65ba9aa08) at /home/hrvoje/worktable/PS-5.7.24-26_dbg/sql/sql_plugin.cc:1252
      #16 0x00000000014f3e35 in plugin_init_initialize_and_reap () at /home/hrvoje/worktable/PS-5.7.24-26_dbg/sql/sql_plugin.cc:1407
      #17 0x00000000014f4a05 in plugin_register_dynamic_and_init_all (argc=0x2d71890 <remaining_argc>, argv=0x7fc67181c900, flags=0) at /home/hrvoje/worktable/PS-5.7.24-26_dbg/sql/sql_plugin.cc:1692
      #18 0x0000000001608f40 in init_server_components () at /home/hrvoje/worktable/PS-5.7.24-26_dbg/sql/mysqld.cc:4361
      #19 0x000000000160a7a5 in mysqld_main (argc=13, argv=0x7fc67181c900) at /home/hrvoje/worktable/PS-5.7.24-26_dbg/sql/mysqld.cc:5129
      #20 0x0000000000e93bbd in main (argc=13, argv=0x7fff894b4938) at /home/hrvoje/worktable/PS-5.7.24-26_dbg/sql/main.cc:25
      

      Originally reproduced on 5.7.24-26 debug build. I was able to reproduce it on 5.7.23-24 and 5.6.23-25 as well.

      Can't reproduce on the 5.7.23-25 release build, server starts ok and variables are set:

      mysql> show variables like 'rocksdb%reads';
      +--------------------------+-------+
      | Variable_name            | Value |
      +--------------------------+-------+
      | rocksdb_allow_mmap_reads | ON    |
      | rocksdb_use_direct_reads | OFF   |
      +--------------------------+-------+
      2 rows in set (0.01 sec)
      
      mysql> show variables like 'rocksdb%compaction';
      +------------------------------------------------+-------+
      | Variable_name                                  | Value |
      +------------------------------------------------+-------+
      | rocksdb_use_direct_io_for_flush_and_compaction | ON    |
      +------------------------------------------------+-------+
      1 row in set (0.00 sec)
      

        Smart Checklist

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  Unassigned
                  Reporter:
                  hrvoje.matijakovic Hrvoje Matijakovic
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  2 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Time Tracking

                    Estimated:
                    Original Estimate - Not Specified
                    Not Specified
                    Remaining:
                    Remaining Estimate - Not Specified
                    Not Specified
                    Logged:
                    Time Spent - 55 minutes
                    55m