Uploaded image for project: 'Percona Server for MySQL'
  1. Percona Server for MySQL
  2. PS-7169

Set rocksdb_validate_tables to disabled RocksDB while upgrading the server from 5.7 to 8.0.20

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Priority: High
    • Resolution: Fixed
    • Affects Version/s: 8.0.20-11
    • Fix Version/s: 8.0.21-12
    • Component/s: None
    • Labels:
      None

      Description

      RocksDB initialization is failing while upgrading the server from 5.7 to 8.0.20

      Note: 5.7 > 8.0.19 upgrade is working fine.

      Testcase

      1) Start PS-5.7.30 and enable RocksDB plugin.
      rm -Rf /qa/ps/data
      /qa/ps/ps5730/bin/mysqld --defaults-file=/qa/ps/57.cnf --initialize-insecure
      
      /qa/ps/ps5730/bin/mysqld --defaults-file=/qa/ps/57.cnf & 
      cnf
      ---
      
      [mysqld]
      innodb_parallel_doublewrite_path=/qa/ps/double_write
      innodb_buffer_pool_instances=8
      innodb_buffer_pool_size=4G
      log_error_verbosity=3
      gtid_mode=ON
      enforce_gtid_consistency=ON
      log_slave_updates=ON
      log_bin=binlog
      binlog_format=ROW
      master_info_repository=TABLE
      relay_log_info_repository=TABLE
      early-plugin-load=keyring_file.so
      keyring_file_data=keyring
      encrypt_tmp_files = ON
      innodb_undo_log_encrypt = ON
      innodb_undo_tablespaces=5
      innodb_temp_tablespace_encrypt = ON
      innodb-sys-tablespace-encrypt = ON
      innodb_parallel_dblwr_encrypt = ON
      core-file
      basedir=/qa/ps/ps5730
      tmpdir=/qa/ps/data
      datadir=/qa/ps/data
      socket=/qa/ps/data/socket.sock
      port=13612
      log-error=/qa/ps/data/master.err
      server-id=100
      plugin-load-add=tokudb=ha_tokudb.so
      tokudb-check-jemalloc=0
      plugin-load-add=rocksdb=ha_rocksdb.so
      
      
      2) Create RocksDB tables
      
      mysql -uroot -S=/qa/ps/data/socket.sock -e"CREATE USER IF NOT EXISTS sysbench_user@'%' identified with mysql_native_password by 'test';GRANT ALL ON *.* TO sysbench_user@'%'";
      
      sysbench /usr/share/sysbench/oltp_insert.lua  --mysql-storage-engine=rocksdb --table-size=10000 --tables=10 --threads=10 --mysql-db=rocksdb_test --mysql-user=sysbench_user --mysql-password=test  --db-driver=mysql --mysql-socket=/qa/ps/data/socket.sock prepare
      
      3) Stop PS-5.7.30 and start the server with PS-8.0.20 binaries 
      
      /qa/ps/ps8020/bin/mysqld --defaults-file=/qa/ps/higher.cnf &
      
      cnf
      ---
      [mysqld]
      innodb_doublewrite=1
      innodb_doublewrite_dir="/qa/ps/nodir"
      innodb_doublewrite_batch_size=1024
      innodb_doublewrite_pages=32
      innodb_buffer_pool_instances=8
      innodb_buffer_pool_size=4G
      log_error_verbosity=3
      gtid_mode=ON
      enforce_gtid_consistency=ON
      log_slave_updates=ON
      log_bin=binlog
      binlog_format=ROW
      master_info_repository=TABLE
      relay_log_info_repository=TABLE
      early-plugin-load=keyring_file.so
      keyring_file_data=keyring
      encrypt_tmp_files = ON
      binlog_encryption = ON
      default_table_encryption = ON
      innodb_redo_log_encrypt = ON
      innodb_undo_log_encrypt = ON
      innodb_undo_tablespaces=5
      innodb_temp_tablespace_encrypt = ON
      innodb-sys-tablespace-encrypt = ON
      innodb_parallel_dblwr_encrypt = ON
      core-file
      basedir=/qa/ps/ps8020
      tmpdir=/qa/ps/data
      datadir=/qa/ps/data
      socket=/qa/ps/data/socket.sock
      port=13612
      log-error=/qa/ps/data/master.err
      server-id=100
      plugin-load-add=tokudb=ha_tokudb.so
      tokudb-check-jemalloc=0
      plugin-load-add=rocksdb=ha_rocksdb.so
      
      

      Error info

      2020-07-13T10:24:09.649653Z 0 [Note] [MY-000000] [Server] Plugin rocksdb reported: '2 column families found'
      2020-07-13T10:24:09.650469Z 0 [Note] [MY-000000] [Server] Plugin rocksdb reported: 'Column Families at start:'
      2020-07-13T10:24:09.651072Z 0 [Note] [MY-000000] [Server] Plugin rocksdb reported: '  cf=default'
      2020-07-13T10:24:09.651535Z 0 [Note] [MY-000000] [Server] Plugin rocksdb reported: '    write_buffer_size=67108864'
      2020-07-13T10:24:09.652023Z 0 [Note] [MY-000000] [Server] Plugin rocksdb reported: '    target_file_size_base=67108864'
      2020-07-13T10:24:09.652663Z 0 [Note] [MY-000000] [Server] Plugin rocksdb reported: '  cf=__system__'
      2020-07-13T10:24:09.653143Z 0 [Note] [MY-000000] [Server] Plugin rocksdb reported: '    write_buffer_size=67108864'
      2020-07-13T10:24:09.653676Z 0 [Note] [MY-000000] [Server] Plugin rocksdb reported: '    target_file_size_base=67108864'
      2020-07-13T10:24:09.655924Z 0 [Note] [MY-000000] [Server] Plugin rocksdb reported: 'Opening TransactionDB...'
      2020-07-13T10:24:09.831420Z 0 [Note] [MY-000000] [Server] Plugin rocksdb reported: 'Initializing data dictionary...'
      2020-07-13T10:24:09.929767Z 0 [Note] [MY-000000] [Server] Plugin rocksdb reported: 'Initializing DDL Manager...'
      2020-07-13T10:24:09.941145Z 0 [Warning] [MY-000000] [Server] Plugin rocksdb reported: 'Schema mismatch - Table rocksdb_test.sbtest1 is registered in RocksDB but does not have a corresponding DD table'
      2020-07-13T10:24:09.941708Z 0 [Warning] [MY-000000] [Server] Plugin rocksdb reported: 'Schema mismatch - Table rocksdb_test.sbtest10 is registered in RocksDB but does not have a corresponding DD table'
      2020-07-13T10:24:09.942231Z 0 [Warning] [MY-000000] [Server] Plugin rocksdb reported: 'Schema mismatch - Table rocksdb_test.sbtest2 is registered in RocksDB but does not have a corresponding DD table'
      2020-07-13T10:24:09.942756Z 0 [Warning] [MY-000000] [Server] Plugin rocksdb reported: 'Schema mismatch - Table rocksdb_test.sbtest3 is registered in RocksDB but does not have a corresponding DD table'
      2020-07-13T10:24:09.943319Z 0 [Warning] [MY-000000] [Server] Plugin rocksdb reported: 'Schema mismatch - Table rocksdb_test.sbtest4 is registered in RocksDB but does not have a corresponding DD table'
      2020-07-13T10:24:09.943847Z 0 [Warning] [MY-000000] [Server] Plugin rocksdb reported: 'Schema mismatch - Table rocksdb_test.sbtest5 is registered in RocksDB but does not have a corresponding DD table'
      2020-07-13T10:24:09.944373Z 0 [Warning] [MY-000000] [Server] Plugin rocksdb reported: 'Schema mismatch - Table rocksdb_test.sbtest6 is registered in RocksDB but does not have a corresponding DD table'
      2020-07-13T10:24:09.944912Z 0 [Warning] [MY-000000] [Server] Plugin rocksdb reported: 'Schema mismatch - Table rocksdb_test.sbtest7 is registered in RocksDB but does not have a corresponding DD table'
      2020-07-13T10:24:09.945469Z 0 [Warning] [MY-000000] [Server] Plugin rocksdb reported: 'Schema mismatch - Table rocksdb_test.sbtest8 is registered in RocksDB but does not have a corresponding DD table'
      2020-07-13T10:24:09.946003Z 0 [Warning] [MY-000000] [Server] Plugin rocksdb reported: 'Schema mismatch - Table rocksdb_test.sbtest9 is registered in RocksDB but does not have a corresponding DD table'
      2020-07-13T10:24:09.946564Z 0 [ERROR] [MY-000000] [Server] Plugin rocksdb reported: 'RocksDB: Problems validating data dictionary against DD tables, exiting'
      2020-07-13T10:24:09.947154Z 0 [ERROR] [MY-000000] [Server] Plugin rocksdb reported: 'Failed to initialize DDL manager.'
      2020-07-13T10:24:09.947740Z 0 [ERROR] [MY-010202] [Server] Plugin 'ROCKSDB' init function returned error.
      2020-07-13T10:24:09.948305Z 0 [ERROR] [MY-010734] [Server] Plugin 'ROCKSDB' registration as a STORAGE ENGINE failed.
      2020-07-13T10:24:15.236343Z 2 [ERROR] [MY-013140] [Server] Unknown storage engine 'ROCKSDB'
      2020-07-13T10:24:15.236984Z 2 [ERROR] [MY-013140] [Server] Can't find file: './rocksdb_test/sbtest9.frm' (errno: 0 - Success)
      2020-07-13T10:24:15.237580Z 2 [ERROR] [MY-013137] [Server] Can't find file: './rocksdb_test/sbtest9.frm' (OS errno: 0 - Success)
      2020-07-13T10:24:15.238176Z 2 [ERROR] [MY-010348] [Server] Error in reading file ./rocksdb_test/sbtest9.frm
      

       

        Attachments

          Activity

            People

            Assignee:
            przemyslaw.skibinski Przemyslaw Skibinski
            Reporter:
            ramesh.sivaraman Ramesh Sivaraman
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - Not Specified
                Not Specified
                Logged:
                Time Spent - 4 hours
                4h

                  Smart Checklist