Uploaded image for project: 'Percona Server for MySQL'
  1. Percona Server for MySQL
  2. PS-4439

Port clustering secondary index patch to 8.0

    XMLWordPrintable

    Details

    • Type: Admin & Maintenance Task
    • Status: Done
    • Priority: Medium
    • Resolution: Fixed
    • Affects Version/s: 8.0.x
    • Fix Version/s: 8.0.12-2rc1
    • Component/s: TokuDB
    • Labels:
      None

      Description

      High Level Description

      The main objective is to use global/new Data Dictionary for clustering keys in TokuDB.

       

      High Level Specification

      Background(5.7 scenario)

      In 5.7, we stored the clustering (HA_CLUSTERING_KEY) as combination of HA_FULLTEXT and HA_SPATIAL. It was done to maintain frm format compatibility.

       

      Reading from DD: HA_FULLTEXT & HA_SPATIAL  converted to HA_CLUSTERING_KEY and stored in Index structure (KEY*).

       

      Write to DD: When KEY* is stored as part of frm, HA_CLUSTERING_KEY is not written. Instead HA_FULLTEXT and HA_SPATIAL is written.

      Generally, index types are exclusive. Ie. they can be either PRIMARY, UNIQUE, NON-UNIQUE, SPATIAL, FULLTEXT. There can’t be combinations of the above.

      One additional constraint with clustering key is that, it can be combination of the above. Ie.

      UNIQUE + CLUSTERING KEY and NON-UNIQUE + CLUSTERING KEY

      8.0 Data Dictionary design

      This section doesn’t discuss the parser or other changes done for 8.0. Only the dictionary changes will be discussed.

      An Index is represented by dd::Index with different types

      In-memory definition: dd::Index See sql/dd/types/index.h

      On-disk representation: See sql/dd/impl/tables/indexes.cc

      We store the CLUSTERING type as an option to Index. ie. dd::Index::options. It is stored as bool with key "clustering"

      It is stored on disk as well because dd::Indexes has column FIELD_OPTIONS

        m_target_def.add_field(FIELD_OPTIONS, "FIELD_OPTIONS", "options MEDIUMTEXT");

      From DD to in-memory representation:

      Parse the dd::Index::options.get_bool("clustering"). If it is present, update the in-memory structure (KEY*) with HA_CLUSTERING

      From in-memory representation to DD:
       Persist the HA_CLUSTERING to dd::index as dd::Index::options.set_bool("clustering", true)

      UPGRADE:

      On opening the 5.7 FRM and creating the TABLE*, we adjust the SPATIAL&FULLTEXT flags for tokudb engine to HA_CLUSTERING_KEY. Then,

      As part of upgrade we register dd::Table & dd::Index from TABLE*.  See handler API:

      virtual bool upgrade_table(THD *thd MY_ATTRIBUTE((unused)),

                                  const char *dbname MY_ATTRIBUTE((unused)),

                                  const char *table_name MY_ATTRIBUTE((unused)),

                                  dd::Table *dd_table MY_ATTRIBUTE((unused)))

      In TokuDB, from KEY* flags, map in the below corresponding way

      if (HA_CLUSTERING) is present as part of KEY, set the dd::Index::options("clustering")

      With this mapping, we have successfully migrated the information from 5.7 to 8.0 DD

        Smart Checklist

          Attachments

            Activity

              People

              Assignee:
              satya.bodapati Satya Bodapati
              Reporter:
              george.lorch George Lorch
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 23 minutes
                  23m