High Level Description
The main objective is to use global/new Data Dictionary for clustering keys in TokuDB.
High Level Specification
In 5.7, we stored the clustering (HA_CLUSTERING_KEY) as combination of HA_FULLTEXT and HA_SPATIAL. It was done to maintain frm format compatibility.
Reading from DD: HA_FULLTEXT & HA_SPATIAL converted to HA_CLUSTERING_KEY and stored in Index structure (KEY*).
Write to DD: When KEY* is stored as part of frm, HA_CLUSTERING_KEY is not written. Instead HA_FULLTEXT and HA_SPATIAL is written.
Generally, index types are exclusive. Ie. they can be either PRIMARY, UNIQUE, NON-UNIQUE, SPATIAL, FULLTEXT. There can’t be combinations of the above.
One additional constraint with clustering key is that, it can be combination of the above. Ie.
UNIQUE + CLUSTERING KEY and NON-UNIQUE + CLUSTERING KEY
8.0 Data Dictionary design
This section doesn’t discuss the parser or other changes done for 8.0. Only the dictionary changes will be discussed.
An Index is represented by dd::Index with different types
In-memory definition: dd::Index See sql/dd/types/index.h
On-disk representation: See sql/dd/impl/tables/indexes.cc
We store the CLUSTERING type as an option to Index. ie. dd::Index::options. It is stored as bool with key "clustering"
It is stored on disk as well because dd::Indexes has column FIELD_OPTIONS
m_target_def.add_field(FIELD_OPTIONS, "FIELD_OPTIONS", "options MEDIUMTEXT");
From DD to in-memory representation:
Parse the dd::Index::options.get_bool("clustering"). If it is present, update the in-memory structure (KEY*) with HA_CLUSTERING
From in-memory representation to DD:
Persist the HA_CLUSTERING to dd::index as dd::Index::options.set_bool("clustering", true)
On opening the 5.7 FRM and creating the TABLE*, we adjust the SPATIAL&FULLTEXT flags for tokudb engine to HA_CLUSTERING_KEY. Then,
As part of upgrade we register dd::Table & dd::Index from TABLE*. See handler API:
virtual bool upgrade_table(THD *thd MY_ATTRIBUTE((unused)),
const char *dbname MY_ATTRIBUTE((unused)),
const char *table_name MY_ATTRIBUTE((unused)),
dd::Table *dd_table MY_ATTRIBUTE((unused)))
In TokuDB, from KEY* flags, map in the below corresponding way
if (HA_CLUSTERING) is present as part of KEY, set the dd::Index::options("clustering")
With this mapping, we have successfully migrated the information from 5.7 to 8.0 DD