Add support for MySQL 8.0

[jeremycole]

Add support for MySQL 8.0. There are a number of changes needed to support MySQL 8.0, and some new data structures to implement.

• Support new headers in FSP pages
• Support for reconstructing full BLOB content
• Support new page types:
  • UNKNOWN (13)
  • COMPRESSED (14)
  • ENCRYPTED (15)
  • COMPRESSED_AND_ENCRYPTED (16)
  • ENCRYPTED_RTREE (17)
  • SDI_BLOB (18) (aka BLOB)
  • SDI_ZBLOB (19) (aka ZBLOB)
  • LEGACY_DBLWR (20)
  • RSEG_ARRAY (21)
  • LOB_INDEX (22)
  • LOB_DATA (23)
  • LOB_FIRST (24)
  • ZLOB_FIRST (25)
  • ZLOB_DATA (26)
  • ZLOB_INDEX (27)
  • ZLOB_FRAG (28)
  • ZLOB_FRAG_ENTRY (29)
  • SDI (17853) (aka INDEX)
  • RTREE (17854)
• Support for new SDI data dictionary structures:
  • Using SDI for data dictionary to make record describers
  • Using SDI for overview information (e.g. system-spaces)
• Support for LOB
• Support for Compressed LOB (ZLOB)
• Support for online-added columns?
• ???

Sub-issue list:

• Add support for Serialized Dictionary Information (SDI) in MySQL 8.0 #185
• Add support for new LOB storage in MySQL 8.0 #186
• Add support for new ZLOB storage in MySQL 8.0 #187

PR list:

• Refactor hex output of BLOB data #184
• Add new page types for MySQL 8.0, remove :usage key in structure #188
• Implement reading new encryption and SDI headers in FSP page #189
• Update spec/data/sakila/8.0 with MySQL 8.0.33 #190
• Add support for reconstructing full BLOB content #191

See also: #41, #44.

[utdrmac]

Related?

# innodb_space -s ibdata1 system-spaces
name                            pages       indexes
Traceback (most recent call last):
	16: from /usr/local/bin/innodb_space:23:in `<main>'
	15: from /usr/local/bin/innodb_space:23:in `load'
	14: from /usr/local/share/gems/gems/innodb_ruby-0.12.0/bin/innodb_space:1714:in `<top (required)>'
	13: from /usr/local/share/gems/gems/innodb_ruby-0.12.0/bin/innodb_space:192:in `system_spaces'
	12: from /usr/local/share/gems/gems/innodb_ruby-0.12.0/bin/innodb_space:188:in `block in system_spaces'
	11: from /usr/local/share/gems/gems/innodb_ruby-0.12.0/bin/innodb_space:188:in `to_a'
	10: from /usr/local/share/gems/gems/innodb_ruby-0.12.0/bin/innodb_space:188:in `each'
	 9: from /usr/local/share/gems/gems/innodb_ruby-0.12.0/lib/innodb/space.rb:341:in `each_index'
	 8: from /usr/local/share/gems/gems/innodb_ruby-0.12.0/lib/innodb/space.rb:321:in `each_index_root_page_number'
	 7: from /usr/local/share/gems/gems/innodb_ruby-0.12.0/lib/innodb/data_dictionary.rb:425:in `each_index_by_space_id'
	 6: from /usr/local/share/gems/gems/innodb_ruby-0.12.0/lib/innodb/data_dictionary.rb:355:in `each_index'
	 5: from /usr/local/share/gems/gems/innodb_ruby-0.12.0/lib/innodb/data_dictionary.rb:324:in `each_record_from_data_dictionary_index'
	 4: from /usr/local/share/gems/gems/innodb_ruby-0.12.0/lib/innodb/data_dictionary.rb:287:in `data_dictionary_index'
	 3: from /usr/local/share/gems/gems/innodb_ruby-0.12.0/lib/innodb/space.rb:312:in `index'
	 2: from /usr/local/share/gems/gems/innodb_ruby-0.12.0/lib/innodb/space.rb:312:in `new'
	 1: from /usr/local/share/gems/gems/innodb_ruby-0.12.0/lib/innodb/index.rb:19:in `initialize'
/usr/local/share/gems/gems/innodb_ruby-0.12.0/lib/innodb/index.rb:34:in `page': undefined method `record_describer=' for #<Innodb::Page::FspHdrXdes:0x000055a140fed580> (NoMethodError)

8.0.28

[lhsaq2009]

[root@MySQL80 mysql]# innodb_space -s ibdata1 system-spaces
name                            pages       indexes
Traceback (most recent call last):
	16: from /root/.gem/ruby/2.6.0/bin/innodb_space:23:in `<main>'
	15: from /root/.gem/ruby/2.6.0/bin/innodb_space:23:in `load'
	14: from /root/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/bin/innodb_space:1714:in `<top (required)>'
	13: from /root/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/bin/innodb_space:192:in `system_spaces'
	12: from /root/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/bin/innodb_space:188:in `block in system_spaces'
	11: from /root/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/bin/innodb_space:188:in `to_a'
	10: from /root/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/bin/innodb_space:188:in `each'
	 9: from /root/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/space.rb:341:in `each_index'
	 8: from /root/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/space.rb:321:in `each_index_root_page_number'
	 7: from /root/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/data_dictionary.rb:425:in `each_index_by_space_id'
	 6: from /root/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/data_dictionary.rb:355:in `each_index'
	 5: from /root/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/data_dictionary.rb:324:in `each_record_from_data_dictionary_index'
	 4: from /root/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/data_dictionary.rb:287:in `data_dictionary_index'
	 3: from /root/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/space.rb:312:in `index'
	 2: from /root/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/space.rb:312:in `new'
	 1: from /root/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/index.rb:19:in `initialize'
/root/.gem/ruby/2.6.0/gems/innodb_ruby-0.12.0/lib/innodb/index.rb:34:in `page': undefined method `record_describer=' for #<Innodb::Page::FspHdrXdes:0x000055b0ae47b968> (NoMethodError)
[root@MySQL80 mysql]#

8.0.18

[jeremycole]

I filed MySQL Bug 111079: InnoDB SDI index description has fields out-of-order as implementing SDI caused me to find a subtle issue with the fields being out of order internally.

[jeremycole]

I filed MySQL Bug 111112: JSON SDI default_value in columns object includes trailing zero bytes as implementing SDI caused me to notice that the included default_value field includes trailing zero bytes (seems to be encoding a raw buffer from C++).