MariaDB vs MySQL adaptations

[ryanrath]

Description

• Due to the inability of MySQL to handle UTF-8 -> latin1 coercion ( which
  mariadb apparently has no problem with ). We cast all values that are provided
  as strings to 'BINARY' during comparison with column values that are stored as
  latin1 ( Which are also cast to BINARY ). This was done for the following files:
  • acl-config
  • create_public_user.sql

Motivation and Context

When executing these scripts against a MySQL box the queries involved errored out against a MySQL db but not against a MariaDB instance.

Tests performed

Manual Testing:

• select a query that is currently failing due to illegal mix of collations error.
• execute: SELECT COLLATION('dfsdf');
  • this will tell you what collation is being used for string literals for your db.
• execute: SELECT table_schema, table_name, column_name, character_set_name, collation_name FROM information_schema.columns WHERE table_schema IN ('<your_schema_here>') AND table_name IN ('<your table_name_here>') ORDER BY table_schema, table_name,ordinal_position;
  - This will tell what encoding / collation is being used for the columns in table_schema.table_name
  • If the collation for the string text is different than that of the columns being compared to then you will need to do some casting wherever the selected query does anything like:
    • table.column = 'textual data' and changing it to
    • BINARY table.column = BINARY 'textual data'

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• My code follows the code style of this project as found in the CONTRIBUTING document.
• I have added tests to cover my changes.
• All new and existing tests passed.

[tyearke]

Discussing it with @ryanrath, this seems like the least risky way of solving the problem. It works so long as:

1. The character set of one string is a subset of the character set of the other string, and
2. Both character sets use the same binary representations for shared characters.

Since I can only ever foresee us using latin1, utf8, or utf8mb4, this should continue to work even if we change character set and collation settings in the future.

EDIT: Only the first 128 characters of latin1 - the ASCII characters - are binary compatible with utf8 and utf8mb4. As long as we stick to those characters for this purpose before we switch to UTF-8, we're fine.

[smgallo]

I agree, any change should be to utf8mb4. We have encountered similar issues ingesting data from Postgres which supports 4-byte utf8 characters by default and ended up converting to latin1 as a temporary solution.