LanguagesService: redirect "en" language code to "en_US"

[cbjeukendrup]

If the system language code happens to be something like "en_CY", that will decay to "en". But we don't have that, only "en_US". That causes some problems. To fix that, we map "en" to "en_US".

Might resolve: #13926
May resolve: #12917 (comment)

[cbjeukendrup]

@Eism Could you please try if this solves the issue for you?

There are multiple ways to solve this; I'm now doing it without touching the languages hash, but alternatively we could make sure that we put "en" in the hash instead of "en_US", by making a special case in the languages.json reading code. We can't edit languages.json directly though, because the language codes need to match the codes from Transifex, and there we use "en" as the "source language" and "en_US" as one of the "target languages", so we can't make any changes there.

[Eism]

@cbjeukendrup
There are other problematic languages in the list of available languages, for example zh (there are zh_CN, zh_HK, zh_TW, but no zh).

Maybe instead of

if (m_languagesHash.contains(code)) {
    return code;
}

should be

std::sort(languages.begin(), languages.end(), [](const Language& l, const Language& r) {
        return l.code < r.code;
});

for (const Language& language : languages) {
    if (language.code.startsWith(code)) {
        return language.code;
    }
}

The first found in the list will be selected, for now it is en_GB. (Or does the language have to be US?)

[cbjeukendrup]

Yes, the language should preferably be en_US if no other variant is specified explicitly.

Anyway, I will try to come up with a more general mechanism to work around these problems.

[Eism]

I think we can move it to configuration
Like fallbacksForLanguages

[cbjeukendrup]

I prefer not moving it to configuration, because this relates quite closely to this tryCode algorithm, and actually all logic about which languages we support happens in LanguagesService. I hope this is okay :)

[Eism]

should be merged to RC