Languages Page

[PythonSemicolon]

Creating this thread to contain all language-related discussion, content & resources as we plan VIM in its early stages.

Useful links:

• @maregold's list of languages
• names of several Western orchestral instruments in six European languages (good starting point)
• wikipedia's list of national instruments (most are listed in their own language)
• MIMO's instrument families (12 languages can be toggled in top nav bar)

[PythonSemicolon]

To summarize thoughts from last week's meeting as well as ideas discussed by @chetbae, @lucasmarchd01, @dchiller and others:

1. We should have a /languages/ url path with a separate page for each stored language (i.e., /languages/svenska)
2. Each language page should have 3 sections:

• instruments in VIM that do not currently have a name in the given language (unnamed)
• instruments in VIM which have a name in the given language which has not yet been verified (unverified)
• instruments in VIM which have a verified name in the given language (named) - this should be at the bottom since the focus should be on the other two sections; however, users should still be able to add alternate names even if verified ones exist

3. When hovering over an image of an unnamed instrument, we could display its name in another language (site language or other) to assist users in identifying it

[PythonSemicolon]

Other things to consider:

• some instruments with different names in English may be called the same name in another language. How will we store/distinguish these in the DB and when displaying?
• when storing language names in the DB, we must choose to either store the names in English or in the native language (this will likely cause problems for URLs when it comes to different alphabets). Whatever we choose, nomenclature must be consistent.
• users should be able to add new languages in two places if they cannot find their language on the site: either when adding a new instrument, or when adding a new language for an existing instrument. For the latter, this means we need a link/function on an instrument's page or perhaps a query page todo this, since the /languages/ path will be nonexistent.

[fujinaga]

In Wikidata, there are 10,285 natural languages (Q34770).
https://query.wikidata.org

SELECT (COUNT(?language) AS ?count)
WHERE {
?language wdt:P31 wd:Q34770.
}

If you want to list first 100:

SELECT ?languageLabel
WHERE {
?language wdt:P31 wd:Q34770.
SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
}
LIMIT 100

[fujinaga]

when storing language names in the DB, we must choose to either store the names in English or in the native language (this will likely cause problems for URLs when it comes to different alphabets). Whatever we choose, nomenclature must be consistent.

We'll store the language name using wikidata URI, e.g.:
https://www.wikidata.org/wiki/Q1860
https://www.wikidata.org/wiki/Q5287

So no need to save the names in any language.

[ahankinson]

Also:

• Languages and Scripts are two different things. Some languages can be written in different scripts, e.g., Arabic and Romanized Arabic; Multiple forms of romanized Chinese.
• There are different kinds of a single language, e.g., Brazillian Portugese vs. Portugal Portugese; British English vs. American English. So you will probably need a regional differentiator on the language code itself. See also Macrolanguages. (This may be different from the location where the instrument is called something...)
• BCP47 and specifically RFC5646 are methods of combining region, language, and script into a single identifier.
• Pidgins and Creoles are a thing, e.g., https://en.wikipedia.org/wiki/Inuktitut-English_Pidgin.
• You will need to consider the font you use, since many fonts do not have characters for all languages. This is particularly true for Canadian indigenous languages. You can always store the underlying character bytes, but you can't always display them!
• You need to consider Unicode normalization in your search design.
• Depending on the input device, there can be two (or more) ways of writing the same character in Unicode. (I have personal experience with how frustrating this can be). I would recommend watching this talk, "Plain Text" by Dylan Beattie.
• You need to be very careful with your database character encoding design at the start. Postgresql only supports UTF-8 which should be fine, but you need to be aware of this when you set up your database.
• A big problem (for us in RISM) is copy-paste errors, where someone will have a Word document that contains character encoding in a non-UTF format, and will copy and paste this into the text fields. Without special handling or verifying that it gets correctly converted, this will get stored as the underlying byte sequence rather than the expected UTF-8 variant.
• There should be an awareness that languages and scripts are political, and instrument names may be tied to colonial or gendered histories.
  • https://www.jstor.org/stable/3557488
  • https://www.jstor.org/stable/852676

[fujinaga]

You will need to consider the font you use, since many fonts do not have characters for all languages. This is particularly true for Canadian indigenous languages. You can always store the underlying character bytes, but you can't always display them!

Good point. How about we use the Noto Sans on our web pages?
https://www.dezeen.com/2016/10/06/google-monotype-noto-font-biggest-typeface-all-written-language/

[ahankinson]

Noto is great, and highly recommended. However, it's useful to know that not all the glyphs are in a single font file. If you go to Google Fonts and search for "Noto" you'll see there are a lot of other "Noto" fonts for specific scripts.

[fujinaga]

I believe Noto Sans includes all its subsets, such as Noto Sans Japanese. On Aug 14, 2023, at 04:40, Andrew Hankinson ***@***.***> wrote:  Noto is great, and highly recommended. However, not all the glyphs are in a single font. If you go to Google Fonts and search for "Noto" you'll see there are a lot of other "Noto" fonts for specific scripts<https://fonts.google.com/noto/fonts?query=Noto>. — Reply to this email directly, view it on GitHub<#20 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAH342MUUVYX5VITSP5XWWLXVHQAHANCNFSM6AAAAAA3NHUNZ4>. You are receiving this because you commented.Message ID: ***@***.***>

[ahankinson]

The "regular" Noto Sans includes support for Latin, Greek, and Cyrillic scripts. If you want to support other scripts outside of this you need to pull in additional font files which include those extra characters. (I've had to do this for Armenian, Georgian, Arabic, Korean, Hebrew, and Sanskrit before).