English keywords vs english punctuation

[matklad]

One of the guidelines for djot is to aviod english keywords, and instead use punctuation for syntax. I just realized that this is not as i18n friendly as I originally thought -- punctuation also might differ between languages.

In particular, on a Russian keyboard there's no keys for #][}{. It's not a big deal, as users of Cyrillic layouts also use an Enghlish one and can switch between the two, but I didn't realize before that switching is necessary at all.

[matklad]

(nothing actionable here, just a fun observation I wanted to share)

[waldyrious]

Portuguese speakers like me often run across the same issue: keys that in the US keyboard are available as a single keypress require combinations like ~+Space for a non-combining "~", or AltGr+8 for "[", Shift+7 for "/", etc. In the worst case , keys that programs use for interaction are not possible in the Portuguese layout because they expect a single keypress to produce them, and don't respond to combinations.

That said, it is indeed very hard to address this problem unless we severely restrict which symbols can be used to only those available in most keyboard layouts, which is unrealistic. In fact this issue has been bothering me for a while now, so I took this discussion as a prompt to start a table summarizing this information. The spreadsheet is editable btw, so anyone can help fill it up with additional information or comments! :) </shameless plug>

[dumblob]

@waldyrious thank you for your observation! Actually I had the same issue and started the ULKL project years ago.

If you think this could solve your problem, I would very much welcome you to make a PR to that repository with a Portugal layout (even if only for one platform)!

[waldyrious]

Interesting! I'll open an issue in your repo to discuss further.

[bpj]

@matklad do you mean that there is no way at all to access these chars on a Russian keyboard? I think I remember someone saying the same about the backtick. My thought on that isn’t fit for public display! 😁 On my Swedish keyboard # is AltGr+something and () [] {} are respectively Shift/AltGr/Shift+AltGr +8/9 which is annoying enough.

Otherwise it is much as @waldyrious¹ describes for Portuguese: I have all printable ASCII characters and an assortment of Unicode punctuation and symbol characters available somewhere though often in unintuitive places, not all of which are printed on the keys. Å Ä Ö have their own keys and ß Æ Ø Œ have sensible AltGr assignments more or less where you would expect. There are two deadkeys which between them have acute, grave, umlaut, circumflex, tilde in more or less shifted positions. Backtick is Shift+acute+Space, which is yucky when writing software documentation in Markdown, which I do more than I’d like to admit. It’s much worse on Windows though!

On my phone and my tablet I use a third party keyboard with multilingual/multilayout support including custom keyboard creation. 😂 In vim I have my own hack for a pleasant keymap experience.

Sometimes I rave about an LML which uses some Unicode punctuation, but if some people can’t even access all ASCII punctuation it’s definitely dead in the womb! I have my own (undocumented, unpublished) text filter, not a djot or Pandoc filter but a Perl/regex stdin → program → stdout thingy taking config from a YAML file, which replaces text in arbitrary/nested balanced delimiters, using any Unicode punctuation characters + optional djot/Pandoc-like attributes, with arbitrary text which may interpolate the content between the delimiters (recursively expanded) and/or attribute values through a small ‘template’ language.² The ‘template’ language can also can be used to define regexes when the “automatic” simpler regexes are not enough, and then named captures from those regexes can be interpolated into the replacement text too. It’s fun and useful but no substitute for an actual LML. I usually pipe its output into pandoc, so its mostly syntactic sugar for (nested) pandoc spans and divs, letting me use silly things like

• ⌈text⌉ → [text]{.smallcaps}
• ⌊text⌋ → [text]{.underline}
• ⁅:TAG:TEXT:⁆ → [TEXT]{lang=TAG}~TAG~
• ⦑cpan:Module::Name⦒ → [Module::Name](https://metacpan.org/pod/Module::Name)³

Footnotes

1. Whose table I will contribute to as and when I have time 😁 ↩

2. String::Formatter which itself is sprintf on steroids on even more steroids! 😁 ↩

3. I can make data from YAML/JSON files available to templates, so this is generated by a template like [%{text}s](%{url|%{prefix}s}§/%{text}s) My subclass eats the brains of String::Formatter almost completely! 😁 ↩

[matklad]

People with non-Latin based languages generally use two keyboard layouts at the same time, with a key combo (typically win+space) to switch between them. One layout would be en-us QWERTY, and the other with whatever non-Latin keys (ЙЦУКЕН for Russian). On the flip side, we don’t use AltGr combos, as between two layouts, everything has a dedicated key somewhere)

…

On Sunday, 15 January 2023, Benct Philip Jonsson ***@***.***> wrote: @matklad <https://github.com/matklad> do you mean that there is no way at all to access these chars on a Russian keyboard? I think I remember someone saying the same about the backtick. My thought on that isn’t fit for public display! 😁 On my Swedish keyboard # is AltGr+something and () [] {} are respectively Shift/AltGr/Shift+AltGr +8/9 which is annoying enough. Otherwise it is much as @waldyrious <https://github.com/waldyrious>1 <#m_911202567365718665_user-content-fn-1-a188400906d55d0f039d6ff735a97888> describes for Portuguese: I have all printable ASCII characters and an assortment of Unicode punctuation and symbol characters available *somewhere* though often in unintuitive places, not all of which are printed on the keys. Å Ä Ö have their own keys and ß Æ Ø Œ have sensible AltGr assignments more or less where you would expect. There are two deadkeys which between them have acute, grave, umlaut, circumflex, tilde in more or less shifted positions. Backtick is Shift+acute+Space, which is yucky when writing software documentation in Markdown, which I do more than I’d like to admit. It’s much worse on Windows though! On my phone and my tablet I use a third party keyboard <https://play.google.com/store/apps/details?id=kl.ime.oh> with multilingual/multilayout support including custom keyboard creation. 😂 In vim I have my own hack <https://github.com/bpj/bpj-vim-keymaps/blob/master/autoload/switchkeymap.vim> for a pleasant keymap <https://vimhelp.org/mbyte.txt.html#mbyte-keymap> experience. Sometimes I rave about an LML which uses some Unicode punctuation, but if some people can’t even access all ASCII punctuation it’s definitely dead in the womb! I have my own (undocumented, unpublished) text filter, not a djot or Pandoc filter but a Perl/regex stdin → program → stdout thingy taking config from a YAML file, which replaces text in arbitrary/nested balanced delimiters, using any Unicode punctuation characters + optional djot/Pandoc-like attributes, with arbitrary text which may interpolate the content between the delimiters (recursively expanded) and/or attribute values through a small ‘template’ language.2 <#m_911202567365718665_user-content-fn-2-a188400906d55d0f039d6ff735a97888> The ‘template’ language can also can be used to define regexes when the “automatic” simpler regexes are not enough, and then named captures from those regexes can be interpolated into the replacement text too. It’s fun and useful but no substitute for an actual LML. I usually pipe its output into pandoc, so its mostly syntactic sugar for (nested) pandoc spans and divs, letting me use silly things like - ⌈text⌉ → [text]{.smallcaps} - ⌊text⌋ → [text]{.underline} - ⁅:TAG:TEXT:⁆ → [TEXT]{lang=TAG}~TAG~ - ⦑cpan:Module::Name⦒ → [Module::Name](https:// metacpan.org/pod/Module::Name)3 <#m_911202567365718665_user-content-fn-3-a188400906d55d0f039d6ff735a97888> Footnotes 1. Whose table I will contribute to as and when I have time 😁 ↩ <#m_911202567365718665_user-content-fnref-1-a188400906d55d0f039d6ff735a97888> 2. String::Formatter <https://metacpan.org/pod/String::Formatter> which itself is sprintf on steroids on even more steroids! 😁 ↩ <#m_911202567365718665_user-content-fnref-2-a188400906d55d0f039d6ff735a97888> 3. I can make data from YAML/JSON files available to templates, so this is generated by a template like [%{text}s](%{url|%{prefix}s}§/%{text}s) My subclass eats the brains of String::Formatter almost completely! 😁 ↩ <#m_911202567365718665_user-content-fnref-3-a188400906d55d0f039d6ff735a97888> — Reply to this email directly, view it on GitHub <#190 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANB3M2RUTVNQOOWHVHF2MTWSR52DANCNFSM6AAAAAAT3RHJLA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[waldyrious]

That sounds even more inconvenient than AltGr combos and dead keys tbh. So if you're writing a Russian text in Markdown you'd have to change to the English keyboard layout to input the brackets for a link, then switch back to write the label, and so forth?

[matklad]

Yeah, that is correct; luckily, I write English mostly (though I’ve moved to Portugal last year)

[bpj]

@waldyrious Actually it need not be more inconvenient than AltGr combinations. In a vay Shift/AltGr/Shift+AltGr are all different shift states, and they all switch layouts while you depress those shifting keys.. The difference between such a shift layout switch and the layout switch @matklad describes is that with the latter you just hit a key combo and then you are effectively in another shift state without keeping any shifting keys depresses, which actually may be more convenient. On the laptop I actually use sticky keys because I have cerebral palsy and the gymnastics of holding down two keys and hitting a third key is tortous for me. In Vim I use keymaps which, at least the way I use them, is sticky keys on steroids, since I can type a sequence of chars, often without using Shift or AltGr, and get another single char or sequence of chars which otherwise are hard to enter.¹ For example I have an extended Latin keymap which maps most punctuation characters to combining marks so that if I type a/ I get á.² This of course raises a problem if I want to type /, so by (my own) convention typing " as an escape char plus any ASCII char gives that ASCII char, e.g. to get a/ I type a"/. Sometimes it matters which characters I escape: o/ → ø; "o/ → ó; o"/ → o/. Now here is the thing: I tend to (a) forget to type that escape character and (b) to prefer to disable/switch keymaps instead, and I do so without thinking, usually just by hitting F12 twice to switch between the currently active keymap and the previously active one, either of which may be none.

It’s the same with the multi-language/multi-layout keyboard on my handhelds: I switch between languages/layouts without thinking, just as I switch between shift states without thinking, and with sticky keys the difference is negligible even though switching languages/layouts involves a long-press-and-swipe. Thats true even between the English and phonetic alphabet layouts even though they are basically identical, because the latter takes over the correction/prediction bar with suggesting phonetic chars not in the Latin 26 and disables automatic correction, which with my low level of typing dexterity is a bad thing. Sometimes of course I forget to switch, e.g. typing in English or Swedish with the Ancient Greek Vim keymap active, with ἱλαριτυ ensuing, but mostly things go smoothly without me thinking very much about it.

Footnotes

1. As it happens forgetting to turn the OS sticky keys feature, which doesn’t interact well with Vim, off/on as I enter/leave Vim is a major source of irritation. The solution is to stay in Vim as much as possible! 😁 ↩

2. Which is subtly different from á. I have some Perl text filters which I can invoke from Vim which use Unicode::Normalize to convert between composed and decomposed marks. ↩