diff --git a/asai/Asai/Logger/Make/index.html b/asai/Asai/Logger/Make/index.html index 8c68bf2..f68e90a 100644 --- a/asai/Asai/Logger/Make/index.html +++ b/asai/Asai/Logger/Make/index.html @@ -68,7 +68,7 @@ (unit -> 'a) -> 'a) -> (unit -> 'a) -> - 'a

adopt m run f runs the thunk f that uses a different Logger instance, with the help of the runner run from that Logger instance, and then uses m to map the diagnostics generated by f into the ones in the current Logger instance. The backtrace within f will include the backtrace that leads to adopt. The intended use case is to integrate diagnostics from a library into those in the main application.

adopt is a convenience function that can be implemented as follows:

let adopt m f run =
+  'a

adopt m run f runs the thunk f that uses a different Logger instance. It takes the runner run from that Logger instance as an argument to handle effects, and will use m to transform diagnostics generated by f into ones in the current Logger instance. The backtrace within f will include the backtrace that leads to adopt, and the innermost specified location will be carried over, too. The intended use case is to integrate diagnostics from a library into those in the main application.

adopt is a convenience function that can be implemented as follows:

let adopt m f run =
   run
     ?init_loc:(get_loc())
     ?init_backtrace:(Some (get_backtrace()))
@@ -77,7 +77,7 @@
     f

Here shows the intended usage, where Lib is the library to be used in the main application:

module MainLogger = Logger.Make(Code)
 module LibLogger = Lib.Logger
 
-let _ = MainLogger.adopt (Diagnostic.map code_mapper) LibLogger.run @@ fun () -> ...
val try_with : +let _ = MainLogger.adopt (Diagnostic.map code_mapper) LibLogger.run @@ fun () -> ...
val try_with : ?emit:(Code.t Diagnostic.t -> unit) -> ?fatal:(Code.t Diagnostic.t -> 'a) -> (unit -> 'a) -> diff --git a/asai/Asai/Logger/module-type-S/index.html b/asai/Asai/Logger/module-type-S/index.html index c0eeb7c..1fb7f06 100644 --- a/asai/Asai/Logger/module-type-S/index.html +++ b/asai/Asai/Logger/module-type-S/index.html @@ -68,7 +68,7 @@ (unit -> 'a) -> 'a) -> (unit -> 'a) -> - 'a

adopt m run f runs the thunk f that uses a different Logger instance, with the help of the runner run from that Logger instance, and then uses m to map the diagnostics generated by f into the ones in the current Logger instance. The backtrace within f will include the backtrace that leads to adopt. The intended use case is to integrate diagnostics from a library into those in the main application.

adopt is a convenience function that can be implemented as follows:

let adopt m f run =
+  'a

adopt m run f runs the thunk f that uses a different Logger instance. It takes the runner run from that Logger instance as an argument to handle effects, and will use m to transform diagnostics generated by f into ones in the current Logger instance. The backtrace within f will include the backtrace that leads to adopt, and the innermost specified location will be carried over, too. The intended use case is to integrate diagnostics from a library into those in the main application.

adopt is a convenience function that can be implemented as follows:

let adopt m f run =
   run
     ?init_loc:(get_loc())
     ?init_backtrace:(Some (get_backtrace()))
@@ -77,7 +77,7 @@
     f

Here shows the intended usage, where Lib is the library to be used in the main application:

module MainLogger = Logger.Make(Code)
 module LibLogger = Lib.Logger
 
-let _ = MainLogger.adopt (Diagnostic.map code_mapper) LibLogger.run @@ fun () -> ...
val try_with : +let _ = MainLogger.adopt (Diagnostic.map code_mapper) LibLogger.run @@ fun () -> ...
val try_with : ?emit:(Code.t Diagnostic.t -> unit) -> ?fatal:(Code.t Diagnostic.t -> 'a) -> (unit -> 'a) -> diff --git a/asai/design.html b/asai/design.html index c7fec2a..ec2f7f3 100644 --- a/asai/design.html +++ b/asai/design.html @@ -1,2 +1,2 @@ -design (asai.design)

Design Principles

Four Factors of a Diagnostic

In addition to the main message, the API should allow implementers to easily specify the following four factors, and they are somewhat independent.

  1. Whether the program terminate now. This is done by the choice between emit for non-fatal messages and fatal for fatal ones.
  2. How the user should classify the message. See Asai.Diagnostic.severity.
  3. A succinct Google-able message code. While severity should be changeable independently of the message code, often the same code implies the same severity. That is why we have Asai.Diagnostic.Code.default_severity to specify the default severity for each code.
  4. The backtrace and locations of other related text. See Asai.Logger.S.tracef and Asai.Diagnostic.t.additional_messages.

Stable Unicode Art Must Avoid Column Numbers

There is a long history of using ASCII printable characters and ANSI escape sequences, and recently also non-ASCII Unicode characters, to draw pictures on terminals. To display compiler diagnostics, this technique has been used to assemble line numbers, code from end users, code highlighting, and other pieces of information in a visually pleasing way. Non-ASCII Unicode characters (from implementers or from end users) greatly expand the vocabulary of ASCII art, and we will call the new art form Unicode art to signify the use of non-ASCII characters. However, these Unicode characters also impose new challenges as their visual widths are unpredictable without knowing the exact terminal (emulator), the exact font, etc. Unicode emoji sequences might be one of the most challenging cases: a pirate flag (🏴‍☠️) may be shown as a single flag on supported platforms but as a sequence with a black flag (🏴) and a skull (☠️) on other platforms. This means the visual width of the pirate flag is unpredictable. (See UTS #51 Section 2.2.) The rainbow flag (🏳️‍🌈), skin tones, and many other emoji sequences have the same issue. Other less chaotic but still challenging cases include characters whose East Asian width is Ambiguous.

It is thus wise for implementers to think twice before using emoji sequences and other tricky characters in Unicode art. To quantify the degree to which a Unicode art can remain visually pleasing on different platforms, we specify the following four levels of stability. Note that if implementers decide to integrate content from end users into their Unicode art, the end users should have the freedom to include arbitrary emoji sequences and tricky characters in their content, and the final Unicode art must remain visually pleasing as defined by the stability levels.

  • Level 0 (the least stable): Stability under the assumption that every character occupies exactly the same visual width. Thanks to the popularity of Unicode, programs of this level are mostly considered outdated.
  • Level 1: Stability under the assumption each Unicode string visually occupies a multiple of some fixed width, where the multiplier is determined by heuristics (such as various implementations of wcwidth and wcswidth). These heuristics are created to help programmers handle more characters, in particular CJK characters, without dramatically changing the code. They however do not solve the core problem (that is, visual width is fundamentally ill-defined) and they often could not handle tricky cases such as emoji sequences. Many compilers are at this level.
  • Level 2a: Stability under very limited assumptions on which characters should have the same widths. For example, if a Unicode art only assumes Unicode box-drawing characters are of the same visual width (which is the case in all conceivable situations), then its stability is at this level. However, the phrase "very limited" is somewhat subjective, and thus we present a more precise version below.
  • Level 3 (the most stable): Stability under only one assumption that the same grapheme clusters will have the same width regardless of the context. This means that the Unicode art will remain visually pleasing in almost all situations. It can even be rendered with a variable-width font.

Unlike most implementations, which are at Level 1, our terminal backend strives to achieve Level 2. That means we must not make any assumption about the visual width of end users' code and must abandon the idea of column numbers. As a result, our terminal backend never shows column numbers and we consider that as a significant improvement. We believe Level 3 is too restricted for compiler diagnostics because we cannot show line numbers along with the end users' code. (We cannot assume the numbers "10" and "99" will have the same visual width at Level 3.)

Note: a fixed-width Unicode font is often technically duospaced, not monospaced, because many CJK characters would occupy a double character width. Thus, we do not use the terminology "monospaced".

Raw Bytes as Positions

All positions are byte-oriented. Here are some popular alternatives which we think are worse:

  1. Unicode characters (which may not match user-perceived characters).
  2. Unicode grapheme clusters or user-perceived characters. See the uuseg library.
  3. Column numbers, the visual width of a string in display.

It takes at least linear time to count Unicode characters (except when UTF-32 is in use) or Unicode grapheme clusters from raw bytes. Column numbers are even worse because they are not well-defined, as elaborated in the previous section. The only well-defined unit that also admits an efficient implementation is raw byte.

Note: Our LSP prototype does not handle positionEncoding yet, and thus an LSP client may be confused about the ranges returned by this library. A proper LSP implementation should negotiate with the client to determine how to represent column positions (and our current prototype does not). On the other hand, it can be tricky to negotiate with the client to use raw bytes because there is not an official predefined encoding scheme for raw bytes yet.

\ No newline at end of file +design (asai.design)

Design Principles

Five Independent Parameters of a Diagnostic

In addition to the main message, the API should allow implementers to easily specify the following five factors of a diagnostic, and it should be possible to specify them independently.

  1. Whether the program terminates after sending the message. This is indicated by the choice between emit (for non-fatal messages) and fatal (for fatal ones).
  2. A message code with a succinct Google-able representation, for example V0003. A succinct representation is useful for an end user to report a bug or ask for help.
  3. How seriously end users should take the message. Is it a warning, an error, or just a hint? See the type severity for available classifications. In practice, messages with the same message code tend to have the same severity, and thus our API requires an implementer to specify a default severity for each message code. While this seems to violate the independence constraint, our API allows overriding the default severity at each call of emit or fatal.
  4. A stack backtrace. There should be a straightforward way to push new stack frames. Our implementation is trace.
  5. Additional messages. It should be possible to attach any numbers of additional related messages. Currently, emit and fatal are taking .

Compositionality: Using Libraries that Use asai

It should be easy for an application to use other libraries who themselves use asai. Our current implementation allows an application to adopt messages from a library.

Stability of Unicode Art: No Column Numbers!

There is a long history of using ASCII printable characters and ANSI escape sequences, and recently also non-ASCII Unicode characters, to draw pictures on terminals. To display compiler diagnostics, this technique has been used to assemble line numbers, code from end users, code highlighting, and other pieces of information in a visually pleasing way. Non-ASCII Unicode characters (from implementers or from end users) greatly expand the vocabulary of ASCII art, and we will call the new art form Unicode art to signify the use of non-ASCII characters.

These non-ASCII Unicode characters impose new challenges as their visual widths are unpredictable without knowing the exact terminal (or terminal emulator), the exact font, etc. Unicode emoji sequences might be one of the most challenging cases: a pirate flag (🏴‍☠️) may be shown as a single emoji flag on supported platforms but as a sequence with a black flag (🏴) and a skull (☠️) on other platforms. This means the visual width of the pirate flag is unpredictable. (See UTS #51 Section 2.2.) The rainbow flag (🏳️‍🌈), skin tones, and many other emoji sequences have the same issue. Other less chaotic but still challenging cases include characters whose East Asian width is Ambiguous. These challenges bear some similarity with the unpredictability of the visual width of a horizontal tab, but in a much wilder way.

Note: "Unicode characters" are not really defined in the Unicode standard, and here they mean Unicode scalar values, that is, all Unicode code points except the surrogate code points for UTF-16 to represent all scalar values. Although the word "character" has many incompatible meanings and usages, we decided to call scalar values "Unicode characters" anyway because (1) most people are not familiar with the official term "scalar values" and (2) scalar values are the only stable primitive unit one can work with in a programming language.

It is thus wise to think twice before using emoji sequences and other tricky characters in Unicode art. To quantify the degree to which a Unicode art can remain visually pleasing on different platforms, we specify the following four levels of stability. Note that if implementers decide to integrate content from end users into their Unicode art, the end users should have the freedom to include arbitrary emoji sequences and tricky characters in their content. The final Unicode art must remain visually pleasing as defined by the stability levels for any reasonable user content.

  • Level 0 (the least stable): Stability under the assumption that every Unicode character occupies exactly the same visual width. Thankfully, programs meeting only this level are mostly considered outdated.
  • Level 1: Stability under the assumption each Unicode string visually occupies a multiple of some fixed width, where the multiplier is determined by heuristics (such as various implementations of wcwidth and wcswidth). These heuristics are created to help programmers handle more characters, in particular CJK characters, without dramatically changing the code. They however do not solve the core problem (that is, visual width is fundamentally ill-defined) and they often could not handle tricky cases such as emoji sequences. Many compilers are at this level.
  • Level 2a: Stability under very limited assumptions on which characters should have the same widths. For example, if a Unicode art only assumes Unicode box-drawing characters are of the same visual width (which is the case in all conceivable situations), then its stability is at this level. However, the phrase "very limited" is somewhat subjective, and thus we present a more precise version below.

  • Level 2b: Stability under only theses assumptions:

    Level 2b is making explicit what Level 2a means; we might update the details of Level 2b later to better match our understanding of Level 2a. Collectively, Levels 2a and 2b are called "Level 2".

  • Level 3 (the most stable): Stability under only one assumption that equivalent (extended) grapheme clusters have the same visual width (the last assumption of Level 2b). This means that the Unicode art will remain visually pleasing in almost all situations. It can even be rendered with a variable-width font.

Unlike most implementations, which are at Level 1, our terminal backend strives to achieve Level 2. That means we must not make any assumption about the visual width of end users' code and must abandon the idea of column numbers. As a result, our terminal backend never uses column numbers and we consider that as a significant improvement. We believe Level 3 is too restricted for compiler diagnostics because we cannot show line numbers along with the end users' code. (We cannot assume the numbers "10" and "99" will have the same visual width at Level 3.)

Note: a fixed-width font with enough glyphs that covers many Unicode characters is often technically duospaced, not monospaced, because many CJK characters would occupy a double character visual width. Thus, we do not use the terminology "monospaced".

Raw Bytes as Positions

All positions should be byte-oriented. We believe other popular alternatives proposals are worse:

  1. Unicode characters (Unicode scalar values): This is a reasonable and technically well-defined choice. The problem is that it may take linear time to count the number of characters from raw bytes without a clever data structure (unless we are using UTF-32), and they often do not match what end users perceive as "characters". In other words, it takes more time to compute and may invite misconceptions about Unicode characters.
  2. Code units used in UTF-16: This is somewhat similar to Unicode characters, but with quirks from UTF-16: a Unicode scalar value above U+FFFF (such as 😎) will require two code units to form a surrogate pair. This scheme was unfortunately chosen by the Language Service Protocol (LSP) as the default unit, and until LSP version 3.17 was the only choice. The developers of the protocol made this choice probably because Visual Studio Code was written in JavaScript (and TypeScript), whose strings use UTF-16 encoding.
  3. Unicode (extended) grapheme clusters or user-perceived characters. The notion of grapheme clusters can help segment a Unicode text for end users to edit or select part of it in an "intuitive" way. It is not trivial to implement the segmentation algorithm (see the OCaml library uuseg) and the default rules can (and maybe should) be overriden for each application. The complexity and external dependency of grapheme clusters make it an unreliable unit for specifying positions. It also takes at least linear time to count the number of grapheme clusters from raw bytes.
  4. Column numbers, the visual width of a string in display. As analyzed in the above section, this is the most ill-defined unit of all, and a heuristic that can give passable results in most cases still takes linear time.

Know Bug: Our LSP prototype does not handle positionEncoding yet, and because the default unit in LSP is based on UTF-16 (see above), an LSP client may be confused about the byte-oriented ranges returned by this library. A proper LSP implementation should negotiate with the client to determine how to represent column positions (and our current prototype does not). On the other hand, it can be tricky to negotiate with the client to use raw bytes because there is not an official predefined encoding scheme for raw bytes yet.

\ No newline at end of file