New look and feel of the AVRDUDE output

[stefanrueger]

The increasing AVRDUDE functionality tends to produce output that puts a relatively high mental load on users: there is a lot to read, every other line starts the same with avrdude. It isn't entirely clear where to focus on a scrolling screen of running commentary. This is bound to become worse when/if things like memory lists in -U are processed. We might do worse than looking into ways of improving AVRDUDE's look and feel. For example, I feel that simple things such as the use of coloured UTF-8 check marks such as ✅ and ❌ instead of the avrdude program name might help. But this discussion isn't to give answers:

I'd like to start a discussion to collect ideas and critique from others.

Now that we might get a neat and shiny GUI example from Joerg for v 8.0, we have a licence for a new output/info/debug style 😄

[MCUdude]

I'm actually quite pleased with the current Avrdude output, but I agree that the ✅ and ❌are very visual and can tell the user how the session went without having to read the entire output.

But I could want something more verbose than -qq, but less than -q. Maybe something like this?

avrdude -cdryrun -patmega1284p -Uflash:w:test.hex:r -q
avrdude: device signature = 0x1e9705 (probably m1284p)
avrdude: ✅ chip erased
avrdude: reading input file test.hex for flash
         with 131072 bytes in 1 section within [0, 0x1ffff]
         using 512 pages and 0 pad bytes
avrdude: writing 131072 bytes to flash [100%] (counting from 0%)
avrdude: ✅ flash written
avrdude: verifying flash memory [100%] (counting from 0%)
avrdude: ✅ flash verified
avrdude: ✅ done.  Thank you.

instead of the usual:

C:\Users\Hans\avrdude>avrdude.exe -cdryrun -patmega1284p -Uflash:w:test.hex:r -q
avrdude: AVR device initialized and ready to accept instructions
avrdude: device signature = 0x1e9705 (probably m1284p)
avrdude: Note: flash memory has been specified, an erase cycle will be performed.
         To disable this feature, specify the -D option.
avrdude: erasing chip

avrdude: processing -U flash:w:test.hex:r
avrdude: reading input file test.hex for flash
         with 131072 bytes in 1 section within [0, 0x1ffff]
         using 512 pages and 0 pad bytes
avrdude: writing 131072 bytes flash ...
avrdude: 131072 bytes of flash written
avrdude: verifying flash memory against test.hex
avrdude: 131072 bytes of flash verified

avrdude done.  Thank you.

[stefanrueger]

Great ideas @MCUdude. Looks much cleaner. I like the idea of a numeric progress bar. Could be replacing the ASCII bar (so no intermediate -q level necessary). What about the left column that is invariably populated with avrdude: ? How about replacing that with a slim two-byte column for ✅ and ❌ (and perhaps debug level indication).

avrdude -cdryrun -patmega1284p -Uflash:w:test.hex:r
✅ ATmega1284P initialized and ready to accept instructions
✅ chip erased
✅ reading 131072 bytes for flash from input file test.hex
   in 1 section within [0, 0x1ffff]: 512 pages and 0 pad bytes
   writing 131072 bytes to flash [100%] (counting from 0%)
✅ flash written
   verifying flash memory [100%] (counting from 0%)
✅ flash verified
avrdude done.  Thank you.

What do others think? What do you miss? What would you change?

[dl8dtl]

What about the left column that is invariably populated with avrdude: ?

I thought about the same: it takes a lot of space, costs some extra loops in the software, and bears no information.

[mcuee]

I am not so sure if everyone likes UTF-8 check marks such as ✅ and ❌ though, even though I myself like the idea.

The issues may be with some non-English Windows versions (especially older versions) -- but I am not 100% sure if this is still an issue or not now as I only use English version of Windows. And we can claim that we no longer support Windows 7.

[dl8dtl]

I am not so sure if everyone likes UTF-8 check marks such as ✅ and ❌ though, even though I myself like the idea.

We need to start acting depending on the current locale. Only locales that are sure to allow UTF-8 would be candidates for them (and even then, I guess I have xterm instances that can't handle them correctly). That makes the code more complicated then, because it needs fallbacks for plain ASCII. I'd also allow this at least as a per-user config setting to fall back to ASCII.

I'm not so sure whether we would always place "success" marks right in front of the message: "Reading blahblah" for example could result in a read error – but the success has already been indicated beforehand?

[stefanrueger]

@mcuee @dl8dtl Heard.

If we treat this as log levels then the complication is only in one place, the messaging function. Here an idea for use of Unicode coloured shapes to replace the program name:

🛑 OS error
🔴 Programmer/AVRDUDE error
🔻 Warning
🟩 info
🟦 notice
🟪 notice2
🟧 debug
🔼 trace 
🔽 trace 2

And...

🖖 avrdude done. Thank you.

OK, just kidding about the Vulcan salute. But could place ✅ and ❌ here.

ASCII fallback could be a user option selecting one of three: Nothing, digits, letters (E, e, w, i, n, N, d, t, T)

[stefanrueger]

@dl8dtl How to figure out whether the terminal can deal with UTF-8? Can't they all? It is 2024, not 2004 (when UTF-8 was 12)...

So many possible different schemes for log levels! So many tastes, so best to let the user define what they want

E	e	w	i	n	N	d	t	T	Log level scheme
E	e	w	i	n	N	d	t	T	ASCII letter log levels (might render in different width)
0	1	2	3	4	5	6	7	8	ASCII numeric log levels
Ｅ	ｅ	ｗ	ｉ	ｎ	Ｎ	ｄ	ｔ	Ｔ	Constant width characters used in Chinese or Japanese text
🅴	🅔	🅦	ⓘ	ⓝ	Ⓝ	ⓓ	ⓣ	Ⓣ	All text, double-width characters
🛑	🔴	🔻	🟩	🟦	🟪	🟧	🔼	🔽	Some might show as text symbols, not emoji
🛑	🟥	🚩	🟢	🔵	🟣	🟠	⚫	⚪	Some might show as text symbols, not emoji
🛑	🟥	🟠	🟩	🟦	🟪	🟧	🟢	🟣	Emoji, not doubling up as text
🟫	🟥	🟨	🟢	🟣	🟤	🟡	🟠	🟧	Within code point region U+1F7E1 ... U+1F7EB
👿	👹	😮	🙂	😄	😀	🤔	🧐	🤕	True emoji
🙎	🙅	🙋	👧	👩	👵	🤦	👲	👷	AVRDUDE is an expert for AVR chips; she is really good!

For the benefit of doubt, here a screenshot of above so you see what my browser renders.

[dl8dtl]

How to figure out whether the terminal can deal with UTF-8?

As I said, the very least is that a respective locale has been set. If the locale is just "C" (the default), you can't assume anything beyond ASCII at all.

But then, UTF-8 is just the encoding only. It doesn't say though how many of the possible 2^21 encodable characters are actually available in the font the user is using. Your example table above has even many squares with just a hex number in them on my machine – no idea what's the default font I'm using …

And then, there's users who simply don't like that kind of gimmick. I still see some Germans transcribing German umlauts using ae, oe, ue, probably because they use English keyboards and just don't care about anything beyond ASCII. So the very least, beyond watching out whether the locale supports UTF-8 at all, would be a knob for the user to turn it off.

For log levels, I've seen color coding even in text terminals becoming popular (e.g. colorlogs in Python), but then, on a terminal with a bright background and dark text, that usually completely fails: yellow text (e.g. for "warning level") completely misses its warning effect when being displayed on a white (or light grey, or light yellow) background …

Btw., for ASCII replacements, I'd write ✅ as [v] and ❌ as [x], rather than trying to encode something with acronym letters.

All in all, doing it in a way that makes as many people happy as possible, both long-time AVRDUDE users as well as newbies, I think it's going to be quite a bit of effort for (IMHO) relatively little added value. If people want a GUI, they'd better use a GUI anyway. 🙂

[mcuee]

Tests under Windows 11, Windows Terminal app, you can see CMD prompt has issues. I usually use latest version of PowerShell but I understand that many are still using Windows CMD prompt...

1. CMD prompt:
   

2. Powershell (7.4.2)
   

3. MSYS2 mingw64
   

[mcuee]

Yes, indeed I know CMD.exe will be there for a long time.

So maybe I have to agree with @dl8dtl's comment earlier.

All in all, doing it in a way that makes as many people happy as possible, both long-time AVRDUDE users as well as newbies, I think it's going to be quite a bit of effort for (IMHO) relatively little added value. If people want a GUI, they'd better use a GUI anyway.

[mcuee]

Btw., for ASCII replacements, I'd write ✅ as [v] and ❌ as [x], rather than trying to encode something with acronym letters.

Or maybe we can just forget about the fancy symbols like ✅ and ❌, and just use [v] and [x].

I have no issues to continue using ✅ and ❌ for test-avrdude, as it requires bash and a proper terminal app.

[stefanrueger]

Great input. Yes, I forgot that some terminals that simply don't do Unicode, eg, Windows' command.exe (tbf UTF-8 is only 30+ years old ;-) So, consensus seems to be

• Remove avrdude: prompt
• Change progress bar to running percentage [xx%] at end of line (unless -q, as before)
• Convention to print [x] or [v] at beginning of message to indicate success/failure
• No Unicode above ASCII code points per default
• Shift some of the xmsg_info() messages to xmsg_notice() to crisp up default output

I still can imagine to offer optional settings in the per-user configs that tells the messaging system to indicate log levels on the left-hand side, sth like,

######
# Optional output modifiers. Put the following into the per-user
# avrdude.rc configuration file and define/uncomment the preferred one
#

# If set, displays a loglevel (error, warning, info, ...) at the start of each message
# show_loglevel = "🅴 |🅔 |🅦 |Ⓘ |Ⓝ |ⓝ |Ⓓ |Ⓣ |ⓣ ";
# show_loglevel = "e: |E: |I: |N: |n: |D: |T: |t: ";

# Continuation of messages on the next line is indented with loglevel_indent spaces
# loglevel_indent = 3;

# Replace begin-of-message [v] with this
# show_success = "✅";
# show_success = "OK.";

# Replace begin-of-line [x] with that
# show_failure = "❌";
# show_failure = "Uh-oh.";

# Break line at spaces after position N unless the string is less than N + MARGIN long 
# line_length = "110 + 20";

Keep the ideas and discussion going!

[dl8dtl]

I don't find -q very useful at the moment, as the output is just as lengthy as without, except for the progress bars.

Well, the translation of the q was "quell progress" – so that's what it is supposed to do … I think we later extended it to -qq to suppress even more messages.

[stefanrueger]

q was "quell progress"

I understand @MCUdude to suggest Make q in 8.0 quell process indication and drop some of the msg_info() messages. We could so with code similar to if(!quell_process) pmsg_info("..."); effectively introducing another log level. I am with @MCUdude on trying to reduce the message volume, though I would suggest starting at the default info level and simply moving messages up log levels: we already have 6 of them from info to trace2. For example I'd suggest moving everything out of the info log level that does not alter the state of the chip or the to be written output files: chip erased is info level, but ready to accept instructions would be between notice and debug... That would be my preferred route to making the default output crisp and clear.