Documentation - general discussion of plan.

[Ben-PH]

This is a follow-up to the discussion raised in the recent community meeting.

The esp-rs ecosystems documentation is ripe for applied effort. My initial intent is along the lines of:

Generate a collective will, and sense of direction, that will result in the esp-hal documentation being an ideal place to initially explore and learn embedded development, develop the basic ins-and-outs, and ultimately serving as a launching platform for deep-dives into embedded/electrical-engineering theory.

For example:

When I came across fn into_pull_up_input() I wandered through a maze of interpretations. If you plotted my path through this maze, it would look a bit like a cross between a breadth-first-search and an A* search that was a bit inaccurate in terms of where the destination was:

• It's an input, so that means that this pin is on the input side of a curcuit, right?
• no, it's an input because i can input data from it.
• it's pull-up, so that means it's a 1 if it's pulled-from-up, right?
• close: it's "resting" in the high state, you can choose it being a 1 or 0.
• So that means the floating input means that it rests in either the high, or low state, right? similar to a d-latch, right?
• no, it's high or low, based mostly on EM noise.
• so that probably means that push-pull-output is a tool to pull another pin high, and to push it low, right?
• no, it pulls a high-pin back down, or pushes a low-pin back up.

This is a rough representation of how I arrived at how I think about gpio pins now. Now consider this at the top of the gpio module:

//! a [`gpio::GpioPin`] is the primary means for data to be shared across the software-hardware
//! boundary by means of connecting to a peripheral of some sort (e.g. a WiFi antenna, USB,
//! keyboard-matrix, etc.)
//!
//! Combining two key mechanisms are used for this: First, a pin configured as an [`InputPin`] can be "forced to a
//! high/low resting state", coupled with an external means to (temporarily) drive an input pin out of its rest-state. 
//!
//! These two mechanisms allow the state of an [`InputPin`] to be read, and interpreted, as a
//! binary value. For example, an input pin that rests in a high-state, would stay high but not if it's
//! connected to ground.
//!
//! One can detect if a switch is pressed or not, by connecting a resting-high input pin to a
//! ground-pin via a switch: Is the pin reading a low-state? Button must surely be pressed! See the
//! `button-led` example code to see this in action.
//!
//! A Gpio pin can also act as ground, by working in the `OutputPin` role, and being set to low. There
//! is the added flexibility, that it can be set to high as well, thus driving a "resting-low/high" pin
//! into a high/low state, depending on how the two pins are configured. 

...combine that with doc-comments on the functions, such as:

    /// Configures this pin such that its state (high, or low) can be read. Rests in a high state,
    /// but can be driven into a low state by external means
    /// ```ignore
    /// // connected to ground, or a push-pull output pin set to low
    /// let pin3 = io.pins.gpio3.into_pull_up_input();
    /// // normally "pulled up" by means of internal mechanism, a pushed button means that ground
    /// // is pulling it back down to low
    /// let button_pressed = pin3.is_low();
    /// ```
    pub fn into_pull_up_input(self) -> GpioPin<Input<PullUp>, GPIONUM> {
        self.init_input(false, true);
        GpioPin { _mode: PhantomData }
    }

This example intentionally glosses over some of the lower level details, such as why a floating pin isn't normally used, what a pull-up resistor is, why the jargon is the way it is, and is likely to have an experienced embedded developer thinking "duh...". The goal here is to allow the first-time embedded developer to feel like they have the tools to get out of tutorial hell without leaving docs.rs, and a foundation on which they can start asking questions like "how does a pull-up-input pin rest in a high state" to start going down that rabbit hole when they are ready.

To be clear, this is coming from someone that had a complete lack of familiarity going into these things. Coming to the needed understandings has been productive, and I've benefited greatly from the help recieved in the matrix chat. With that said, I believe the labor needed to absorb and internalise these understandings can be reduced, and here are some ideas on how to think about making this happen:

• Does a function/method/const pass the "ELI5 test": For anything likely to be completely new to an embedded first-timer, is it documented in such a way that it covers the majority of use-cases in simple projects, and equips the user with the background and context to do a productive deep-dive?
• If the examples were removed, could someone with a very basic grasp of embedded be able to implement the examples going by the doc-comments only?
• enable the clippy::missing_docs lint, and create a macro that can be used as so: #[allow_reason(clippy::missing_docs, "TODO")] which gets stripped down to a regular allow. At first, we could just do something like use a tool to automagically allow missing docs, then do a sed to replace the those allows with #[allow_reason(clippy::missing_docs, "AUTO GENERATED"]. This will be easy to rg for. At this point, the work needed to pair everything with a good doc-comment (if needed) is clearly laid out, and self-coordinating.
• Consider docs.rs a first-class-citizen. This is probably MUCH easier said than done. This might include things like: clicking the source code behaves more like an LSP goto-definition in well-configed IDE, instead of taking you to this page, using doc-tests where possible and sensible.

ChatGPT4-generated code for the allow_reason macro:

use proc_macro::TokenStream;
use quote::quote;
use syn::{parse_macro_input, LitStr, Meta};

#[proc_macro]
pub fn allow_reason(input: TokenStream) -> TokenStream {
    let parsed_input = parse_macro_input!(input as AllowReasonInput);
    let lint_name = parsed_input.lint_name;
    let reason = parsed_input.reason;

    let expanded = quote! {
        // Reason: #reason
        #[allow(#lint_name)]
    };

    expanded.into()
}

struct AllowReasonInput {
    lint_name: Meta,
    reason: LitStr,
}

impl syn::parse::Parse for AllowReasonInput {
    fn parse(input: syn::parse::ParseStream) -> syn::Result<Self> {
        let lint_name: Meta = input.parse()?;
        input.parse::<syn::Token![,]>()?;
        let reason: LitStr = input.parse()?;
        Ok(AllowReasonInput { lint_name, reason })
    }
}

[bjoernQ]

Our documentation can and should certainly get improved a lot. When reading this, I initially I thought "why not include those things"

After thinking about it a bit more I'm not too sure we should try to explain electronic eginieering things in the API docs.

I see how it could improve the onboarding of new users w/o an embedded background but I don't think we can fully explain those things in the API docs and will end up trying in a poor way with lots of inaccuracies.

Also, if we start explaining those things here, we probably should continue that and explain I2C protocol, SPI protocol including half-duplex, dual and quad SPI etc. etc.

I totally think that linking to e.g. Wikipedia or other resources might be a good idea, however.

What I could imagine is to have more background information in e.g. the no_std-training-book (if we want that - but I think of it more like a resource to learn about the APIs).

Or even better a more general book which just happens to use esp-hal for code examples (pretty much how those Ardunio introductions are which come with those sensor kits you can buy from Amazon and others).

That's at least what I think at me moment - I'm happy to hear other's opinions on this

But don't get me wrong - we should definitely improve our documentation.

[Ben-PH]

Thanks for taking the time to write-up such a detailed reply.

After thinking about it a bit more I'm not too sure we should try to explain electronic eginieering things in the API docs.

I see how it could improve the onboarding of new users w/o an embedded background but I don't think we can fully explain those things in the API docs and will end up trying in a poor way with lots of inaccuracies.

I think this is an important comment. My example came from my personal experience, leaving the goal-posts intentionally absent. Where they settle is anyones guess

Also, if we start explaining those things here, we probably should continue that and explain I2C protocol, SPI protocol including half-duplex, dual and quad SPI etc. etc.

It might help to settle on some guiding principals, as opposed to strictly defining the scope. I would imagine it would be a short list of some sort, made up of clearly defined goals and non-goals. E.g.

Goals of docs.rs/esp32*-hal

• embedded-jargon (e.g. "pull-up") is de-obfuscated to the a reader that has zero embedded experience
• the primary target-audience is $AUDIENCE (e.g. experienced rust-dev, with zero embedded? an experienced C/C++ embedded dev? combination?)
• the $AUDIENCE is referred to reading material for deeper background/context learnings where appropriate
• Lower the barrier-of-entry of embedded learning

Non-goals:

• To be a direct learning resource for general micro-controller theory.
• Simplification where quality of learning is compromised.

I totally think that linking to e.g. Wikipedia or other resources might be a good idea, however.

I would personally push-back on Wikipedia specifically, but (everything before "but" is just context) the general sentiment, I like.

What I could imagine is to have more background information in e.g. the no_std-training-book (if we want that - but I think of it more like a resource to learn about the APIs).

Perhaps a partnership with ferrous, or other orgs, that focus more on the broad embedded space in general?

Or even better a more general book which just happens to use esp-hal for code examples (pretty much how those Ardunio introductions are which come with those sensor kits you can buy from Amazon and others).

I got my start based on the ferrous systems board. I definitely would have spent the money on a kit that walked me through some sort of program, explaining things along the way.

That's at least what I think at me moment - I'm happy to hear other's opinions on this

But don't get me wrong - we should definitely improve our documentation.

We could get started: we could start using an "allow-with-reason" annotation in combination with the lint to ensure everything is documented? Won't change the warnings emitted, but littered through the codebase might be something like #[document_me(note = "add note here to provide context")]. the note could be a reference to an Issue, not there at all, "auto-generated as part of discussion #742", etc.

My thinking: set up the code base so that the desire to see doc-improvements is clearly telegraphed to anyone that looks through the code-base, the context that would shape the work to be done, and sets a low barrier to open-source contribution.