Clean up documentation output

[adam-fowler]

Currently it just strips out all the html tags and outputs that. Which means for some API documentation it just creates super long lines of text, which Xcode turns into a wall of text which is unreadable.

• break text based on <p>,<h*> tags
• add bullet points for <li> tags
• Remove href and only show body for <a> tags
• Limit line length to 100 characters

[RobbeRamon]

Hi Adam, can I provide you some help with this?

[adam-fowler]

@RobbeRamon yes that'd be great if you could look into this. Do you need some guidance on where to start?

[RobbeRamon]

I definitely could use some guidance given that this will be my first deep dive into this codebase

[adam-fowler]

I'll write something up tomorrow

[adam-fowler]

@RobbeRamon
Here we go. Below is basic description of how the CodeGenerator works.
The CodeGenerator code can be found here https://github.com/soto-project/soto/tree/main/CodeGenerator. It uses a series of Mustache templates to render each file. For each service file (api/shapes/errors/paginators) the CodeGenerator creates a context struct which is rendered with the relevant mustache template. Code for this can be found here.

The contexts are created in AWSService.swift. There are a number of places where comments are generated inside the contexts. They all call stripHTMLTags. In general they are written as follows

self.stripHTMLTags(docs)?.split(separator: "\n") ?? []

which strips all the HTML tags from the docs and splits the text on all newlines, giving you an array of Substrings

First thing I would do is break this off into a separate function and use this function throughout, not sure why I haven't done this already. You can then use this function as the base for all your work.

While developing you probably don't want to be re-creating all the services on every run it is a very expensive operation that creates 900,000 lines of code. If you are running of a laptop it will kill your battery. You can limit to just outputting one service by using the command line argument --module eg

soto-codegenerator --module s3 --format

I include the --format argument because the final generated code is generated with this argument (it runs swiftformat on the generated code). If you want to do comparisons with what is already there this'll make it easier.

I've added some thoughts on what needs to be done to clean up code in the main comment of this issue.

Given code generation is already an expensive operation trying to keep your code as optimal as you can. I don't really want to see code generation time double, but I also understand this work will have impact on code generation speed.

Oh and finally if you need any more guidance please ask and thank you for the offer of doing this