Dedicated compiler flag for building documentations

[stevapple]

Vapor uses a BUILDING_DOCC flag with documentation building, which opts out of @_exported imports that are annoying in the documentation. We would like similar feature with SPI too.

Note that Vapor doesn’t use the DocC plugin. AFAIK the plugin doesn’t support such feature, but it’s rather useful.
https://github.com/vapor/api-docs/blob/e282e02bee7c7f25a61e4744c9add5f067a5af08/generate-package-api-docs.swift#L47

[finestructure]

We've very recently added support for custom docc flags. Would that help?

You can set them via custom_documentation_parameters in your .spi.yml file. For example:

version: 1
builder:
    configs:
    - documentation_targets: [t0]
      custom_documentation_parameters: [--foo, bar]

[stevapple]

Note that Vapor doesn’t use the DocC plugin. AFAIK the plugin doesn’t support such feature.

[finestructure]

Ah, I see. I that case that doesn't help, I suppose. We generate the docs via the plugin.

[stevapple]

I may expect some workaround here🤔️ eg. hooking into the underlying build process and inject some arguments to Swift compiler invocations. I’ll also forward the feature request to the DocC plugin.

[stevapple]

How did SPI inject the DocC plugin dependency?🤔️ Can we use similar technique to inject swiftSettings: [.define("SPI_DOCC")] for the target to build documentation against?

[finestructure]

So if I understand correctly, there's a chance you're already able to configure this yourself right now via the package manifest? Do you have a package to test this with?

And more generally I wonder, is this something we should consider enabling globally for our doc builds? Is it ever useful to include the documentation of @_exported imports?

[stevapple]

So if I understand correctly, there's a chance you're already able to configure this yourself right now via the package manifest?

Not really. We need at least an environment variable to judge the build context, and that’s what I wish SPI can do and it should not affect other users without opting in.

And more generally I wonder, is this something we should consider enabling globally for our doc builds?

It’s hard to do actually. Most packages using @_exported imports don’t support building without these re-exported interfaces and if there’s no dedicated CI check the support is very fragile and easy to break. Also, removing @_exported requires to audit all source files, which is possible with SwiftSyntax but just not worth it.

Is it ever useful to include the documentation of @_exported imports?

I can come up with a use case where a module is an extension or umbrella for others, and since the underlying module is rarely used or not intended to be referenced, re-exported interface should be treated like the native one.

Such target include NIO in https://github.com/apple/swift-nio (umbrella, going to be deprecated though) and AWSLambdaRuntime in https://github.com/swift-server/swift-aws-lambda-runtime (extension/wrapper).

[finestructure]

Oh, I see. I didn't pay close attention to the name of the env variable and thought this was one defined by docc which would handle opting out of generating docs for exported imports. I see now that it's used in the package source to fence the exports.

If we do this I'd like to try and avoid injecting more things into the Package.swift file. These changes make it harder to debug build failures due to invalid manifests when our injections fail.

We've had a recent request for an SPI_BUILD env variable and it seems to me the two are actually the same thing, for the two different "build steps".

So if we set SPI_BUILD and SPI_DOCGEN authors could perform custom logic during the build and doc generation stages.

That would help your use case as well, wouldn't it?

[stevapple]

So if we set SPI_BUILD and SPI_DOCGEN authors could perform custom logic during the build and doc generation stages.

That would help your use case as well, wouldn't it?

Certainly!

[finestructure]

Ok, tracking this here: #2336

[finestructure]

This is implemented as SPI_GENERATE_DOCS (see #2336)

[stevapple]

Huge thanks! I wonder if it’s possible to trigger a test build before it’s merged into main. Also, I’d like to test it on all platforms (especially iOS which is using DocC through xcodebuild) although we only build documentation for Linux. Could you offer some help here? @finestructure

[finestructure]

The change is already live. We can run builds on branches but if you were to set a pre-release tag that would be picked up by our build system. Would that work?

[stevapple]

Well, it’s interesting to find that a swift-symbol-extract bug actually prevents us from hitting this issue with swift-docc-plugin… This means there’s basically no difference in SPI documentation with or without @_exported imports, until the bug is addressed.

I’ll postpone updating my package until the bug is addressed. Anyway, thanks a lot for the quick response and maybe someone else might also want it❤️