Bindgen failing only in a Github pipeline

[KnorrFG]

Hey,
we use bindgen to link some of our C code into a rust program. Generally this works. It works if I build that particular library locally, and it works if I build it in a docker container. It does not, however, work in a github actions pipeline that uses the exact same container. What is even stranger is that it used to work, and now it fails, if I just rerun a job that used to pass (for the same commit it already passed for).

The error is this:

Unable to generate bindings: ClangDiagnostic("../../firmware/lib/hal/hal_config.h:17:5: error: redefinition of enumerator 'BOOTLOADER'\n../../firmware/lib/hal/hal_config.h:18:5: error: redefinition of enumerator 'APPLICATION'\n../../firmware/lib/hal/hal_config.h:19:3: error: typedef redefinition with different types ('enum ImageType_t' vs 'enum ImageType_t')\n../../firmware/lib/hal/hal_config.h:25:3: error: typedef redefinition with different types ('struct IFlashAppConfig_t' vs 'struct IFlashAppConfig_t')\n../../firmware/lib/hal/hal_config.h:29:5: error: redefinition of enumerator 'SPI_INTERFACE_SPI1'\n../../firmware/lib/hal/hal_config.h:30:5: error: redefinition of enumerator 'SPI_INTERFACE_SPI2'\n../../firmware/lib/hal/hal_config.h:31:5: error: redefinition of enumerator 'SPI_INTERFACE_USART1'\n../../firmware/lib/hal/hal_config.h:32:5: error: redefinition of enumerator 'SPI_INTERFACE_USART2'\n../../firmware/lib/hal/hal_config.h:33:3: error: typedef redefinition with different types ('enum SpiInterface_t' vs 'enum SpiInterface_t')\n../../firmware/lib/hal/hal_config.h:36:5: error: redefinition of enumerator 'SPI_BAUDRATEPRESCALER_2'\n../../firmware/lib/hal/hal_config.h:37:5: error: redefinition of enumerator 'SPI_BAUDRATEPRESCALER_4'\n../../firmware/lib/hal/hal_config.h:38:5: error: redefinition of enumerator 'SPI_BAUDRATEPRESCALER_8'\n../../firmware/lib/hal/hal_config.h:39:5: error: redefinition of enumerator 'SPI_BAUDRATEPRESCALER_16'\n../../firmware/lib/hal/hal_config.h:40:5: error: redefinition of enumerator 'SPI_BAUDRATEPRESCALER_32'\n../../firmware/lib/hal/hal_config.h:41:5: error: redefinition of enumerator 'SPI_BAUDRATEPRESCALER_64'\n../../firmwBindgen failing only in a Github pipeline are/lib/hal/hal_config.h:42:5: error: redefinition of enumerator 'SPI_BAUDRATEPRESCALER_128'\n../../firmware/lib/hal/hal_config.h:43:5: error: redefinition of enumerator 'SPI_BAUDRATEPRESCALER_256'\n../../firmware/lib/hal/hal_config.h:44:3: error: typedef redefinition with different types ('enum SpiBaudratePrescaler_t' vs 'enum SpiBaudratePrescaler_t')\n../../firmware/lib/hal/hal_config.h:47:5: error: redefinition of enumerator 'SPI_PHASE_1EDGE'\nfatal error: too many errors emitted, stopping now [-ferror-limit=]\n")

It seems that clang includes hal_config.h multiple times. Even though it has a #pragma once. Replacing the pragma with include guards makes the pipeline pass, but I'd rather not do that, since we don't use include guards anywhere else, and since the problem only occurs in the pipeline.

I was hoping someone has an Idea, or could point me to some usefull information regarding what bindgen is doing under the hood exactly.

[pvdrz]

given that this is a clang error I'd suggest you check whether you're using the same clang version on CI and locally

[pvdrz]

Is there any chance that different arguments are being passed to clang?

[KnorrFG]

The arguments are the same, however the order in which the headers are passed to the header method of the builder seems to be random.

There are multiple jobs running in the pipeline, most of them build the crate that uses bindgen. Three of them fail. I added some code to write a log to disk, so I can see what it does locally. That code was also executed in the pipeline, and it made one job pass. Even when the file has no content. Just calling _ = fs::write("bindgenargs", ""); before constructing the builder makes one of the problem jobs pass. The others, however, still fail.

This led me to think it might somehow be a timing issue, and I added a sleep for 20 ms instead of the file write before builder construction, but it does not help. Neither do 500ms. By now, I'm willing to just use the include guards. Since writing an empty file solves the problem in one place but not in another one, this might even be a bug in a driver ...

[pvdrz]

One option would be to dump the preprocessed headers on CI and locally to discard any differences. If the preprocessed input happens to be different we can start looking back from there.

[KnorrFG]

I did what you proposed, but since the order was random, the dumps were hard to compare. So I sorted the headers before adding them via the header method. This made the problem go away, in all places, without include guards.

My untested assumption is the following: if you add a file via header, the #pragma once is ignored. If you have two files, a.h and b.h, where b.h includes a.h, and you add b.h first, then clang/bindgen will process the contents of a.h indirectly first, and then a second time, when it is a.h's turn. If it's the other way around, the include guard works, when a.h is encountered indirectly while processing b.h. This also makes sense because the header list is generated from a call to fs::read_dir, and the order of the files returned can differ between computers, even if it is the same commit of the same repository.

Thank you for your help and time.