Make picolibc the default C library for Zephyr

[stephanosio]

#44096 introduced the picolibc, a full-featured yet light-weight C library and fork of newlib, as a module for Zephyr that can be configured and fine-tuned at the Zephyr build time to minimise its footprint.

This issue describes the tasks required for making the picolibc the default C library support in the Zephyr RTOS.

Tasks

• Adding picolibc as a module #44143
• Add picolibc documentation #49620
  • doc/develop: Add picolibc documentation #54293
• Ensure that picolibc is compatible with and tested on all architectures supported by Zephyr
  • aarch64-zephyr-elf
  • arc-zephyr-elf
  • arc64-zephyr-elf
  • arm-zephyr-eabi
  • mips-zephyr-elf
  • nios2-zephyr-elf
  • riscv64-zephyr-elf
  • sparc-zephyr-elf
  • x86_64-zephyr-elf
  • xtensa-*zephyr-elf
• Bundle picolibc in Zephyr SDK for libstdc++ integration
  (Add picolibc sdk-ng#602)
• Fix any incompatibilities with the picolibc in the Zephyr codebase. These are validated in Complete set of changes to enable picolibc by default #60124.
  • modules/hal/openisa: Support building without -ffreestanding #51908
  • Add -lc -lgcc from picolibc toolchain library after module libraries #51949
  • (modules: Update picolibc to 1.8 #53337)
  • (Allow use of SDK version of picolibc with C++ #53338)
  • (libc: Add remaining picolibc-supported targets #53339)
  • (samples/modules/chre: Support picolibc %p for NULL pointers #53574)
  • (samples/philosophers: Make output atomic with picolibc #53575)
  • (sample/zbus: qemu_nios2 needs faster tick for zbus test with picolibc #53684)
  • (Enabling TLS detects stack overflow in a few tests #53687)
  • (modules/cmsis: Change libc requirement to allow picolibc #53950)
  • subsys/bluetooth: Avoid RX overflow in lt_tx_real_no_encode #54119 (no longer appears to cause test suite failures)
  • qemu_x86_tiny places picolibc text outside of pinned.text #54148
  • lib/libc: Enable picolibc on qemu_x86_tiny #54186
  • modules/picolibc: Include clang+cmake and old GCC changes #54394
  • Switch return type of main from 'void' to 'int' #54628
  • Allow tests to filter on availability of full C library #54637
  • Only use -ffreestanding with minimal C library #54638
  • modules/audio/sof: Pull fix for main return type #57398
  • Fix linker scripts to define _end correctly #57727
  • libc/common: Add memalign #58012
  • libc/common: malloc fixes #58450
  • tests, samples: Fix a bunch of new instances of void main(void) #58458
  • tests: Increase a couple of test stack sizes #58459
  • tests/c_lib: strerror_r is POSIX #58460 (Removes _POSIX_C_SOURCE from CFLAGS: not a compatibility fix)
  • libc/picolibc: Picolibc fixes #58462
  • tests/iterable_sections: Set common stack arena size to zero #58464 (no longer necessary for this test to work)
  • tests/spinlock: Terminate temporary thread after tests #60123 (Fixed via Testcases fixes #60567)
  • samples/boards/google_twinkie_v2_pda: fix main return #61233
• Document the policy to ensure that the Zephyr kernel can keep building with the minimal libc
  • Add coding guidelines on C standard library function usage #57598
• Document the policy to prevent the abuse of non-standard functions available in Picolibc
  • Add coding guidelines on C standard library function usage #57598
• Implement tests and CI checks to enforce the above policies
  • Enforce "Rule A.4: C Standard Library Usage Restrictions in Zephyr Kernel"
    • Build the "Zephyr kernel" tests with the minimal libc to ensure that they are compatible with the minimal libc. tests/kernel/common: Add an explicit test using the minimal C library #58108
  • Enforce "Rule A.5: C Standard Library Usage Restrictions in Zephyr Codebase"
    • Add _ZEPHYR_SOURCE support to Picolibc to limit the visibility of non-standard functions from the Zephyr codebase. modules/lib/picolibc: Use version with _ZEPHYR_SOURCE support #58106
    • Add a CI check to prevent any upstream source files from defining the feature test macros (e.g. _GNU_SOURCE) to pull "disallowed" functions. scripts/checkpatch: Check for patches adding #defines for libc APIs #60487
• Document module and toolchain-bundled Picolibc synchronisation policy
  • Initial proposal: Make picolibc the default C library for Zephyr #49922 (comment)
  • Zephyr doc update: Add module and toolchain-bundled Picolibc synchronisation policy #58006
  • sdk-ng doc update: README: Add submodule update process sdk-ng#673
• Make PICOLIBC the default LIBC_IMPLEMENTATION
  • libc: Default to picolibc where supported #57340

Related discussions

• picolibc footprint comparison
• Rationale and overall requirements
• Configuration requirements
• Adding picolibc as a module

[keith-packard]

I think we should use an SDK version that includes picolibc -- that way we can run both C and C++ tests and use the picolibc module and the SDK version. I think that means getting picolibc into a version of the SDK that we can use from the github CI system?

[stephanosio]

I think we should use an SDK version that includes picolibc -- that way we can run both C and C++ tests and use the picolibc module and the SDK version. I think that means getting picolibc into a version of the SDK that we can use from the github CI system?

Yes, zephyrproject-rtos/sdk-ng#287 will need to be completed as part of this.

[stephanosio]

cc @nashif @carlescufi @tejlmand

[stephanosio]

Toolchain WG on 2022-09-20

• @nashif: This will need to be discussed and reviewed at the project level (TSC).
• @nashif: In general, no strong objection to the general direction of making picolibc the default C library; however, there are some concerns regarding the subsequent effects that this can have:
  • @nashif: How do we prevent the core parts of Zephyr (e.g. kernel) from abusing all the available C library functions that may lead to overall footprint increase?
    • @stephanosio: We can define a minimal set of C standard library functions that are sanctioned to be used in the core parts of Zephyr and guard any extraneous functions by a Kconfig such that they are not available for use by default.
  • @nashif: Using the C standard library functions that are supported by the picolibc but not by the minimal libc will render Zephyr unusable with the minimal libc. How do we prevent that?
    • @stephanosio: We can take a similar approach as the above. But, the minimal libc is TOO minimal and it should not be the "golden standard" to which all Zephyr subsystems are doomed.
    • @stephanosio: For kernel, limiting the availability of the C standard library functions to what is currently provided in the minimal libc seems reasonable. For other subsystems, more flexibility should be allowed even at the cost of breaking the compatibility with the minimal libc.

[keith-packard]

Is the eventual plan to get rid of the minimal libc?

[stephanosio]

Is the eventual plan to get rid of the minimal libc?

Not in the foreseeable future (it will be too controversial). The minimal libc will stay to provide somewhat self-contained? way of testing the kernel.

To quote what @nashif posted in the Teams chat:

[1:10 AM] Nashif, Anas
in zephyr 1.0.0....

[1:10 AM] Nashif, Anas
config MINIMAL_LIBC        bool        prompt "Build minimal c library"        help        Build integrated minimal c library. This integrated library is available        to support kernel functionality and test cases. It is not designed to be        used with applications. For applications, please use an external C        library such as newlib.

[nashif]

Is the eventual plan to get rid of the minimal libc?

Not in the foreseeable future (it will be too controversial). The minimal libc will stay to provide somewhat self-contained? way of testing the kernel.

My point above was that we need to just be cautious with such a move (picolibc as the default libc, dropping minimal, etc.) and think about all possible consequences that might affect users. Last time we tried to make such a move (make newlib default), there were countless issues and that did not happen. (#3102)

[keith-packard]

Last time we tried to make such a move (make newlib default), there were countless issues and that did not happen. (#3102)

Thanks for that link, really useful to be able to review what happened in the past. It looks like picolibc has a few benefits that should help resolve some of the issues from the past:

• picolibc's printf is smaller than printk, so we can use printf everywhere and save space.
• picolibc uses TLS for all thread-specific data, avoiding reentrancy issues and the large struct _reent
• picolibc can be built as a module, instead of being tied to the toolchain.

We also need to ensure that picolibc sees more testing and that bugs related to using picolibc are resolved. I had been working on this slowly during the early picolibc integration work, and many of the patches required to work with picolibc have been merged.

We also need to get Zephyr-specific changes integrated into picolibc -- to make sure the right POSIX APIs are exposed in Picolibc headers when compiling Zephyr code. There are already bits for a few operating systems present, so it's easy and natural to add another one.

Finally, we need to get the Zephyr SDK built to include picolibc along with libstdc++ support. When building with C++, it would be quite a bit of work to get libstdc++ built as a module, so instead we should use the existing crosstool-ng support to get libstdc++ built for picolibc and then use picolibc from the SDK instead of as a module. That's blocked on deciding how to package the SDK to include picolibc. Simply adding picolibc will increase the SDK size quite a bit, potentially going over 2GB. Add to that the requests for including debug symbols in the SDK to ease development.

[nashif]

@stephanosio @keith-packard how far can we get by enabling picolibc as the default in Zephyr as a module? Would this work with all supported toolchains? Adding picolibc to the SDK is fine, however we need to make sure other non-SDK toolchains still work and are not left behind and there need to be parity. We should not get into the situation where using zephyr with some features enabled forces the use the Zephyr SDK.

[stephanosio]

Would this work with all supported toolchains

Noting that even the minimal libc seems to be having issues with some proprietary toolchains (e.g. with armclang as mentioned by @tejlmand during the meeting), it will likely need some tweaking.

But, in principle, it is no different to the minimal libc in that it can be adapted to work with any toolchains we support now and in the future because it is integrated to the Zephyr build process and we build it from source.

however we need to make sure other non-SDK toolchains still work and are not left behind and there need to be parity. We should not get into the situation where using zephyr with some features enabled forces the use the Zephyr SDK.

As long as the toolchain itself allows using a custom C library (i.e. not linking to the default C library embedded in the toolchain), there is nothing fundamentally preventing the picolibc module from working with them (basically, the same requirement as the minimal libc).

p.s. one of the reasons for adding picolibc as a module was to allow it to be used in a toolchain agnostic way (see #44143).

[keith-packard]

What kind of per-architecture testing are you looking for? Do you want external picolibc tests? Or internal Zephyr tests? External picolibc tests rely on architecture-specific support for stand-alone applications to be included in picolibc, which would be nice, but is a pretty big lift for some architectures. Picolibc currently has this for arm, aarch64, power9, risc-v and x86. Internal zephyr tests don't use much of the library, so coverage is weak. I'd be happy to include additional architectures in the picolibc test suite, if someone wants to provide the code.

[evgeniy-paltsev]

Just a note for the case of ARC (both arc and arc64) & ARC MWDT toolchain we want to keep Zephyr minimal libc as a default - as we doesn't support piclolib with MWDT toolchain.

Not fully sure if we want to switch ARC & GNU toolchain to have picolibc as default (as we don't support it for ARC officially)

[keith-packard]

Just a note for the case of ARC (both arc and arc64) & ARC MWDT toolchain we want to keep Zephyr minimal libc as a default - as we doesn't support piclolib with MWDT toolchain.

Is that 'don't currently support' or 'aren't interested in ever supporting'? Iow, would picolibc support be possible in the future? Picolibc supports a fairly wide range of compilers, so adding arc mwdt compiler support should be pretty simple.

Not fully sure if we want to switch ARC & GNU toolchain to have picolibc as default (as we don't support it for ARC officially)

Makes sense; providing support for multiple toolchains isn't free.

[keith-packard]

I'm gonna mark arm32, arm64, riscv and x86_64 as 'tested' -- they're all tested with the picolibc (and glibc) test suites, and they all pass the twister tests (with the pending PRs, ofc).

[stephanosio]

Technically, I think we're almost there (blocked on #54637, which is waiting for @galak to review). Given the scope of the change here, should this be brought to the TSC for a ruling?

@keith-packard I believe it should be brought up and at least mentioned to give people a chance to weigh in.

@jgl-meta @keith-packard This was brought up in the TSC meeting a few weeks ago during the Toolchain WG report. The only concern was ensuring that the core parts of Zephyr (mainly kernel) can still build using the minimal libc (i.e. coming up with a test strategy to ensure that the kernel does not make use of any C standard functions that are not part of the minimal libc).

[stephanosio]

BTW, in case of ARC we are keeping minimal libc as default, however we want to do testing with picolibc on all platforms (in case of GNU toolchain).

I'm wondering if there is a way to do that easily without duplicate each platform to have picolibc platform version?

@evgeniy-paltsev We will need to keep Picolibc as the default libc for the Zephyr SDK ARC GNU toolchain (i.e. what is used in the CI) so that we can build the tests and samples that make use of the C standard functions that are not part of the minimal libc.

As for the MetaWare toolchain, you can keep PICOLIBC_SUPPORTED de-selected to indicate that Picolibc is not supported on it and therefore cannot be the default libc when using the MetaWare toolchain.

[stephanosio]

During the discussion of #57340 in the TSC meeting on 2023-05-03, the following concerns were raised:

1. What is the scope of the "Zephyr kernel" that is supposed to be compatible with the minimal libc?
2. How do we prevent people from committing code that uses non-standard functions available in Picolibc.

I will look into addressing these concerns by:

1. Documenting the scope of the "Zephyr kernel."
2. Implementing a test to ensure that "Zephyr kernel" does not make use of any functions that are not available in the minimal libc.
3. Documenting the list of allowed standard C library functions that can be used in the broader Zephyr codebase.
4. Implementing a CI check to ensure that disallowed standard C library functions are not used.

[keith-packard]

2. Implementing a test to ensure that "Zephyr kernel" does not make use of any functions that are not available in the minimal libc.

I think this duplicates 4. below?

3. Documenting the list of _allowed_ standard C library functions that can be used in the broader Zephyr codebase.

I suggest that this should be explicit about whether Zephyr expects these functions to follow the POSIX or C standard, as well as a general policy about whether functions that exist only in the POSIX standard could be considered for inclusion in this list.

4. Implementing a CI check to ensure that _disallowed_ standard C library functions are not used.

Should this instead say "that only allowed standard C library functions are used"? Seems like an allow list will be easier to manage than a disallow list.

[andyross]

Should this instead say "that only allowed standard C library functions are used"? Seems like an allow list will be easier to manage than a disallow list.

Or just "the allowed list is whatever is in minimal libc", and the needed CI check is "build everything in tests/kernel with minimal libc".

[keith-packard]

Or just "the allowed list is whatever is in minimal libc", and the needed CI check is "build everything in tests/kernel with minimal libc".

That's certainly easier than scanning code for use of symbols present in picolibc but not present in the minimal C library.

[keith-packard]

I hacked up a script that generates a list of symbols exported by picolibc and not exported by the minimal C library by scanning the public header files from both. It's a terrible combination of coccinelle and classic Unix tools; there's probably a much cleaner way of doing this ...

keith-packard@310e7ed

It reads the picolibc headers from the SDK and the minimal C library headers from Zephyr sources.

[stephanosio]

Should this instead say "that only allowed standard C library functions are used"? Seems like an allow list will be easier to manage than a disallow list.

Or just "the allowed list is whatever is in minimal libc", and the needed CI check is "build everything in tests/kernel with minimal libc".

That was the original plan and the TSC was not satisfied with that.

As mentioned in #49922 (comment), first of all, we need to clearly define the scope of "Zephyr kernel" because limiting its scope to kernel/ is not enough. After that, we can document the policy and CI check for that -- now that is the first part.

The second part is to come up with an allow list of the allowed standard C library functions (and naturally a disallow list of the non-standard and deprecated standard C library functions available in Picolibc and other common "full" LIBCs) for the broader Zephyr codebase (i.e. not just the kernel), because we do not want people abusing the non-standard functions and the deprecated standard functions provided in Picolibc.

The first part is quite straight forward once we clearly define the scope of "Zephyr kernel" (though, that it is by no means an easy task because everyone will have a slightly different opinion on what "Zephyr kernel" or "core parts of Zephyr" really is).

The second part can be implemented in a few different ways; for example:

1. Modifying Picolibc to self-limit the scope of functions available (something like __ZEPHYR_VISIBLE).
2. Code scan (CI check) using the disallow list of non-standard and deprecated standard C library functions

I am leaning towards the second option because it is a more general solution (as in not libc-specific) and can be used to enforce the our "libc function usage policy" across multiple LIBCs, which is important because we may have CI workflows for third-party toolchains that do not use Picolibc in the future.

[stephanosio]

The second part can be implemented in a few different ways; for example:

1. Modifying Picolibc to self-limit the scope of functions available (something like __ZEPHYR_VISIBLE).
2. Code scan (CI check) using the disallow list of non-standard and deprecated standard C library functions

I am leaning towards the second option because it is a more general solution (as in not libc-specific) and can be used to enforce the our "libc function usage policy" across multiple LIBCs, which is important because we may have CI workflows for third-party toolchains that do not use Picolibc in the future.

Another simpler option could be to:

1. Use Picolibc with _ANSI_SOURCE so that only the standard functions are available.
2. Define the prototypes of the allowed extension functions in the Zephyr-side local Picolibc headers to only permit the allowed extension functions to be used (Add coding guidelines on C standard library function usage #57598).
3. Add a CI check to prevent any upstream source files from defining the feature test macros (e.g. _GNU_SOURCE).

This will effectively allow us to restrict the Picolibc function usage to what is proposed in #57598, without coming up with an explicit disallow list for the CI check.

Given that any other C standard libraries implementing extensions will have some form of feature test macros, this should also be adaptable across multiple LIBCs.

[keith-packard]

Another simpler option could be to:

1. Use Picolibc with `_ANSI_SOURCE` so that only the _standard_ functions are available.

2. Define the prototypes of the allowed extension functions in the Zephyr-side local Picolibc headers to only permit the allowed extension functions to be used ([Add coding guidelines on C standard library function usage #57598](https://github.com/zephyrproject-rtos/zephyr/pull/57598)).

Alternatively, adding support for _ZEPHYR_SOURCE to picolibc so that it would expose the Zephyr API set directly, allowing code integrated into Zephyr to define _POSIX_SOURCE, _BSD_SOURCE, _DEFAULT_SOURCE or _GNU_SOURCE as desired. As we want to move to that solution in the future anyways, how about we just get there now instead? It's all the same work -- auditing the set of functions exposed by the library, but placing the results in the picolibc source code will ensure that it's handled uniformly wrt the other _SOURCE macros.

3. Add a CI check to prevent any upstream source files from defining the feature test macros (e.g. `_GNU_SOURCE`).

There are existing uses of _DEFAULT_SOURCE, _GNU_SOURCE, _BSD_SOURCE and _POSIX_C_SOURCE in the code, which would need to get fixed. Fortunately not many though.

This will effectively allow us to restrict the Picolibc function usage to what is proposed in #57598, without coming up with an explicit disallow list for the CI check.

This would also allow code which happens to use these names not get caught by the disallow list on accident. It wouldn't catch code which includes explicit prototypes for these functions from gaining access to the underlying C library function. Which error would you prefer?

Given that any other C standard libraries implementing extensions will have some form of feature test macros, this should also be adaptable across multiple LIBCs.

What other C libraries are relevant here? Aside from newlib, I thought most of the others were basic C libraries without POSIX support? And I would not be surprised if we could get newlib to adopt the _ZEPHYR_SOURCE changes.

[stephanosio]

Alternatively, adding support for _ZEPHYR_SOURCE to picolibc so that it would expose the Zephyr API set directly, allowing code integrated into Zephyr to define _POSIX_SOURCE, _BSD_SOURCE, _DEFAULT_SOURCE or _GNU_SOURCE as desired. As we want to move to that solution in the future anyways, how about we just get there now instead? It's all the same work -- auditing the set of functions exposed by the library, but placing the results in the picolibc source code will ensure that it's handled uniformly wrt the other _SOURCE macros.

No problem with this approach whatsoever; in fact, I prefer this to #57619. I am kind of rushing because I want to get this done in one way or another before the 3.4 feature freeze.

There are existing uses of _DEFAULT_SOURCE, _GNU_SOURCE, _BSD_SOURCE and _POSIX_C_SOURCE in the code, which would need to get fixed. Fortunately not many though.

Yes, fixing those should be quite trivial (in theory).

This would also allow code which happens to use these names not get caught by the disallow list on accident.

Correct.

It wouldn't catch code which includes explicit prototypes for these functions from gaining access to the underlying C library function. Which error would you prefer?

Sure; but, brute-force attempts like that will be easily noticed and rejected during a human code review.

What other C libraries are relevant here? Aside from newlib, I thought most of the others were basic C libraries without POSIX support?

This was more or less hypothetical -- if we ever want to add support for a libc that includes extensions in the future.

And I would not be surprised if we could get newlib to adopt the _ZEPHYR_SOURCE changes.

• libc: newlib/picolibc: define prototypes for non-ansi C functions #57619 (comment)

[keith-packard]

Which kernel tests do you want built with the minimal C library? I'm working on a PR to add kernel tests that use CONFIG_MINIMAL_LIBC=y to select the minimal C library.

• tests/kernel/common: Add an explicit test using the minimal C library #58108

[JordanYates]

Potential issue: printing 64bit integers (long long support) using the pre-built libraries requires using the full floating point library variant. Currently PICOLIBC_IO_LONG_LONG doesn't do anything for pre-built libraries, and values are silently truncated to 32bits.

• picolibc: long-long print support not enabled on prebuilt library sdk-ng#684

[tejlmand]

Toolchain wg: @keith-packard anything left before this is ready for merge ?
All issues in #57340 are resolved.

[keith-packard]

Toolchain wg: @keith-packard anything left before this is ready for merge ? All issues in #57340 are resolved.

The last run of #57340 appears to have hit a regression, so we should get that fixed. Otherwise, yes, this is ready to merge.

[keith-packard]

The regression isn't related to picolibc at all; it's caused by a change in how the microbit emulator is run. #61719 has the details.