boot_info: A subsystem to share bootloader info

[Laczen]

boot_info is a simple subsystem to share information between a bootloader and an application. It can store the info in a bbram device or a zephyr,memory-region (RAM).

The proposed subsystem can easily be extended to support using a flash partition as backend.
Flash partition backend has been added.

2023/07/20: Eeprom backend has been added.

[nordicjm]

Duplicate of #59025

[Laczen]

Duplicate of #59025

This has nothing to do with #59025. It might be considered an alternative to retained memory but it is not a driver, it is a subsystem that does not require any extra device to be created.

It also provides a solution to use bbram as the backend for storage.

[henrikbrixandersen]

This has nothing to do with #59025. It might be considered an alternative to retained memory but it is not a driver, it is a subsystem that does not require any extra device to be created.

It also provides a solution to use bbram as the backend for storage.

Thanks. Alternatives are nice when evaluating the best solution for introducing new functionality.

To ensure I am understanding the two proposals correctly - this PR adds a generic way of storing/retrieving a byte array between bootloader and application, but it doesn't define any semantics for the contents of this byte array? Whereas #59025 provides a similar functionality, but also adds semantics for the stored data and allows storing/retrieving using the settings API?

[nordicjm]

This has nothing to do with #59025. It might be considered an alternative to retained memory but it is not a driver, it is a subsystem that does not require any extra device to be created.
It also provides a solution to use bbram as the backend for storage.

Thanks. Alternatives are nice when evaluating the best solution for introducing new functionality.

To ensure I am understanding the two proposals correctly - this PR adds a generic way of storing/retrieving a byte array between bootloader and application, but it doesn't define any semantics for the contents of this byte array? Whereas #59025 provides a similar functionality, but also adds semantics for the stored data and allows storing/retrieving using the settings API?

Correct, this PR adds a shared memory area that does not signify anything about the data stored there and does not do anything with mcuboot or other bootloaders out of the box, the other PR adds mcuboot code to store the bootloader information and zephyr code to read that information, and allows it to be accessed via settings keys (the infrastructure is designed so that other bootloader support could also be added, it would have to work in a similar fashion of having the data accessible via settings with the blinfo prefix)

[henrikbrixandersen]

Blocking for now as I cannot see this adding any functionality not already present in the existing retention system implementation.

[Laczen]

Blocking for now as I cannot see this adding any functionality not already present in the existing retention system implementation.

Hi @henrikbrixandersen, thank you for the feedback.

Regarding the added functionality: the PR also provides using bbram as the backend for the boot_info system. The way it is designed also allows to add a flash partition or a general purpose register as backend.

Regarding a comparison with existing solutions: the PR provides a solution without any drivers that need to be created, saving scarce resources. The proposed solution is also simple to setup in a dts file and very simple to use. The retained_mem driver is also so young that it should be allowed to be questioned if this is the best solution for the problem.

The PR does not try to define what the data format is that should be used, and it does so for several reasons:
a. zephyr shouldn't dictate the format, it should follow whatever the bootloader is supplying, any agreement between the format should be done in bootloader-support specific code.
b. the data area can be very small (e.g. 1 byte) dictating that alternative solutions should be found to (if needed) validate the data that is inside the data area (e.g. a parity bit). The agreement on how to use it should also be done in bootloader-support specific code.

To summarize: the PR adds functionality by extending the backends that can be used to share information and it reduces the resources used. At the same time it is also easy to expand to other storage solutions.

Finally, since there is some concern about duplicate functionality you could consider it to replace the retained_mem driver (not that this is a request from my side), the proposed PR would provide the same functionality with reduced resource usage, reduced code size and easy extensibility (one yaml file, one line in boot_info.h, and +/-50 lines in boot_info.c).

[carlescufi]

Architecture WG:

• System to share information between bootloader and application
  • Image positions, product numbers, product data
• Backed can be flash, RAM, BBRAM, EEPROM
• Bootloader and app decide the shared format of the data, no specific format
• As simple as possible so that the bootloader has no overhead, no struct device
• Instances defined in Devicetree
• @Laczen argues that this is leaner than the retention subsystem, MCUboot can be used without any modifications
  • When the system boots you define an area of RAM, that you get with boot_info_get() where MCUboot stores the boot_record info, and then at the end you write boot_info_set(). This means that only the Zephyr port will need to be modified
• @bjarki-trackunit states that the differences between retention and boot_info doesn't seem big enough to justify a separate subsystem. Footprint measurements would be useful
• @Laczen retention is not designed for this usecase in particular, which means it would add overhead

[Laczen]

@carlescufi @henrikbrixandersen @de-nordic @nordicjm mcuboot can easily be modified to share information trough a shared memory (ram) method using:

1. Define a memory region (in dts):

/ {
    bl_shared_ram {
        #address-cells = < 0x1 >;
        #size-cells = < 0x1 >;

        mcuboot_shared_data: sram@20000000 {
            compatible = "zephyr,memory-region", "mmio-sram";
            reg = < 0x20000000 0x0020 >;
            zephyr,memory-region = "MCUBOOT_SHARED_DATA";
        };
    };
};

2. Add the following to mcuboot_config.h (some protection could be added, but this is the base):

#if defined(CONFIG_BOOT_SHARE_DATA) || defined(CONFIG_MEASURED_BOOT)
#define MCUBOOT_SHARED_DATA_BASE DT_REG_ADDR(DT_NODELABEL(mcuboot_shared_data))
#define MCUBOOT_SHARED_DATA_SIZE DT_REG_SIZE(DT_NODELABEL(mcuboot_shared_data))
#endif

3. Add a routine to save what is wanted for image data as (this could be a nop):

int boot_save_shared_data(const struct image_header *hdr, const struct flash_area *fap);

4. Modify the CMakeLists.txt to include boot_record.c.
5. Enable CONFIG_BOOT_SHARE_DATA.

The zephyr application can then read the area using mcuboot specific routines to convert the tlv stored data.

This method could be used by any bootloader.

[nordicjm]

So your solution to what you brought up at the architecture working group is not to use the system you brought up at the architecture working group at all.

@carlescufi can we move forward with this case now being resolved in favour of #59025 ?

[Laczen]

So your solution to what you brought up at the architecture working group is not to use the system you brought up at the architecture working group at all.

No, this is about the bootloader side, the zephyr app side does need some interface to work with.

But as you have noticed, it is indeed possible to use mcuboot without any zephyr specific subsystem. This illustrates how bad the proposal of #59025 really is, it adds lots of things without being required. If this was not known from the start it clearly justifies reworking #59025 to a simpler solution, if it was known there was misleading of developers.

[Laczen]

@carlescufi, during the architecture meeting it was requested to put some figures on the size difference between the retention system and the boot_info system. I have added a test to allow an easy comparison (https://github.com/Laczen/zephyr/tree/retention_test). The result of this test: The retention subsystem adds 2.3kB of code compared to the boot_info subsystem.

To put this in perspective the mcuboot bootloader is considered to be getting obese at around 22kB.

[Laczen]

@carlescufi, during the architecture meeting it was requested to put some figures on the size difference between the retention system and the boot_info system. I have added a test to allow an easy comparison (https://github.com/Laczen/zephyr/tree/retention_test). The result of this test: The retention subsystem adds 2.3kB of code compared to the boot_info subsystem.
To put this in perspective the mcuboot bootloader is considered to be getting obese at around 22kB.

Cannot replicate these results, taking hello_world in a virgin state and building for nrf52840dk_nrf52840 yields the following:

Probably because you have forgotten something (e.g. the retained_mem driver ?), the reference to the test is given and this is a comparison between both systems. Nothing to hide, easy to replicate.

As I said, you forgot to include a part in the comparison (the retained_mem driver):

on the nr52840dk_nrf52840 vanilla hello_world:

Memory region         Used Size  Region Size  %age Used
           FLASH:       20184 B         1 MB      1.92%
             RAM:        5568 B        63 KB      8.63%
     RetainedMem:          0 GB         1 KB      0.00%
        IDT_LIST:          0 GB         2 KB      0.00%

on the nrf52840dk_nrf52840 modified hello_world (your version):

Memory region         Used Size  Region Size  %age Used
           FLASH:       22428 B         1 MB      2.14%
             RAM:        5632 B        63 KB      8.73%
     RetainedMem:          0 GB         1 KB      0.00%
        IDT_LIST:          0 GB         2 KB      0.00%

So the conclusion remains.

[nashif]

@Laczen how come you have ram of 63k on nr52840dk_nrf52840? This board has 256 of RAM as shown here as well: #59375 (comment)

[Laczen]

@Laczen how come you have ram of 63k on nr52840dk_nrf52840? This board has 256 of RAM as shown here as well: #59375 (comment)

Because in the overlay to define the boot_info area or the retained_ram area I (accidently) reduced the ram size. When using both these systems on different boards this is something to be carefull about, I am normally not using the nrf52840dk so I was copying the overlay from qemu_cortex_m3.

[nordicjm]

@Laczen how come you have ram of 63k on nr52840dk_nrf52840? This board has 256 of RAM as shown here as well: #59375 (comment)

Because in the overlay to define the boot_info area or the retained_ram area I (accidently) reduced the ram size. When using both these systems on different boards this is something to be carefull about, I am normally not using the nrf52840dk so I was copying the overlay from qemu_cortex_m3.

So your size post can be entirely then discarded then, since neither the "virgin state" nor the "modified state" are valid (note: I provided the overlay, so use of your own overlay doesn't even make sense).

[Laczen]

So your size post can be entirely then discarded then, since neither the "virgin state" nor the "modified state" are valid (note: I provided the overlay, so use of your own overlay doesn't even make sense).

Man oh man, any of my code size comparison between boot_info and boot_retention remain valid as well as the link to the comparison. The comment by @nashif was about the device ram size and has nothing to do with the code size. The comparison provided by you however left out a part as to reduce the code size of boot_retention (I am unsure if this was not intentionally).

I really feel sorry for you that you have so much problems accepting that the boot_retention solution is 2kB larger.

[nordicjm]

The astute observer would also notice that your RAM usage makes no sense, with the configuration file I posted (which would be a configuration used with a bootloader such as MCUboot), one can see that the retained memory driver has no global objects that would contribute to RAM usage: https://github.com/zephyrproject-rtos/zephyr/blob/main/drivers/retained_mem/retained_mem_zephyr_ram.c
And with retention, one would see that the RAM usage would be a single byte for the single bool that the data structure contains: https://github.com/zephyrproject-rtos/zephyr/blob/main/subsys/retention/retention.c

Given that yours shows a 64 byte difference in RAM usage, seems like you aren't being honest with your testing.

[Laczen]

The astute observer would also notice that your RAM usage makes no sense, with the configuration file I posted (which would be a configuration used with a bootloader such as MCUboot), one can see that the retained memory driver has no global objects that would contribute to RAM usage: https://github.com/zephyrproject-rtos/zephyr/blob/main/drivers/retained_mem/retained_mem_zephyr_ram.c And with retention, one would see that the RAM usage would be a single byte for the single bool that the data structure contains: https://github.com/zephyrproject-rtos/zephyr/blob/main/subsys/retention/retention.c

Given that yours shows a 64 byte difference in RAM usage, seems like you aren't being honest with your testing.

The astute observer would just ask: "what is the reason for this difference ?" and the answer would be:
The comparison made between the two systems is as if they where used in a zephyr application with multithreading enabled,
The astute observer could then ask: "But why have you compared it this way ?" and the answer would be:
Because the boot_info solution would completely disappear when used in a bootloader like mcuboot and I considered this comparison inappropiate because it would not be fair to the boot_retention solution

[henrikbrixandersen]

The astute observer would just ask: "what is the reason for this difference ?" and the answer would be:
The comparison made between the two systems is as if they where used in a zephyr application with multithreading enabled,
The astute observer could then ask: "But why have you compared it this way ?" and the answer would be:
Because the boot_info solution would completely disappear when used in a bootloader like mcuboot and I considered this comparison inappropiate because it would not be fair to the boot_retention solution

I don’t understand this last comment.

[Laczen]

The astute observer would just ask: "what is the reason for this difference ?" and the answer would be:
The comparison made between the two systems is as if they where used in a zephyr application with multithreading enabled,
The astute observer could then ask: "But why have you compared it this way ?" and the answer would be:
Because the boot_info solution would completely disappear when used in a bootloader like mcuboot and I considered this comparison inappropiate because it would not be fair to the boot_retention solution

I don’t understand this last comment.

Mcuboot has everything to use a part of RAM to store information for the application. It does not need boot_info for this. So the size of boot_info for mcuboot would be 0. Of course for an application the boot_info subsystem would be required. Comparing the size of boot_retention with 0 (as it would be for mcuboot) didn't seem appropiate, so I compared the two systems for a zephyr app instead of a bootloader.

If wanted I can also add the difference in size for a bootloader.

[thedjnK]

The astute observer would just ask: "what is the reason for this difference ?" and the answer would be:
The comparison made between the two systems is as if they where used in a zephyr application with multithreading enabled,
The astute observer could then ask: "But why have you compared it this way ?" and the answer would be:
Because the boot_info solution would completely disappear when used in a bootloader like mcuboot and I considered this comparison inappropiate because it would not be fair to the boot_retention solution

I don’t understand this last comment.

Mcuboot has everything to use a part of RAM to store information for the application.

No it does not. You're actually talking nonsense and pulling things from the void. If it supports it right now, this entire PR is superfluous to requirements rights?

[Laczen]

Mcuboot has everything to use a part of RAM to store information for the application.

No it does not. You're actually talking nonsense and pulling things from the void. If it supports it right now, this entire PR is superfluous to requirements rights?

@thedjnK It was shown above (#59375 (comment)) how to configure mcuboot to share a part of ram and how data can be added to this.

This does not make the entire PR superfluous at all: there are two parts when sharing information between two applications: the first one is adding the data in a bootloader, the second one is providing secure access to this information for the application. boot_info is about the second part (but it can also be used for the first part): providing secure access to this information by a (multithreading) application.

[Laczen]

@henrikbrixandersen, maybe the explanation was not clear enough. What boot_info provides is a secure access to some area (in ram, flash, eeprom, bbram, ...) that has been written by a bootloader. How the bootloader writes to this area is independent of boot_info, it can use boot_info, but it can also use its own methods if it has these. Mcuboot has a method to share information trough a part of ram, so it does not need to use boot_info to write the data there. So boot_info has no influence on mcuboot size because it isn't used (for sharing trough ram). What boot_info then does is setup this area for the application to use.

This is what makes boot_info a great asset for zephyr, it doesn't matter how the data gets in the area. The bootloader can be zephyr or non-zephyr based. Just point (trough the dts) boot_info to the correct location (in ram, flash, eeprom, bbram) and securely use it.

When it comes to making a comparison in size between boot_info and boot_retention there are two comparisons that can be made: the bootloader size and the application size. The application code size was chosen because this adds something for boot_info.

[nordicjm]

You specifically brought this to the architecture working group with the guise that it would write "to flash or BBRAM" from the bootloader (these were your words, as captured in Carle's notes from the meeting above, therefore you need to add your code to do this to MCUboot, it does not magic itself onto the device out of thin air with no additional code additions, and so far you have failed to demonstrate what the additional flash usage for this would be.

What boot_info provides is a secure access

What's secure about it? How are applications prevented from changing the data on it?

[Laczen]

You specifically brought this to the architecture working group with the guise that it would write "to flash or BBRAM" from the bootloader (these were your words, as captured in Carle's notes from the meeting above, therefore you need to add your code to do this to MCUboot, it does not magic itself onto the device out of thin air with no additional code additions, and so far you have failed to demonstrate what the additional flash usage for this would be.

This is exactly the comparison that was made, when mcuboot uses the boot_info interface it is like a application that uses
boot_info . When using the RAM shared region it does magically insert itself out of thin air: the compiler just translates the defined ram area into the following config definition for mcuboot:

#define MCUBOOT_SHARED_DATA_BASE DT_REG_ADDR(DT_NODELABEL(mcuboot_shared_data))
#define MCUBOOT_SHARED_DATA_SIZE DT_REG_SIZE(DT_NODELABEL(mcuboot_shared_data))

If you would like a comparison for flash or bbram, no problem. Which one would you like to be compared ?

What boot_info provides is a secure access

What's secure about it? How are applications prevented from changing the data on it?

Secure like providing multithreads/userthreads support. If a option read-only would be requested (this should be part of a review) it can be added easily (it would just remove the _set method).

[nordicjm]

If you would like a comparison for flash or bbram, no problem. Which one would you like to be compared ?

RAM, using your boot_info code which this whole PR is about

Secure like providing multithreads/userthreads support

But it's not? 2 threads can call the same functions at the same time, there is no security/safety in this code. And not only that but this code uses the whole parent device be it a block of RAM or the whole BBRAM device, there is no segmenting of data/access and no way to ensure different systems making use of it do not clash or even to know which system should use which addresses of the system. This is as far from secure as you can get

[Laczen]

If you would like a comparison for flash or bbram, no problem. Which one would you like to be compared ?

RAM, using your boot_info code which this whole PR is about

This is exactly the comparison that is made. If you would like to try and reduce the size of boot_retention for mcuboot by disabling some mutexes and things like that accept that for boot_info the mcuboot additional code is 0.

Secure like providing multithreads/userthreads support

But it's not? 2 threads can call the same functions at the same time, there is no security/safety in this code. And not only that but this code uses the whole parent device be it a block of RAM or the whole BBRAM device, there is no segmenting of data/access and no way to ensure different systems making use of it do not clash or even to know which system should use which addresses of the system. This is as far from secure as you can get

Two threads cannot access the same function at the same time, this is prevented by using a semaphore. Tests are carried out in userspace if supported to test any unauthorized access from threads.

If segmenting of the data is required it can also be added, but first this needs to be required.

Regarding clashing of data: the dts needs to correctly describe the boot_info block, just as the boot_retention block, if they are incorrectly setup clashing of data can appear in both systems.

[Laczen]

I have updated the boot_info subsystem to further simplify. It now no longer has a direct RAM backend but instead uses the flash simulator to provide this functionality. Boot_info can now share info trough: bbram, eeprom, flash and ram (trough the flash simulator).

To compare the boot_info subsystem with the retention subsystem I created a branch (https://github.com/Laczen/zephyr/tree/retention_test) that has both the boot_infosubsystem and a test for retention (https://github.com/Laczen/zephyr/tree/retention_test/tests/subsys/retention) that do exactly the same (for RAM sharing of information).

The comparison has the following result:

west build -p -b qemu_cortex_m3 tests/subsys/boot_info

Memory region         Used Size  Region Size  %age Used
           FLASH:       19857 B       256 KB      7.57%
             RAM:       12432 B        63 KB     19.27%
        FlashSim:          1 KB         1 KB    100.00%
        IDT_LIST:          0 GB         2 KB      0.00%

west build -p -b qemu_cortex_m3 tests/subsys/retention

Memory region         Used Size  Region Size  %age Used
           FLASH:       22771 B       256 KB      8.69%
             RAM:       12480 B        63 KB     19.35%
     RetainedMem:          0 GB         1 KB      0.00%
        IDT_LIST:          0 GB         2 KB      0.00%

Or about 2.8kB in favor of boot_info.

[nordicjm]

I have updated my PRs and posted patches for new tests which test my full mcuboot -> application data sharing system, the commit from #61583 is also required.
mcuboot diff: https://pastebin.com/ZM5cAxKu
application (hello world) diff: https://pastebin.com/5RFDfzfg

mcuboot build with retention and additional features disabled for nrf52840dk_nrf52840:

Memory region         Used Size  Region Size  %age Used
           FLASH:       31910 B        48 KB     64.92%
             RAM:       23808 B       255 KB      9.12%
     RetainedMem:          0 GB         1 KB      0.00%
        IDT_LIST:          0 GB         2 KB      0.00%

with it enabled:

Memory region         Used Size  Region Size  %age Used
           FLASH:       33112 B        48 KB     67.37%
             RAM:       23808 B       255 KB      9.12%
     RetainedMem:          0 GB         1 KB      0.00%
        IDT_LIST:          0 GB         2 KB      0.00%

total size increase: +1202 bytes flash

Application without changes (note: includes building with mcuboot support but no retention/shared data support):

Memory region         Used Size  Region Size  %age Used
           FLASH:       20000 B       472 KB      4.14%
             RAM:        5568 B       256 KB      2.12%
        IDT_LIST:          0 GB         2 KB      0.00%

Application with changes and including some printf output to show it is working:

Memory region         Used Size  Region Size  %age Used
           FLASH:       20984 B       472 KB      4.34%
             RAM:        5568 B       255 KB      2.13%
     RetainedMem:          0 GB         1 KB      0.00%
        IDT_LIST:          0 GB         2 KB      0.00

total size increase: +984 bytes flash

Output from running:

*** Booting Zephyr OS build zephyr-v3.4.0-2077-g10f3d87750cd ***
I: Starting bootloader
I: Primary image: magic=good, swap_type=0x4, copy_done=0x1, image_ok=0x1
I: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Boot source: none
I: Image index: 0, Swap type: none
I: Bootloader chainload address offset: 0xc000
I: Jumping to the first image slot
*** Booting Zephyr OS build zephyr-v3.4.0-2666-g9a73c4a4d43d ***
Hello World! nrf52840dk_nrf52840
Mode (1): 3
Signature type (1): 1
Slot (1): 0

I therefore find the above post by laczen to be, as per the previous information posts he has provided, deceitful.

[Laczen]

Proposal #62175 is more versatile and provides the same possibilities.