applications: serial_lte_modem: Full modem fota support for SLM #11892

VTPeltoketo · 2023-07-27T07:37:50Z

- new user interface parameter for #XFOTA AT command
- full modem fota is using an external flash

NordicBuilder · 2023-07-27T07:40:21Z

Test specification

CI/Jenkins/NRF

Integration Platforms

CI/Jenkins/integration

Test Module	File based changes	Manually selected	West overwrite
test-fw-nrfconnect-nrf-iot_serial_lte_modem	X

Detailed information of selected test modules

Note: This message is automatically posted and updated by the CI

NordicBuilder · 2023-07-27T08:15:25Z

You can find the documentation preview for this PR at this link. It will be updated about 10 minutes after the documentation build succeeds.

Note: This comment is automatically posted by the Documentation Publishing GitHub Action.

applications/serial_lte_modem/prj.conf

VTPeltoketo · 2023-08-10T06:00:08Z

This PR is now ready to review.

applications/serial_lte_modem/Kconfig

applications/serial_lte_modem/doc/FOTA_AT_commands.rst

applications/serial_lte_modem/boards/nrf9160dk_nrf9160_ns.overlay

applications/serial_lte_modem/src/slm_at_fota.c

MarkusLassila · 2023-08-10T11:31:16Z

I was also kind of waiting for there to be an actual overlay file. Were there no other Kconfigs that needed to be enabled?

applications/serial_lte_modem/Kconfig

applications/serial_lte_modem/doc/FOTA_AT_commands.rst

applications/serial_lte_modem/overlay-full_fota.conf

applications/serial_lte_modem/src/slm_at_fota.c

tomi-font · 2023-08-22T10:19:10Z

applications/serial_lte_modem/src/slm_at_fota.c

+
+ err = nrf_modem_lib_bootloader_init();
+ if (err != 0) {
+ LOG_ERR("nrf_modem_lib_bootloader_init() failed: %d\n", err);


Maybe nrf_modem_lib_init() in that case to still try to exit from this function with a running modem?
And I think the same kind of fine tuning can be done to some of the other failure cases below.

Though I agree with your comment above, about the trickiness of this. And certain errors are likely not even possible during a modem firmware update.

Would it be better to reset the device if something goes wrong after nrf_modem_lib_shutdown? If something is truly wrong in the sequence, nrf_modem_lib_init() may not be successful either.

If we make reset here, it has to delayed one because we have to return error event. Sounds quite complicated

What error event?

@VTPeltoketo

After FOTA and reset/modemreset, slm_fota_post_process() function is called. If the FOTA activation fails, slm_fota_post_process function sends an error AT event.

I see. slm_fota_post_process() is called later in the boot process and only depends on slm_fota_stage not being FOTA_STAGE_INIT to print the FOTA result. So it seems that @MarkusLassila's proposal to reset the device would be fine as the FOTA result would end up getting printed after the additional reset. Though one potential risk I see is entering into a reset loop, so it would be preferable to not attempt to do this more than once.
One scenario I can imagine working in these hypothetical failure cases is:

Set slm_fota_stage to FOTA_STAGE_COMPLETE, with the other FOTA variables indicating an error.

Save the FOTA settings.

Reset the device.

Avoid calling slm_finish_modem_fota() again.

Reach slm_fota_post_process() that will print the FOTA result. If for some reason the code fails before this, it means that something is terribly wrong and there is nothing we can do anymore as a reboot didn't help.

Step 4 would require changing main.c:445, probably to if (slm_fota_stage == FOTA_STAGE_ACTIVATE).
This is very hypothetical, but adds a safety net in case one of these calls in slm_finish_modem_fota() would fail in a non-fatal way. Otherwise it's not impossible that the code would return from the function without a running modem. So doing something about it would allow SLM to kind of self-heal.

What do you think?

Yes, this might work. I have to simulate a finalization error and test. There are actually two different use cases to test, modem reset and device reset.

Error handling of the full FOTA activation has been changed.

SeppoTakalo

LGTM.
Please fix the compliance issues.

applications/serial_lte_modem/Kconfig

applications/serial_lte_modem/overlay-full_fota.conf

applications/serial_lte_modem/src/slm_at_fota.c

applications/serial_lte_modem/Kconfig

applications/serial_lte_modem/doc/FOTA_AT_commands.rst

MarkusLassila · 2023-09-07T10:01:16Z

applications/serial_lte_modem/src/slm_at_fota.c

+
+ err = nrf_modem_lib_bootloader_init();
+ if (err != 0) {
+ LOG_ERR("nrf_modem_lib_bootloader_init() failed: %d\n", err);


Would it be better to reset the device if something goes wrong after nrf_modem_lib_shutdown? If something is truly wrong in the sequence, nrf_modem_lib_init() may not be successful either.

applications/serial_lte_modem/boards/nrf9161dk_nrf9161_ns.overlay

applications/serial_lte_modem/doc/FOTA_AT_commands.rst

applications/serial_lte_modem/boards/nrf9160dk_nrf9160_ns.overlay

applications/serial_lte_modem/doc/FOTA_AT_commands.rst

applications/serial_lte_modem/overlay-full_fota.conf

applications/serial_lte_modem/src/slm_at_fota.c

tomi-font · 2023-09-11T11:01:47Z

applications/serial_lte_modem/src/slm_at_fota.c

+
+ err = nrf_modem_lib_bootloader_init();
+ if (err != 0) {
+ LOG_ERR("nrf_modem_lib_bootloader_init() failed: %d\n", err);


What error event?

doc/nrf/releases_and_maturity/releases/release-notes-changelog.rst

applications/serial_lte_modem/overlay-full_fota.conf

applications/serial_lte_modem/doc/FOTA_AT_commands.rst

wiba-nordic

Docs look good.

melwee01 · 2023-09-13T07:49:14Z

applications/serial_lte_modem/doc/FOTA_AT_commands.rst

+ The external flash is ereased automatically after a new firmware activation.
+
+ The activation of a new full modem firmware is done identically to a modem delta update, by resetting either the whole device or only the modem.


'Activation' sounds weird to me here, but I can't find a clear precedent on the Cloud side.

My gut says application is a better word based on https://docs.nrfcloud.com/Devices/FirmwareUpdate/FOTAOverview.html, but I've also realized we don't ever actually say what this step is called. The entire process is deployment, and in some APIs the act of installing new firmware is called application, but that's still more general than what this doc is referring to.

Switching 'activation' and 'application' would be confusing. Here 'activation' means the action to be made and 'application' the software itself which is downloaded.

melwee01 · 2023-09-13T07:54:45Z

applications/serial_lte_modem/doc/FOTA_AT_commands.rst

@@ -47,10 +49,17 @@ Syntax

 .. note::

- When doing modem FOTA, erasing the modem DFU area is optional since the update process will automatically erase the area if needed.
+ When doing modem delta FOTA, erasing the modem DFU area is optional since the update process will automatically erase the area if needed.


Suggested change

When doing modem delta FOTA, erasing the modem DFU area is optional since the update process will automatically erase the area if needed.

During a delta update to the modem over FOTA, the modem DFU area is automatically erased as needed as part of the update process. Erasing this area manually is optional.

Trying to make the structure a little clearer. I hope I understood this correctly.

melwee01 · 2023-09-13T07:55:33Z

applications/serial_lte_modem/doc/FOTA_AT_commands.rst


 However, this leads to the command starting the update taking longer to complete, and also leaves the connection to the FOTA server idle while the area is being erased, which can provoke issues.

+.. note::
+
+ When doing full modem FOTA, external flash is used to store the firmware.


Suggested change

When doing full modem FOTA, external flash is used to store the firmware.

The firmware image is stored in external flash memory during a full modem update over FOTA.

modem update over Firmware Over The Air

Fixed, but the FOTA appreviation has been used.

@melwee01 over FOTA sounds really wrong to me. But I'll leave this to the documentation people.

Perhaps 'using FOTA', then.

Fixed according to the teams chat

tomi-font

Looks good to me. Just a last little thing left.

tomi-font · 2023-09-14T12:38:19Z

applications/serial_lte_modem/src/slm_at_commands.c

@@ -230,7 +230,8 @@ static int handle_at_modemreset(enum at_cmd_type type)
 ++step;

 if (ret > 0 || (slm_fota_stage != FOTA_STAGE_INIT


Now that the condition is slm_fota_stage == FOTA_STAGE_ACTIVATE when doing an application reset, I suggest having the same here. Better yet, make a function that is called by those two places to make sure that they remain in sync.

Yes they can be in sync even if it does not affect to the execution. I will change it

tomi-font · 2023-09-14T12:41:43Z

applications/serial_lte_modem/doc/FOTA_AT_commands.rst


 However, this leads to the command starting the update taking longer to complete, and also leaves the connection to the FOTA server idle while the area is being erased, which can provoke issues.

+.. note::
+
+ When doing full modem FOTA, external flash is used to store the firmware.


@melwee01 over FOTA sounds really wrong to me. But I'll leave this to the documentation people.

MarkusLassila

One finding.

applications/serial_lte_modem/src/slm_at_fota.c

melwee01 · 2023-09-18T06:27:03Z

applications/serial_lte_modem/doc/FOTA_AT_commands.rst

+.. note::
+
+ The firmware image is stored in external flash memory during a full modem update over FOTA.
+ The external flash is ereased automatically after a new firmware activation.


Suggested change

The external flash is ereased automatically after a new firmware activation.

The external flash is erased automatically after a new firmware activation.

melwee01 · 2023-09-18T06:34:58Z

applications/serial_lte_modem/doc/FOTA_AT_commands.rst

+ The firmware image is stored in external flash memory during a full modem update over FOTA.
+ The external flash is ereased automatically after a new firmware activation.
+
+ The activation of a new full modem firmware is done identically to a modem delta update, by resetting either the whole device or only the modem.


Suggested change

The activation of a new full modem firmware is done identically to a modem delta update, by resetting either the whole device or only the modem.

Activating the new full modem firmware is done identically to a modem delta update, by resetting either the whole device or only the modem.

- new user interface parameter for #XFOTA AT command - full modem fota is using an external flash - full fota is activated via overlay Signed-off-by: Veli-Tapani Peltoketo <veli-tapani.peltoketo@nordicsemi.no>

VTPeltoketo requested a review from junqingzou July 27, 2023 07:37

VTPeltoketo requested review from SeppoTakalo, MarkusLassila, rlubos and tomi-font as code owners July 27, 2023 07:37

github-actions bot added changelog-entry-required Update changelog before merge. Remove label if entry is not needed or already added. doc-required PR must not be merged without tech writer approval. labels Jul 27, 2023

SeppoTakalo reviewed Jul 27, 2023

View reviewed changes

applications/serial_lte_modem/prj.conf Outdated Show resolved Hide resolved

VTPeltoketo force-pushed the NRF91-1741-slm-full-modem-fota-support-2 branch from dd659d6 to f8ed3aa Compare July 28, 2023 11:05

VTPeltoketo changed the title ~~applications: serial_lte_modem: Full modem fota support for SLM~~ DRAFT applications: serial_lte_modem: Full modem fota support for SLM Aug 7, 2023

VTPeltoketo force-pushed the NRF91-1741-slm-full-modem-fota-support-2 branch from f8ed3aa to 742623b Compare August 7, 2023 07:53

VTPeltoketo requested a review from carlescufi as a code owner August 7, 2023 09:43

github-actions bot removed the changelog-entry-required Update changelog before merge. Remove label if entry is not needed or already added. label Aug 7, 2023

VTPeltoketo requested a review from tejlmand as a code owner August 9, 2023 10:56

VTPeltoketo force-pushed the NRF91-1741-slm-full-modem-fota-support-2 branch 2 times, most recently from ab8da56 to 332524f Compare August 9, 2023 12:14

VTPeltoketo requested a review from b-gent August 9, 2023 12:40

VTPeltoketo changed the title ~~DRAFT applications: serial_lte_modem: Full modem fota support for SLM~~ applications: serial_lte_modem: Full modem fota support for SLM Aug 9, 2023

MarkusLassila requested changes Aug 10, 2023

View reviewed changes

MarkusLassila reviewed Aug 10, 2023

View reviewed changes

applications/serial_lte_modem/Kconfig Show resolved Hide resolved

tomi-font requested changes Aug 22, 2023

View reviewed changes

VTPeltoketo force-pushed the NRF91-1741-slm-full-modem-fota-support-2 branch 2 times, most recently from 5beb85e to c80d1f9 Compare September 6, 2023 12:19

SeppoTakalo approved these changes Sep 7, 2023

View reviewed changes

applications/serial_lte_modem/Kconfig Outdated Show resolved Hide resolved

VTPeltoketo force-pushed the NRF91-1741-slm-full-modem-fota-support-2 branch from f9bfa96 to 14763d9 Compare September 7, 2023 11:29

MarkusLassila reviewed Sep 8, 2023

View reviewed changes

VTPeltoketo requested a review from tomi-font September 11, 2023 05:50

MarkusLassila approved these changes Sep 11, 2023

View reviewed changes

tomi-font requested changes Sep 11, 2023

View reviewed changes

b-gent requested review from melwee01 and wiba-nordic September 11, 2023 11:56

wiba-nordic requested changes Sep 11, 2023

View reviewed changes

wiba-nordic reviewed Sep 12, 2023

View reviewed changes

applications/serial_lte_modem/doc/FOTA_AT_commands.rst Outdated Show resolved Hide resolved

applications/serial_lte_modem/doc/FOTA_AT_commands.rst Outdated Show resolved Hide resolved

wiba-nordic requested changes Sep 12, 2023

View reviewed changes

applications/serial_lte_modem/doc/FOTA_AT_commands.rst Outdated Show resolved Hide resolved

wiba-nordic approved these changes Sep 12, 2023

View reviewed changes

VTPeltoketo force-pushed the NRF91-1741-slm-full-modem-fota-support-2 branch 3 times, most recently from e634f71 to a6b365d Compare September 12, 2023 13:13

VTPeltoketo requested a review from tomi-font September 12, 2023 13:14

tomi-font requested a review from MarkusLassila September 13, 2023 07:12

melwee01 reviewed Sep 13, 2023

View reviewed changes

tomi-font reviewed Sep 14, 2023

View reviewed changes

tomi-font approved these changes Sep 15, 2023

View reviewed changes

VTPeltoketo requested a review from melwee01 September 15, 2023 06:11

MarkusLassila approved these changes Sep 15, 2023

View reviewed changes

applications/serial_lte_modem/src/slm_at_fota.c Outdated Show resolved Hide resolved

VTPeltoketo force-pushed the NRF91-1741-slm-full-modem-fota-support-2 branch from 42d6bdf to 8f50b55 Compare September 18, 2023 05:51

melwee01 reviewed Sep 18, 2023

View reviewed changes

applications: serial_lte_modem: Full modem fota support for SLM

1142e90

- new user interface parameter for #XFOTA AT command - full modem fota is using an external flash - full fota is activated via overlay Signed-off-by: Veli-Tapani Peltoketo <veli-tapani.peltoketo@nordicsemi.no>

VTPeltoketo force-pushed the NRF91-1741-slm-full-modem-fota-support-2 branch from 98409ef to 1142e90 Compare September 18, 2023 09:34

VTPeltoketo requested a review from melwee01 September 19, 2023 03:41

melwee01 approved these changes Sep 19, 2023

View reviewed changes

nordicjm merged commit 55f0a73 into nrfconnect:main Sep 19, 2023
13 checks passed

		The external flash is ereased automatically after a new firmware activation.

		The activation of a new full modem firmware is done identically to a modem delta update, by resetting either the whole device or only the modem.

	When doing modem delta FOTA, erasing the modem DFU area is optional since the update process will automatically erase the area if needed.
	During a delta update to the modem over FOTA, the modem DFU area is automatically erased as needed as part of the update process. Erasing this area manually is optional.

	When doing full modem FOTA, external flash is used to store the firmware.
	The firmware image is stored in external flash memory during a full modem update over FOTA.

		@@ -230,7 +230,8 @@ static int handle_at_modemreset(enum at_cmd_type type)
		++step;

		if (ret > 0 \|\| (slm_fota_stage != FOTA_STAGE_INIT

	The external flash is ereased automatically after a new firmware activation.
	The external flash is erased automatically after a new firmware activation.

	The activation of a new full modem firmware is done identically to a modem delta update, by resetting either the whole device or only the modem.
	Activating the new full modem firmware is done identically to a modem delta update, by resetting either the whole device or only the modem.

applications: serial_lte_modem: Full modem fota support for SLM #11892

applications: serial_lte_modem: Full modem fota support for SLM #11892

Conversation

VTPeltoketo commented Jul 27, 2023

NordicBuilder commented Jul 27, 2023 • edited Loading

Test specification

NordicBuilder commented Jul 27, 2023

VTPeltoketo commented Aug 10, 2023

MarkusLassila commented Aug 10, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SeppoTakalo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wiba-nordic left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

VTPeltoketo Sep 18, 2023 • edited Loading

Choose a reason for hiding this comment

tomi-font left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MarkusLassila left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NordicBuilder commented Jul 27, 2023 •

edited

Loading

VTPeltoketo Sep 18, 2023 •

edited

Loading