modules: update cmsis-nn, cmsis-dsp, and tflite-micro

[XenuIsWatching]

Related Issue: #56917

CMSIS-NN and CMSIS-DSP have been moved out of CMSIS. This brings them up to date with the latest of those repos.
A dependency of updating cmsis-nn also required tflite-micro to be updated.

New Zephyr Modules have already been created for cmsis-dsp and cmsis-nn found here:
zephyrproject-rtos/cmsis-nn#1
zephyrproject-rtos/cmsis-dsp#1

cmsis also had to be updated to remove the cmsis-nn and cmsis-dsp code within it
zephyrproject-rtos/cmsis#21

tflite-micro has also been updated to the latest head (at the time of the pull) here:
zephyrproject-rtos/tflite-micro#4

This did require updating test cases for tflite-micro as some apis were deprecated or 'moved around'

cmsis-nn and cmsis-dsp now use the cmake-ext with the cmake defined within the zephyr repo

[zephyrbot]

The following west manifest projects have been modified in this Pull Request:

Name	Old Revision	New Revision	Diff
cmsis	zephyrproject-rtos/cmsis@1abf291	zephyrproject-rtos/cmsis@5a00331 (master)	zephyrproject-rtos/cmsis@1abf2913..5a003314
cmsis-dsp	N/A	zephyrproject-rtos/cmsis-dsp@ff7b5fd (zephyr)	N/A
cmsis-nn	N/A	zephyrproject-rtos/cmsis-nn@0c8669d (zephyr)	N/A
tflite-micro	zephyrproject-rtos/tflite-micro@9156d05 (zephyr)	zephyrproject-rtos/tflite-micro@1a34dca (main)	zephyrproject-rtos/tflite-micro@9156d050..1a34dcab

Note: This message is automatically posted and updated by the Manifest GitHub Action.

[XenuIsWatching]

wow.... ok, well... I may have uncovered a likely compiler bug. I noticed this on my local machine when running it, but it shows up here on twister build (20) here.

ccache /opt/toolchains/zephyr-sdk-0.16.0/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc -DARM_MATH_LOOPUNROLL -DKERNEL -DTC_RUNID=90f4392f064e00bf4181a16b857c89d0 -D_ANSI_SOURCE -D_FORTIFY_SOURCE=1 -D__LINUX_ERRNO_EXTENSIONS__ -D__PROGRAM_START -D__ZEPHYR__=1 -I/__w/zephyr/modules/lib/cmsis-dsp/Source/DistanceFunctions -I/__w/zephyr/modules/lib/cmsis-dsp/Source/../Include -I/__w/zephyr/modules/hal/cmsis/CMSIS/Core/Include -I/__w/zephyr/modules/lib/cmsis-dsp/Source/../PrivateInclude -I/__w/zephyr/zephyr/include -I/__w/zephyr/zephyr/twister-out/mps3_an547/tests/subsys/dsp/basicmath/zdsp.basicmath.fpu/zephyr/include/generated -I/__w/zephyr/zephyr/soc/arm/arm/mps3 -I/__w/zephyr/zephyr/lib/libc/newlib/include -I/__w/zephyr/zephyr/subsys/testsuite/include -I/__w/zephyr/zephyr/subsys/testsuite/ztest/include -I/__w/zephyr/zephyr/subsys/dsp/cmsis/public -I/__w/zephyr/modules/lib/cmsis-dsp/Include -I/__w/zephyr/modules/lib/cmsis-dsp/PrivateInclude -fno-strict-aliasing -Werror -Os -imacros /__w/zephyr/zephyr/twister-out/mps3_an547/tests/subsys/dsp/basicmath/zdsp.basicmath.fpu/zephyr/include/generated/autoconf.h -fno-common -g -gdwarf-4 -fdiagnostics-color=always -mcpu=cortex-m55 -mthumb -mabi=aapcs -mfpu=auto -mfloat-abi=hard -mfp16-format=ieee --sysroot=/opt/toolchains/zephyr-sdk-0.16.0/arm-zephyr-eabi/arm-zephyr-eabi -imacros /__w/zephyr/zephyr/include/zephyr/toolchain/zephyr_stdint.h -Wall -Wformat -Wformat-security -Wno-format-zero-length -Wno-pointer-sign -Wpointer-arith -Wexpansion-to-defined -Wno-unused-but-set-variable -Werror=implicit-int -fno-pic -fno-pie -fno-asynchronous-unwind-tables -fno-reorder-functions --param=min-pagesize=0 -fno-defer-pop -fmacro-prefix-map=/__w/zephyr/zephyr/tests/subsys/dsp/basicmath=CMAKE_SOURCE_DIR -fmacro-prefix-map=/__w/zephyr/zephyr=ZEPHYR_BASE -fmacro-prefix-map=/__w/zephyr=WEST_TOPDIR -ffunction-sections -fdata-sections -mcmse -std=c99 -flax-vector-conversions -MD -MT modules/cmsis-dsp/CMakeFiles/..__modules__lib__cmsis-dsp.dir/Source/DistanceFunctions/arm_dtw_distance_f32.c.obj -MF modules/cmsis-dsp/CMakeFiles/..__modules__lib__cmsis-dsp.dir/Source/DistanceFunctions/arm_dtw_distance_f32.c.obj.d -o modules/cmsis-dsp/CMakeFiles/..__modules__lib__cmsis-dsp.dir/Source/DistanceFunctions/arm_dtw_distance_f32.c.obj -c /__w/zephyr/modules/lib/cmsis-dsp/Source/DistanceFunctions/arm_dtw_distance_f32.c
/tmp/ccwUTx6v.s: Assembler messages:
/tmp/ccwUTx6v.s:172: Error: syntax error -- `vcvtne.f32.s32 s15,s15'

[stephanosio]

wow.... ok, well... I may have uncovered a likely compiler bug. I noticed this on my local machine when running it, but it shows up here on twister build (20) here.

It looks you did. Can you please file a bug to the sdk-ng repository? Filed zephyrproject-rtos/sdk-ng#665 to track this.

p.s. It might be related to https://bugs.launchpad.net/gcc-arm-embedded/+bug/1982289 (try without -Os).
p.s. It looks like it is a bug in gas, which has been fixed in bminor/binutils-gdb@656412a

[XenuIsWatching]

Why does the zephyr bot keep adding the the "DNM" tag?

[cfriedt]

Why does the zephyr bot keep adding the the "DNM" tag?

The mobile app actually caches state incorrectly. Currently, the app shows 1 check failing, but all tests are clearly passing on the website. Might have been because Stephanos requested changes months ago - is that compiler issue sorted out / is there a workaround?

Outdated

[cfriedt]

Resisting the urge to immediately merge this. It's been cooking a while and has had plenty of time for review but the new policy says someone from a different org should merge.

[XenuIsWatching]

My only concern is with the branch naming of tflite-micro that happened with the merge here zephyrproject-rtos/tflite-micro#4

It looks like it got merged in to main rather than a new zephyr branch.... but since the west.yml points at a hash. I don't think it matters to much. I don't have the power to force branch points to move around :(

[kartben]

@XenuIsWatching
I am confused as to why this is not caught by CI, but I think this is breaking docs build for me locally, with mentions to something ZEPHYR_CMSIS_DSP_KCONFIG something (see below)

Shouldn't modules/Kconfig have been updated to add a source "modules/cmsis_dsp/Kconfig" entry? (note that adding it does not seem to be quite enough to fix things)

ninja html
[0/2] Generating Devicetree bindings documentation...
[1/2] Running Sphinx HTML build...Running Sphinx v6.2.1
Building Kconfig database...... failed

Traceback (most recent call last):
  File "C:\Users\kartben\zephyrproject\zephyr\scripts\kconfig\kconfiglib.py", line 2188, in _enter_file
    self._readline = self._open(filename, "r").readline
                     ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\kartben\zephyrproject\zephyr\scripts\kconfig\kconfiglib.py", line 3921, in _open
    open(filename, mode, encoding=self._encoding)
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\kartben\\zephyrproject\\zephyr\\'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\kartben\zephyrproject\zephyr\doc\_extensions\zephyr\kconfig\__init__.py", line 117, in kconfig_load
    return kconfiglib.Kconfig(ZEPHYR_BASE / "Kconfig"), module_paths
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\kartben\zephyrproject\zephyr\scripts\kconfig\kconfiglib.py", line 947, in __init__
    self._init(filename, warn, warn_to_stderr, encoding)
  File "C:\Users\kartben\zephyrproject\zephyr\scripts\kconfig\kconfiglib.py", line 1085, in _init
    self._parse_block(None, self.top_node, self.top_node).next = None
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\kartben\zephyrproject\zephyr\scripts\kconfig\kconfiglib.py", line 2988, in _parse_block
    prev = self._parse_block(None, parent, prev)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\kartben\zephyrproject\zephyr\scripts\kconfig\kconfiglib.py", line 3027, in _parse_block
    self._parse_block(_T_ENDMENU, node, node)
  File "C:\Users\kartben\zephyrproject\zephyr\scripts\kconfig\kconfiglib.py", line 2988, in _parse_block
    prev = self._parse_block(None, parent, prev)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\kartben\zephyrproject\zephyr\scripts\kconfig\kconfiglib.py", line 2988, in _parse_block
    prev = self._parse_block(None, parent, prev)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\kartben\zephyrproject\zephyr\scripts\kconfig\kconfiglib.py", line 3027, in _parse_block
    self._parse_block(_T_ENDMENU, node, node)
  File "C:\Users\kartben\zephyrproject\zephyr\scripts\kconfig\kconfiglib.py", line 2987, in _parse_block
    self._enter_file(filename)
  File "C:\Users\kartben\zephyrproject\zephyr\scripts\kconfig\kconfiglib.py", line 2191, in _enter_file
    raise _KconfigIOError(
kconfiglib._KconfigIOError: C:\Users\kartben\AppData\Local\Temp\tmp96ltv01q/Kconfig.modules:26: Could not open 'C:\Users\kartben\zephyrproject\zephyr\' (in 'osource "$(ZEPHYR_CMSIS_DSP_KCONFIG)"') (ENOENT: No such file or directory)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python311\Lib\shutil.py", line 620, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\kartben\\AppData\\Local\\Temp\\tmp96ltv01q\\Kconfig.modules'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python311\Lib\tempfile.py", line 878, in onerror
    _os.unlink(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\kartben\\AppData\\Local\\Temp\\tmp96ltv01q\\Kconfig.modules'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\kartben\zephyrproject\.venv\Lib\site-packages\sphinx\events.py", line 96, in emit
    results.append(listener.handler(self.app, *args))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\kartben\zephyrproject\zephyr\doc\_extensions\zephyr\kconfig\__init__.py", line 247, in kconfig_build_resources
    kconfig, module_paths = kconfig_load(app)
                            ^^^^^^^^^^^^^^^^^
  File "C:\Users\kartben\zephyrproject\zephyr\doc\_extensions\zephyr\kconfig\__init__.py", line 69, in kconfig_load
    with TemporaryDirectory() as td:
  File "C:\Python311\Lib\tempfile.py", line 904, in __exit__
    self.cleanup()
  File "C:\Python311\Lib\tempfile.py", line 908, in cleanup
    self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
  File "C:\Python311\Lib\tempfile.py", line 890, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\Python311\Lib\shutil.py", line 759, in rmtree
    return _rmtree_unsafe(path, onerror)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\shutil.py", line 622, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "C:\Python311\Lib\tempfile.py", line 881, in onerror
    cls._rmtree(path, ignore_errors=ignore_errors)
  File "C:\Python311\Lib\tempfile.py", line 890, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\Python311\Lib\shutil.py", line 759, in rmtree
    return _rmtree_unsafe(path, onerror)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\shutil.py", line 603, in _rmtree_unsafe
    onerror(os.scandir, path, sys.exc_info())
  File "C:\Python311\Lib\shutil.py", line 600, in _rmtree_unsafe
    with os.scandir(path) as scandir_it:
         ^^^^^^^^^^^^^^^^
NotADirectoryError: [WinError 267] The directory name is invalid: 'C:\\Users\\kartben\\AppData\\Local\\Temp\\tmp96ltv01q\\Kconfig.modules'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\kartben\zephyrproject\.venv\Lib\site-packages\sphinx\cmd\build.py", line 280, in build_main
    app = Sphinx(args.sourcedir, args.confdir, args.outputdir,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\kartben\zephyrproject\.venv\Lib\site-packages\sphinx\application.py", line 268, in __init__
    self._init_builder()
  File "C:\Users\kartben\zephyrproject\.venv\Lib\site-packages\sphinx\application.py", line 341, in _init_builder
    self.events.emit('builder-inited')
  File "C:\Users\kartben\zephyrproject\.venv\Lib\site-packages\sphinx\events.py", line 107, in emit
    raise ExtensionError(__("Handler %r for event %r threw an exception") %
sphinx.errors.ExtensionError: Handler <function kconfig_build_resources at 0x000001C115A82F20> for event 'builder-inited' threw an exception (exception: [WinError 267] The directory name is invalid: 'C:\\Users\\kartben\\AppData\\Local\\Temp\\tmp96ltv01q\\Kconfig.modules')

Extension error (zephyr.kconfig):
Handler <function kconfig_build_resources at 0x000001C115A82F20> for event 'builder-inited' threw an exception (exception: [WinError 267] The directory name is invalid: 'C:\\Users\\kartben\\AppData\\Local\\Temp\\tmp96ltv01q\\Kconfig.modules')

FAILED: CMakeFiles/html C:/Users/kartben/zephyrproject/zephyr/doc/_build/CMakeFiles/html
cmd.exe /C "cd /D C:\Users\kartben\zephyrproject\zephyr\doc\_build && "C:\Program Files\CMake\bin\cmake.exe" -E env "DOXYGEN_EXECUTABLE=C:/Program Files/doxygen/bin/doxygen.exe" DOT_EXECUTABLE=C:/ProgramData/chocolatey/bin/dot.exe C:/Users/kartben/zephyrproject/.venv/Scripts/sphinx-build.exe -b html -c C:/Users/kartben/zephyrproject/zephyr/doc -d C:/Users/kartben/zephyrproject/zephyr/doc/_build/doctrees -w C:/Users/kartben/zephyrproject/zephyr/doc/_build/html.log -t development -j auto -W --keep-going -T C:/Users/kartben/zephyrproject/zephyr/doc/_build/src C:/Users/kartben/zephyrproject/zephyr/doc/_build/html"
ninja: build stopped: subcommand failed.

[kartben]

I am definitely tempted to say it's probably a bit of a PEBKAC too, as CI is not catching it, but OTH I can reproduce on three different workspaces / OSes. It looks like kconfiglib is getting confused. Again, modules/Kconfig doesn't sound quite right to me (why would it not need updating after this PR, since now it only includes modules/cmsis/Kconfig but not cmsis_dsp and cmsis_nn?)

[XenuIsWatching]

I looked in to this. This is just a folder naming issue. cmsis_nn and cmsis_dsp need to be renamed to cmsis-nn and cmsis-dsp. I push up a PR soon

[XenuIsWatching]

@kartben Please take a look at #63120