External C++ library undefined behavior

[shalauko]

I want to use an external C++ library linked to a Zephyr application as a static library. For a background - an external library is AgIsoStack

I have a problem with including it in the binary. Building goes well, but as soon as the application refers to any classes defined in the external library, the FW crashes.

To simplify the situation, I reproduced the issue on a smaller example.

Testing platform: NUCLEO-F446RE
Zephyr SDK version: 0.16.8
C and CXX compilers version: 12.2.0 both
Host: Ubuntu 22.04

Directory structure (In the real application, the external library is outside of app repo):

├── app
│   ├── CMakeLists.txt
│   ├── prj.conf
│   └── src
│       └── main.cpp
├── lib
│   └── mathoperations
│       ├── CMakeLists.txt
│       ├── include
│       │   └── MathOperations.hpp
│       └── src
│           └── MathOperations.cpp
└── west.yml

west.yml:

manifest:
  defaults:
    remote: zephyrproject

  remotes:
    - name: zephyrproject
      url-base: https://github.com/zephyrproject-rtos

  projects:
    - name: zephyr
      revision: v3.7.1
      west-commands: scripts/west-commands.yml
      remote: zephyrproject
      import:
        name-allowlist:
          - hal_stm32
          - cmsis

prj.conf:

CONFIG_CPP=y
CONFIG_REQUIRES_FULL_LIBCPP=y
CONFIG_CPP_EXCEPTIONS=y
CONFIG_NEWLIB_LIBC=y

CMakeLists.txt in app:

cmake_minimum_required(VERSION 3.20.0)
find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
project(linkage-example)
target_sources(app PRIVATE src/main.cpp)
add_subdirectory(../lib/mathoperations mathoperations)
target_link_libraries(app PRIVATE MathOperations)

CMakeLists.txt in mathoperations:

cmake_minimum_required(VERSION 3.10)
project(MathOperationsLibrary)

set(INCLUDE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/include)
set(SRC_DIR ${CMAKE_CURRENT_SOURCE_DIR}/src)
set(SOURCES ${SRC_DIR}/MathOperations.cpp)
add_library(MathOperations STATIC ${SOURCES})

target_include_directories(MathOperations PUBLIC ${INCLUDE_DIR})
target_include_directories(MathOperations
    PUBLIC
        $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
        $<INSTALL_INTERFACE:include>
)

MathOperations.hpp:

namespace MathLib {
class MathOperations {
public:
    int add(int a, int b);
    int subtract(int a, int b);
};
} 

MathOperations.cpp:

#include "MathOperations.hpp"

namespace MathLib {
int MathOperations::add(int a, int b) {
    return a + b;
}
int MathOperations::subtract(int a, int b) {
    return a - b;
}
} 

main.cpp:

#include <stdio.h>
#include <zephyr/kernel.h>
#include "MathOperations.hpp"

int main(void)
{
	printf("APP STARTED\n");
	auto operations = new MathLib::MathOperations();
	int a = operations->add(1, 2);
	printf("a = %d\n", a);
	while (1) {
		static int counter = 0;
		printf("COUNTER: %d\n", counter++);
		k_msleep(1000);
	}
	return 0;
}

Problem. The result of adding is not printed. Debugger stuck after the line operations->add(1, 2);.

In the real library case, I got BUS FAULT during object allocation:

[00:00:00.005,000] <err> os: ***** BUS FAULT *****
[00:00:00.005,000] <err> os:   Precise data bus error
[00:00:00.005,000] <err> os:   BFAR Address: 0x28118163
[00:00:00.005,000] <err> os: r0/a1:  0x28118163  r1/a2:  0x00000001  r2/a3:  0x00000002
[00:00:00.005,000] <err> os: r3/a4:  0xff381821 r12/ip:  0x00000005 r14/lr:  0x08001645
[00:00:00.005,000] <err> os:  xpsr:  0x01010000
[00:00:00.005,000] <err> os: Faulting instruction address (r15/pc): 0x0800107c
[00:00:00.005,000] <err> os: >>> ZEPHYR FATAL ERROR 25: Unknown error on CPU 0
[00:00:00.005,000] <err> os: Current thread: 0x20000f90 (unknown)
[00:00:00.070,000] <err> os: Halting system

I guess there is some linkage issue, which is not seen by linker. Looks like linker puts wrong addresses in the memory. Symbols defined in the library are present in target zephyr.map file. So, linker linked it, but looks like it links to the wrong addresses.

As a limitation. I can't get rid of the CMakeLists.txt defined in the library, I can modify my CMakeLists.txt in the app directory. Optionally, I can write some wrapper/adapter to the lib if needed.