Linking issue with libmqttc

[hobbes1069]

I've been working on getting all the build dependencies not already in Fedora in my COPR[0] and running into a problem:

 [ 18%] Linking C shared library libneuron-base.so
/usr/bin/cmake -E cmake_link_script CMakeFiles/neuron-base.dir/link.txt --verbose=1
/usr/bin/gcc -fPIC -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1   -m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -Wall -Wextra -g -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1   -m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -Wall -Wextra -g -fsanitize=address   -fsanitize-recover=address -fsanitize-address-use-after-scope   -fno-stack-protector -fno-omit-frame-pointer -fno-var-tracking -Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -Wl,-dT,/builddir/build/BUILD/neuron-2.0.1/.package_note-emqx-neuron-2.0.1-1.fc37.x86_64.ld -shared -Wl,-soname,libneuron-base.so -o libneuron-base.so "CMakeFiles/neuron-base.dir/src/types.c.o" "CMakeFiles/neuron-base.dir/src/base/tag_table.c.o" "CMakeFiles/neuron-base.dir/src/base/neu_tag.c.o" "CMakeFiles/neuron-base.dir/src/base/neu_datatag_table.c.o" "CMakeFiles/neuron-base.dir/src/base/neu_tag_group_config.c.o" "CMakeFiles/neuron-base.dir/src/base/neu_plugin_common.c.o" "CMakeFiles/neuron-base.dir/src/base/neu_data_expr.c.o" "CMakeFiles/neuron-base.dir/src/base/tag_class.c.o" "CMakeFiles/neuron-base.dir/src/connection/tcp_client.c.o" "CMakeFiles/neuron-base.dir/src/connection/tcp_server.c.o" "CMakeFiles/neuron-base.dir/src/connection/connection.c.o" "CMakeFiles/neuron-base.dir/src/connection/mqtt_client_intf.c.o" "CMakeFiles/neuron-base.dir/src/event/event_linux.c.o" "CMakeFiles/neuron-base.dir/src/event/event_unix.c.o" "CMakeFiles/neuron-base.dir/src/utils/atomic_data.c.o" "CMakeFiles/neuron-base.dir/src/utils/idhash.c.o" "CMakeFiles/neuron-base.dir/src/utils/hash_table.c.o" "CMakeFiles/neuron-base.dir/src/utils/panic.c.o" "CMakeFiles/neuron-base.dir/src/utils/log.c.o" "CMakeFiles/neuron-base.dir/src/utils/json.c.o" "CMakeFiles/neuron-base.dir/src/utils/neu_jwt.c.o" "CMakeFiles/neuron-base.dir/src/utils/file.c.o" "CMakeFiles/neuron-base.dir/src/utils/base64.c.o" "CMakeFiles/neuron-base.dir/src/config.c.o" "CMakeFiles/neuron-base.dir/src/connection/mqtt_c_client.c.o" "CMakeFiles/neuron-base.dir/src/parser/neu_json_error.c.o" "CMakeFiles/neuron-base.dir/src/parser/neu_json_fn.c.o" "CMakeFiles/neuron-base.dir/src/parser/neu_json_group_config.c.o" "CMakeFiles/neuron-base.dir/src/parser/neu_json_license.c.o" "CMakeFiles/neuron-base.dir/src/parser/neu_json_log.c.o" "CMakeFiles/neuron-base.dir/src/parser/neu_json_login.c.o" "CMakeFiles/neuron-base.dir/src/parser/neu_json_mqtt.c.o" "CMakeFiles/neuron-base.dir/src/parser/neu_json_node.c.o" "CMakeFiles/neuron-base.dir/src/parser/neu_json_plugin.c.o" "CMakeFiles/neuron-base.dir/src/parser/neu_json_rw.c.o" "CMakeFiles/neuron-base.dir/src/parser/neu_json_tag.c.o" "CMakeFiles/neuron-base.dir/src/parser/neu_json_tty.c.o"   -L/usr/local/lib  -Wl,-rpath,./:/usr/local/lib: -lzlog extern/vector/libvector-static.a extern/libcsptr/libcsptr.a -ljansson -lcrypto -lssl -lmqttc -lyaml -ljwt
/usr/bin/ld: /usr/lib/gcc/x86_64-redhat-linux/12/../../../../lib64/libmqttc.a(mqtt.c.o): warning: relocation against `mqtt_fixed_header_rules' in read-only section `.text'
/usr/bin/ld: /usr/lib/gcc/x86_64-redhat-linux/12/../../../../lib64/libmqttc.a(mqtt.c.o): relocation R_X86_64_PC32 against symbol `mqtt_fixed_header_rules' can not be used when making a shared object; recompile with -fPIC

I added the following to the libmqttc library and verified -fPIC -pie is in the build flags[1] per the recommendation from the hardening page[2] but the error remains.

Any ideas?

[0] https://copr.fedorainfracloud.org/coprs/hobbes1069/IIoT/
[1] https://download.copr.fedorainfracloud.org/results/hobbes1069/IIoT/fedora-rawhide-x86_64/04386803-mqtt-c/builder-live.log.gz
[2] https://fedoraproject.org/wiki/Changes/Harden_All_Packages

[gc87]

Can you tweak the cmake compile command like this?

neuron/install.sh

Line 95 in 59b57e2

compile_source neugates/MQTT-C.git HEAD "${ssl_lib_flag} -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DMQTT_C_OpenSSL_SUPPORT=ON -DMQTT_C_EXAMPLES=OFF"

Added -DCMAKE_POSITION_INDEPENDENT_CODE=ON option to turn on -fPIC

[hobbes1069]

Not sure it made any difference than with me forcing it using CFLAGS. Still getting the error. I may just force libmqttc to build as dynamic, it's generally preferred from a distro point of view anyway, I just have to manage a SOVERSION since upstream doesn't.

[fengzeroz]

Use nm to confirm that libmqttc.a contains the _GLOBAL_OFFSET_TABLE_ flag.

nm libmqttc.a

-fPIC is required when compiling libmqttc.a

[hobbes1069]

Looks like it was a bad cache package so I wasn't getting the latest mqtt-c that I built...

I hope this won't be an issue but jwt in Fedora is built with OpenSSL 3 while I had to use the 1.1 compat package for neuron:
/usr/bin/ld: warning: libcrypto.so.3, needed by /usr/lib/gcc/x86_64-redhat-linux/12/../../../../lib64/libjwt.so, may conflict with libcrypto.so.1.1

Is this a MQTT limitation or just an architecture decision for neuron?

[fengzeroz]

Looks like it was a bad cache package so I wasn't getting the latest mqtt-c that I built...

I hope this won't be an issue but jwt in Fedora is built with OpenSSL 3 while I had to use the 1.1 compat package for neuron: /usr/bin/ld: warning: libcrypto.so.3, needed by /usr/lib/gcc/x86_64-redhat-linux/12/../../../../lib64/libjwt.so, may conflict with libcrypto.so.1.1

Is this a MQTT limitation or just an architecture decision for neuron?

Neuron's mqtt plugin depends on some APIs of openssl1.1, and we will improve it later to support openssl3.