Feat/add unreal engine 5.3

[Thogsit]

This Pull Request attempts to make the necessary changes for running the project in Unreal Engine 5.3. The initial motivation came from the discussion in another PR and in the Airsim project here due to experiencing the same error.

Switching to UE5.3 doesn't seem to fix this though. Even after modifying the code to obey the new UE5.3 rules, it prints the following build log including the exact same error as before:

Setting up bundled DotNet SDK
Running command : dotnet Engine/Binaries/DotNET/UnrealBuildTool/UnrealBuildTool.dll Development Linux -Project=<redacted>/fsds/UE4Project 5.3/FSOnline.uproject -TargetType=Editor -Progress -NoHotReloadFromIDE
Log file: <redacted>/unreal-engine-5.3/Engine/Programs/UnrealBuildTool/Log.txt
DEPRECATION: 'IniKeyBlacklist', which is a config key in section [/Script/UnrealEd.ProjectPackagingSettings], in '<redacted>/fsds/UE4Project 5.3/Config/DefaultGame.ini', has been deprecated. Using 'IniKeyDenylist' instead. It is recommended you update your .ini files as soon as possible, and replace IniKeyBlacklist with IniKeyDenylist
Using 'git status' to determine working set for adaptive non-unity build (<redacted>/unreal-engine-5.3).
Using 'git status' to determine working set for adaptive non-unity build (<redacted>/fsds).
@progress push 5%
@progress 'Generating code...' 0%
@progress 'Generating code...' 67%
@progress 'Generating code...' 100%
@progress pop
Building BlocksEditor...
[Upgrade]
[Upgrade] Using backward-compatible build settings. The latest version of UE sets the following values by default, which may require code changes:
[Upgrade]     bLegacyParentIncludePaths = false               => Omits module parent folders from include paths to reduce compiler command line length. (Previously: true).
[Upgrade]     CppStandard = CppStandardVersion.Default        => Updates C++ Standard to C++20 (Previously: CppStandardVersion.Cpp17).
[Upgrade]     WindowsPlatform.bStrictConformanceMode = true   => Updates MSVC strict conformance mode to true (Previously: false).
[Upgrade] Suppress this message by setting 'DefaultBuildSettings = BuildSettingsVersion.V4;' in BlocksEditor.Target.cs, and explicitly overriding settings that differ from the new defaults.
[Upgrade]
[Upgrade]
[Upgrade] Using backward-compatible include order. The latest version of UE has changed the order of includes, which may require code changes. The current setting is:
[Upgrade]     IncludeOrderVersion = EngineIncludeOrderVersion.Unreal5_0
[Upgrade] Suppress this message by setting 'IncludeOrderVersion = EngineIncludeOrderVersion.Latest;' in BlocksEditor.Target.cs.
[Upgrade] Alternatively you can set this to 'EngineIncludeOrderVersion.Latest' to always use the latest include order. This will potentially cause compile errors when integrating new versions of the engine.
[Upgrade]
Determining max actions to execute in parallel (16 physical cores, 16 logical cores)
  Executing up to 16 processes, one per physical core
@progress 'Compiling C++ source code...' 0%
------ Building 3 action(s) started ------
@progress 'Compiling C++ source code...' 33%
[1/3] Link (lld) libUnrealEditor-AirSim.so
ld.lld: error: undefined symbol: pthread_cond_clockwait
>>> referenced by condition_variable.h:227 (/usr/bin/../include/c++/v1/__condition_variable/condition_variable.h:227)
>>>               client.cc.o:(std::__1::condition_variable::__do_timed_wait[abi:ue170006](std::__1::unique_lock<std::__1::mutex>&, std::__1::chrono::time_point<std::__1::chrono::steady_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l>>>)) in archive <redacted>/fsds/UE4Project 5.3/Plugins/AirSim/Source/AirLib/deps/rpclib/lib/librpc.a
ld.lld: error: undefined symbol: UPhysicalMaterial::GetPrivateStaticClass()
>>> referenced by PhysicalMaterial.h:61 (Runtime/PhysicsCore/Public/PhysicalMaterials/PhysicalMaterial.h:61)
>>>               <redacted>/fsds/UE4Project 5.3/Plugins/AirSim/Intermediate/Build/Linux/x64/UnrealEditor/Development/AirSim/Module.AirSim.cpp.o:(ACarPawn::ACarPawn())
>>> referenced by PhysicalMaterial.h:61 (Runtime/PhysicsCore/Public/PhysicalMaterials/PhysicalMaterial.h:61)
>>>               <redacted>/fsds/UE4Project 5.3/Plugins/AirSim/Intermediate/Build/Linux/x64/UnrealEditor/Development/AirSim/Module.AirSim.cpp.o:(ACarPawn::ACarPawn())
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
Total time in Parallel executor: 0.19 seconds
Total execution time: 1.56 seconds

Opening the project in UE5.3 is as such currently at least from my Linux machine not possible so it couldn't be tested; it is expected to not work as is, but maybe serve as a base to do the last few changes necessary so this PR is currently marked as a draft.

Also, I'm completely new to UE and 3D programming so make sure to check my changes twice :). The most interesting part is the CarPawnSimApi.cpp where indeed the logic needed to be changed due to UE's switch from PhysX to Chaos.
In all changes I mainly tried to find the corresponding "new" UE5 function and replace with it; the compiler not crying anymore doesn't mean it actually works though.

EDIT: Most of the changes were taken from here and adjusted as needed.

[Thogsit]

@wouter-heerwegh Any idea on how to circumvent this clang error on Linux currently? Another peer of our group works on Windows and the master branch opens there correctly; this branch will be checked on Windows soon too, but I'd highly prefer to stay on Linux.

[wouter-heerwegh]

Hi @Thogsit

Thanks for this PR! This is a big help. I'll try to see if I can package it for Windows and Linux as well.

When it comes to the compile error, the only thing I've been able to find is that the pthread_cond_clockwait symbol does not show up when compiling Airsim with clang-8 for some reason. I haven't exactly figured out why, but I was hoping that UE5+ would use a cross compiler that would use a newer version of glibc.

So possible ways to fix it are:

• build Airsim with clang-8. This was easy in the past by using the llvm.sh script (see the setup.sh script from Airsim), but llvm 8 is no longer mentioned in the script. Manually adding it also doesn't work, at least not for Ubuntu 22.04, because there is no apt package for it. The only way I see you using clang-8 is by compiling it from source, or manually downloading an executable from one of the releases of the github. I haven't tested this (mainly because I'm still on Ubuntu 20.04).
• Another way is to open a docker container, and build the Airsim library in Ubuntu 18.04, this worked surprisingly well for me.
• the properway to fix it would be to use the cross compiler tool from UE5 to compile Airsim. This would compile the library as if it would be built for centos-7. This would also fix further future incompatibilities.

I've left the Airsim script to use clang-12, so people who want to use the sim, and not use UE can build Airsim and use the executables, because most are on Ubuntu 22.04.

[Thogsit]

FSDS is a really nice project, glad that we can help!

I'll try to see if I can package it for Windows and Linux as well.

We're currently testing the changes under Windows to circumvent the clang issue on my machine. I'll ping here when we know better whether the migration not only compiles, but indeed seems to work somehow - until then I'll leave this in draft mode :).

Thank you very much for the help regarding clang! I'm currently occupied with another topic to work on but will have a look at it again in a few weeks and try to fix it.

[wouter-heerwegh]

I've been looking a bit into the Airsim compilation, where I could find time. We should use these toolchains: https://dev.epicgames.com/documentation/en-us/unreal-engine/linux-development-requirements-for-unreal-engine?application_version=5.3#versionhistory

They provide a toolchain for arm64 (aarch64) and x86 that we could try to use. If we're missing libraries, we'll probably have to try and create our own with crosstool-ng. I've not completely understood however to properly use the toolchains. As far as I understand, do the following:

• Add the path to the toolchain/bin folder to PATH
• export CROSS={$YOUR_TARGET}- , so in my case export CROSS=x86_64-unknown-linux-gnu-
• Modify the build script of AirSim to contain export CC=clang and export=C++=clang++, this should use clang from the cross compile toolchain
• Pass the proper include directories and libraries, this step I've not fully understood how to do. I haven't fully gone through these docs, it's probably in there.