Feature/dfi cleanup

[PengZheng]

Introduction

This PR is a preliminary work of #590. The main motivation behind this is to familiarize myself of the underlying mechanism of RSA, i.e. libdfi. Reading the code base alone is not enough, it should also be thoroughly tested and debugged. That way not only did I get a full understanding, I also found and fixed several ambiguities and bugs along the way.

Hopefully, after this PR merged, libdfi is robust enough to handle most (if not all) of malformed descriptors and all malicious JSON requests/responses.

Precise Definitions of DFI Argument Types

Dyn function argument meta (am) as meta info, with the following possible values
am=handle #void pointer for the handle
am=pre #output pointer with memory pre-allocated, it should be pointer to trivial types, check dynType_isTrivial for more info.
am=out #output pointer, it should be pointers to text or pointer to serializable point type
Without meta info the argument is considered to be a standard argument, which can be of any serializable type.

Previously, we don't have such "formal" definitions. As mentioned by #723, it is fairly easy to construct "legal" interface descriptor to introduce use-after-free bugs. To address this, we introduce the notion of trivial type and perform strict checking in dynFunction_parse. Note that "serializability" check is left for the jsonSerializer to perform.

RSA Interface Convention Enforcement

• The first parameter of a method must be handle.
• am=handle can appear exactly once.
• If exists, output parameter (either am=pre or am=out) is only allowed as the last one. Therefore, there is at most one output parameter.

We enforce the convention in dynInterface_checkInterface so that arguments handling in json_rpc.c can be greatly simplified.

Please consider adding the above two into #690. @xuzhenbao

Early Return Error Handling Pattern

Previously, error handling is done using chained status check:

    status = do_something();
    if (status == OK) {
        status = do_something_else();
    }
    if (status == OK) {
        status = do_other_thing();
    }

With chained status check, it is relative easy to achieve high line coverage.
However, the branch coverage is still low and the control flow is often difficult to follow.
By applying early return uniformly in libdfi, by archiving high line coverage, we are almost guaranteed high branch coverage and the readability is often improved.

Remove AVRO

It is currently incomplete and unused, so is removed by this PR.
After completing #590, we can reconsider introducing more efficient serialization mechanism (including AVRO).

Other Enhancements and Improvements

1. We can now serialize/deserialize nullptr for pointer type.
2. Double pointer is not serializable. For exmaple, we can not tell from a json null whether int* == NULL or int** == NULL.
3. celix_auto is used extensively. Please note that a caveat related to stack variable declaration order is highlighted in this commit: 876471d The interesting thing is that this issue only manifests itself in clang builds.
4. const qualifier is applied where appropriate.
5. Add dynType_realType to deal with reference type and fix several related crashes.

[codecov-commenter]

Codecov Report

Attention: 10 lines in your changes are missing coverage. Please review.

Comparison is base (29fa7d0) 88.50% compared to head (90d643a) 88.90%.

Files	Patch %	Lines
...te_service_admin_dfi/src/import_registration_dfi.c	52.63%	9 Missing ⚠️
...te_service_admin_dfi/src/export_registration_dfi.c	50.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #699      +/-   ##
==========================================
+ Coverage   88.50%   88.90%   +0.40%     
==========================================
  Files         212      216       +4     
  Lines       24606    24293     -313     
==========================================
- Hits        21777    21598     -179     
+ Misses       2829     2695     -134     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[pnoltes]

I will try to find to time to review this PR, this week.

[pnoltes]

Still reviewing.

Nice to see that libdfi gets some attention and refactoring :)
I really enjoy seeing te early returns and usage of celix_err.

It has been a while since I looked into this code, so reviewing will take some days.
I do remember that I really liked creating libdfi; It is quite a powerful concept, complex to implement and wrap your head around (ffi with a pointer to a argument array, where the argument entries are pointers to the argument values and the argument values can be pointers or even a double output pointers) , but it possible to nice split up the parsing and usage in small functions.

Originally my idea was also to allow optional usage of libdfi in the framework so that it would be possible to (e.g.) use a service struct of version 1.0.0 while the provided service is 1.1.0. With libdfi is should be possible to ad-hoc create a 1.0.0 version based on the 1.0.0 and 1.1.0 descriptor (assuming correct usage of semver). But I digress.

[pnoltes]

Suggested change

	if (!dynType_isTrivial(entry->type)) {
	type->trivial = false;
	type->trivial = dynType_isTrivial(entry->type);

[PengZheng]

The intention here is if ANY of its field is non-trivial, then the complex itself is non-trivial, otherwise it is trivial (as set in line 247).

But with the suggested modification, the triviality of a complex is determined by its last field.

[PengZheng]

It has been a while since I looked into this code, so reviewing will take some days.
I do remember that I really liked creating libdfi; It is quite a powerful concept, complex to implement and wrap your head around (ffi with a pointer to a argument array, where the argument entries are pointers to the argument values and the argument values can be pointers or even a double output pointers) , but it possible to nice split up the parsing and usage in small functions.

Originally my idea was also to allow optional usage of libdfi in the framework so that it would be possible to (e.g.) use a service struct of version 1.0.0 while the provided service is 1.1.0. With libdfi is should be possible to ad-hoc create a 1.0.0 version based on the 1.0.0 and 1.1.0 descriptor (assuming correct usage of semver). But I digress.

Thanks for sharing the background with me. I pretended to be cool, but let me admit it: the libdfi work is eyeopening, which leads to another "WOW, things could really be done like this" awe.

Inspired by the Rust POC, to be more concrete, the build.rs and procedural macro, I think clang/libclang integrated with the build system is worth trying for framework side code generation/transformation. RSA descriptor is an ideal code generation target.

[pnoltes]

LGTM. Nice and needed refactoring libdfi.