new ExprTK-based Expression evaluation blocks #486

RalphSteinhagen · 2024-12-12T13:47:11Z

This PR introduces a set of new expression-based blocks -- ExpressionSISO, ExpressionDISO, and ExpressionBulk -- which use the ExprTK library for dynamically defined mathematical operations on input samples based on string-based expressions at runtime:

ExpressionSISO -- Single-Input-Single-Output (SISO):
evaluates an expressions a*x + b, sin(pi*x), or even recursive forms like y := y + 0.1*x.
ExpressionDISO -- Dual-Input-Single-Output (DISO):
Process two input streams simultaneously, e.g., z := a*(x+y) or z := inrange(-1, x+y, 1) ? (x+y) : 0.
ExpressionBulk -- Bulk (Vector) Processing:
Operate on entire arrays at once, enabling expressions like:
```
vecOut := a * vecIn;
for (i,0,vecIn.size()) vecOut[i] := vecIn[i] + c;
```
This supports loops, indexing, and complex transformations.
Optional runtime checks can detect out-of-range vector indexing and raise exceptions.

For more examples -- especially regarding the possible syntax -- see:

ExprTK Documentation: https://www.partow.net/programming/exprtk
ExprTK GitHub: https://github.com/ArashPartow/exprtk

sqrt(1 - (x^2))
clamp(-1, sin(2 * pi * x) + cos(y / 2 * pi), +1)
sin(2 * x)
if (((x + 2) == 3) and ((y + 5) <= 9), 1 + w, 2 / z)
inrange(-2, m, +2) == (({-2 <= m} and [m <= +2]) ? 1 : 0)
({1 / 1} * [1 / 2] + (1 / 3)) - {1 / 4} ^ [1 / 5] + (1 / 6) -({1 / 7} + [1 / 8] * (1 / 9))
a * exp(2 * t) + c
z := x + sin(2 * pi / y)
2x + 3y + 4z + 5w == 2 * x + 3 * y + 4 * z + 5 * w
3(x + y) / 2 + 1 == 3 * (x + y) / 2 + 1
(x + y)3 + 1 / 4 == (x + y) * 3 + 1 / 4
(x + y)z + 1 / 2 == (x + y) * z + 1 / 2
(sin(x / pi)cos(2y) + 1)==(sin(x / pi) * cos(2 * y) + 1)
while(x <= 100) { x += 1; }

wirew0rm

Very nice, and surprisingly small implementation footprint for such a powerful feature 👍

Could be merged as is as soon as the CI and formatting is fixed.

Since the CI anyway needs another iteration I added some minor comments. I only commented on the first Block implementation but most of them are valid to the other 2 as well. In addition I noticed that the naming of the inputs and outputs in the expression strings is a bit inconsistent between the different implementations now, i wrote up some alternative ideas to make it more consistent, but since consistency isn't the only goal here, the original might still be the best choice.

ExpressionSISO: x     -> y       (proposal: x     -> y | x   -> p | in       -> out)
ExpressionDISO: x,y   -> z       (proposal: x1,x2 -> y | x,y -> p | in1, in2 -> out)
ExpressionBulk: inVec -> outVec  (proposal: X     -> Y | X   -> P | inVec    -> outVec)

wirew0rm · 2024-12-13T13:02:39Z

blocks/math/include/gnuradio-4.0/math/ExpressionBlocks.hpp

+        auto [vecSize, vecIndex] = computeVectorInfo(context.base_ptr, context.end_ptr, typeSize, context.access_ptr);
+        throw gr::exception(fmt::format("vector access '{name}[{index}]' outside of [0, {size}[ (typesize: {typesize})", //
+            fmt::arg("name", vector_name), fmt::arg("size", vecSize), fmt::arg("index", vecIndex), fmt::arg("typesize", typeSize)));
+        return false; // should never reach here


Dead code due to the throw directly above.
For custom state handling logic, I see the argument of having such a "never reached" statement, but here we effectively guard against something the compiler guarantees.

Unless exceptions are disabled...

wirew0rm · 2024-12-13T13:11:28Z

blocks/math/include/gnuradio-4.0/math/ExpressionBlocks.hpp

+    A<T, "a", Doc<"free parameter 'a' for use in expressions">, Visible>                        param_a     = T(1.0);
+    A<T, "b", Doc<"free parameter 'b' for use in expressions">, Visible>                        param_b     = T(0.0);
+    A<T, "c", Doc<"free parameter 'c' for use in expressions">, Visible>                        param_c     = T(0.0);


Consider changing this so all 3 parameters have the same initial value.

wirew0rm · 2024-12-13T13:19:56Z

blocks/math/include/gnuradio-4.0/math/ExpressionBlocks.hpp

+    PortIn<T>  in;
+    PortOut<T> out;
+
+    A<std::string, "expr string", Doc<"for syntax see: https://github.com/ArashPartow/exprtk">> expr_string = "clamp(-1.0, sin(2 * pi * x) + cos(x / 2 * pi), +1.0)";


I would use something more trivial as the default expression, maybe:

Suggested change

A<std::string, "expr string", Doc<"for syntax see: https://github.com/ArashPartow/exprtk">> expr_string = "clamp(-1.0, sin(2 * pi * x) + cos(x / 2 * pi), +1.0)";

A<std::string, "expr string", Doc<"for syntax see: https://github.com/ArashPartow/exprtk">> expr_string = "0";

Just in case the initialization fails, no signal is easier to diagnose than getting some arbitrary signal. We have examples in the docstring and unittests.

wirew0rm · 2024-12-13T13:21:06Z

blocks/math/include/gnuradio-4.0/math/ExpressionBlocks.hpp

+    exprtk::symbol_table<T> _symbol_table;
+    exprtk::expression<T>   _expression;


Suggested change

exprtk::symbol_table<T> _symbol_table;

exprtk::expression<T> _expression;

exprtk::symbol_table<T> _symbol_table{};

exprtk::expression<T> _expression{};

allows to use the parent's constructors.

wirew0rm · 2024-12-13T13:23:51Z

blocks/testing/include/gnuradio-4.0/testing/NullSources.hpp

@@ -35,7 +35,7 @@ Commonly used for testing and simulations where consistent output and finite exe
    Annotated<gr::Size_t, "max samples", Doc<"count>n_samples_max -> signal DONE (0: infinite)">> n_samples_max = 0U;
    Annotated<gr::Size_t, "count", Doc<"sample count (diagnostics only)">>                        count         = 0U;

-    GR_MAKE_REFLECTABLE(ConstantSource, out, n_samples_max, count);
+    GR_MAKE_REFLECTABLE(ConstantSource, out, default_value, n_samples_max, count);


Nice catch 👍

Had to add a small followup for this: since default_value was of type T and the Sinks are instantiated with a T=UncertainValue, which is not supported as a setting type i had to change the type to value_t -> cannot set default values with errors or imaginary parts for now.

qa_FileIO is working again, but for non-trivial types the sources still cause some conversion errors and unexpected results.

wirew0rm · 2024-12-13T13:35:14Z

blocks/math/test/qa_ExpressionBlocks.cpp

+        try {
+            expect(sched.runAndWait().has_value());
+            expect(false) << fmt::format("should have failed");
+        } catch (const gr::exception& ex) {
+            expect(true);
+            fmt::println("failed correctly with:\n{}\n", ex);
+        } catch (...) {
+            expect(false) << fmt::format("caught unknown/unexpected exception");
+        }


Why not use ut's builtin exception matcher?

Suggested change

try {

expect(sched.runAndWait().has_value());

expect(false) << fmt::format("should have failed");

} catch (const gr::exception& ex) {

expect(true);

fmt::println("failed correctly with:\n{}\n", ex);

} catch (...) {

expect(false) << fmt::format("caught unknown/unexpected exception");

}

expect(throws<gr::exception>([] { expect(sched.runAndWait().has_value()); }))

https://github.com/boost-ext/ut/blob/master/README.md?plain=1#L1116

* ExpressionSISO: Single-Input-Single-Output * ExpressionDISO: Dual -Input-Single-Output * ExpressionBulk -> std:span-Input-to-std::span-Output. Signed-off-by: rstein <r.steinhagen@gsi.de> Signed-off-by: Ralph J. Steinhagen <r.steinhagen@gsi.de> Signed-off-by: Alexander Krimm <A.Krimm@gsi.de>

sonarqubecloud · 2024-12-16T17:57:05Z

Quality Gate failed

Failed conditions
62.9% Coverage on New Code (required ≥ 80%)
24.1% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

RalphSteinhagen force-pushed the exprtk_block branch 3 times, most recently from b389099 to 270e6f6 Compare December 13, 2024 07:23

wirew0rm approved these changes Dec 13, 2024

View reviewed changes

wirew0rm force-pushed the exprtk_block branch from ba5c799 to e4f3cf7 Compare December 13, 2024 17:16

RalphSteinhagen force-pushed the exprtk_block branch from e4f3cf7 to d2c1757 Compare December 16, 2024 15:05

RalphSteinhagen force-pushed the exprtk_block branch from d2c1757 to 389d30b Compare December 16, 2024 15:13

RalphSteinhagen merged commit 639bba6 into main Dec 16, 2024
12 of 14 checks passed

RalphSteinhagen deleted the exprtk_block branch December 16, 2024 20:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

new ExprTK-based Expression evaluation blocks #486

new ExprTK-based Expression evaluation blocks #486

RalphSteinhagen commented Dec 12, 2024 •

edited

Loading

wirew0rm left a comment

wirew0rm Dec 13, 2024

RalphSteinhagen Dec 13, 2024

wirew0rm Dec 13, 2024

wirew0rm Dec 13, 2024

wirew0rm Dec 13, 2024

wirew0rm Dec 13, 2024

wirew0rm Dec 13, 2024

wirew0rm Dec 13, 2024

wirew0rm Dec 13, 2024

sonarqubecloud bot commented Dec 16, 2024

	A<std::string, "expr string", Doc<"for syntax see: https://github.com/ArashPartow/exprtk">> expr_string = "clamp(-1.0, sin(2 * pi * x) + cos(x / 2 * pi), +1.0)";
	A<std::string, "expr string", Doc<"for syntax see: https://github.com/ArashPartow/exprtk">> expr_string = "0";

		exprtk::symbol_table<T> _symbol_table;
		exprtk::expression<T> _expression;

new ExprTK-based Expression evaluation blocks #486

new ExprTK-based Expression evaluation blocks #486

Conversation

RalphSteinhagen commented Dec 12, 2024 • edited Loading

wirew0rm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sonarqubecloud bot commented Dec 16, 2024

Quality Gate failed

RalphSteinhagen commented Dec 12, 2024 •

edited

Loading