C++ interface prototyping

[samuelpmishLLNL]

this PR implements a basic tuple class and a sketch of some pieces for a C++ interface requested here: #1367 (comment)

Also, some other potential interface choices:

    //////////////////
    // forward mode //
    //////////////////

    // passing fn + arguments to enzyme::forward returns a functor that evalutes J * dx
    auto J = enzyme::forward(foo, enzyme::active{arg1}, arg2, enzyme::active{arg3}); 

    // df := (df/darg1 * darg1) + (df/darg3 * darg3)
    auto df = J(darg1, darg3); 

    // passing fn + duplicated arguments to enzyme::forward evaluates J * dx directly
    //
    // df := (df/darg1 * darg1) + (df/darg3 * darg3)
    auto df = enzyme::forward(foo, enzyme::duplicated{arg1, darg1}, arg2, enzyme::duplicated{arg3, darg3}); 

    //////////////////
    // reverse mode //
    //////////////////

    // passing fn + arguments to enzyme::reverse returns a functor that evalutes J^T * lambda
    auto JT = enzyme::reverse(foo, enzyme::active{arg1}, arg2, enzyme::active{arg3}); 

    // mu := J^T * lambda, note: outputs correspond to active arguments in call to enzyme::reverse
    auto [mu_1, _0, mu_3] = JT(lambda); 
    // or ?
    auto [mu_1, mu_3] = JT(lambda);    

    // passing fn + duplicated arguments to enzyme::forward evaluates J * dx directly
    auto [mu_1, mu_3] = enzyme::forward(foo, lambda, enzyme::active{arg1}, arg2, enzyme::active{arg3}); 

}

[wsmoses]

mind adding some example usage tests?

also could we drop the hpp suffix?

[samuelpmishLLNL]

Sorry for putting this down for a couple weeks-- what sorts of examples would you like in this PR?

I think most examples won't actually compile until the appropriate changes to the enzyme backend are in place, though.

[samuelpmishLLNL]

@wsmoses any idea how to address these undefined reference errors: https://fwd.gymni.ch/aptybv ?

I don't know that it's possible (in conventional C++) to implement the variadic enzyme::autodiff template without std::apply, but I can't seem to get __enzyme_autodiff to work as an argument to std::apply.

I also tried providing a trivial implementation of __enzyme_autodiff (in hopes that it would be replaced by Enzyme). While this did stop the compilation errors, it also seems to result in incorrect output (see enzyme/test/test_find_package/cpp_interface.cpp).

[wsmoses]

How about this: https://fwd.gymni.ch/pSZCir I think the std apply functions weren't being inlined at O0. Inspiration from https://en.cppreference.com/w/cpp/utility/apply and https://stackoverflow.com/questions/52449163/what-is-the-difference-between-stdinvoke-and-stdapply

[samuelpmishLLNL]

How about this: https://fwd.gymni.ch/pSZCir

Implementing our own apply is a good idea (we're already implementing the other parts of tuple).

---

I'm still seeing an undefined reference when putting apply inside a function template though: https://fwd.gymni.ch/zijpaB

the undefined reference does go away when compiling with optimization.

[wsmoses]

Some attempted improvements: https://fwd.gymni.ch/oSe4rO

…

On Fri, Sep 15, 2023 at 3:22 PM samuelpmishLLNL ***@***.***> wrote: How about this: https://fwd.gymni.ch/pSZCir Implementing our own apply is a good idea (we're already implementing the other parts of tuple). ------------------------------ I'm still seeing an undefined reference when putting apply inside a function template though: https://fwd.gymni.ch/zijpaB the undefined reference does go away when compiling with optimization. — Reply to this email directly, view it on GitHub <#1371 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJTUXFRU2MNQLYXT6Q7X6TX2S2HJANCNFSM6AAAAAA3KN3MKE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[wsmoses]

@samuelpmishLLNL I fixed the autodiff function detector to do a primitive mem2reg and find more functions. However, we need to switch from std tuple to our tuple since otherwise the use of enzyme_dup ints are hidden behind non-inlined functions at O0

[samuelpmishLLNL]

we need to switch from std tuple to our tuple since otherwise the use of enzyme_dup ints are hidden behind non-inlined functions at O0

Implementing tuple_cat is pretty ugly, but I can look into it.

[samuelpmishLLNL]

I took at stab at the tuple_cat implementation and sprinkled __attribute__((always_inline)) everywhere, but I'm still getting undefined symbol issues with -O0 and -O3.

[build] Undefined symbols for architecture arm64:
[build]   "double __enzyme_autodiff<double, int, double>(void*, int, double)", referenced from:
[build]       _main in cpp_interface.cpp.o
[build] ld: symbol(s) not found for architecture arm64

Is there a way to figure out which part is not getting inlined?

[samuelpmishLLNL]

here's an updated compiler explorer link with a reproducer for the undefined reference error @ -O0:
https://fwd.gymni.ch/a3Ysvi

[wsmoses]

This https://fwd.gymni.ch/YzKLEy gets us closer. Specifically (1), we need enzyme_autodiff in the innermost level rather than being applied.

(2), I propagated the function to differentiate explicitly/separately. However, now with the inlining complete (still progress) the enzyme_dup isn't being recognized since its an element of a struct needing to be sroa'd

[wsmoses]

Superceeded by #1752