AD with complex variables

[colinjcotter]

I have been doing a bit of digging about what needs to be done for AD in the complex valued case. I think this is probably covered in AD textbooks but I didn't know where to look, but I found some examples in ML papers such as this one:
https://arxiv.org/abs/1701.00392

A general function of a complex variable can be equivalently written as a function of the real and imaginary parts, f(x, y). Everything will simply work if we do that, treating these parts separately. The problem is that the symbolics of UFL do not take this into account, because UFL assumes real calculus with only one variable. A solution which tries to maximise code reuse is to use the "Wirtinger calculus" which rewrites f as a function of z and z^*, the complex conjugate of z, so that

df/dz = df/dx + i df/dy (partial derivatives here but I didn't take time to latex them)
df/dz^* = df/dx + idf/dz^*
For holomorphic functions, df/dz^*=0, and we can just do normal calculus pretending that z is one variable instead of two. All the AD operations in UFL just work in that case.
However, we can't assume holomorphic in general, e.g. formulations require complex inner product etc.

For general functions, one can compute as if z and z^* are independent variables, using the identity (3.7) in the reference above. This would require intervention in UFL of course.