Isnt this just forward mode automatic differentiation implemented with operator overloading on taylor series.

[MarisaKirisame]

No description provided.

[keithalewis]

Hi @MarisaKirisame . Thanks for taking an interest. I'm not sure what you mean by 'operator overloading on taylor series'. Can you point me at code I can run to help me understand that?

[MarisaKirisame]

I assume you understand forward mode automatic differentiation via operator overloading, aka dual number.
Dual Number is just taylor series truncated at first order epsilon.
You build a library that does forward mode automatic differentiation, at arbitrary truncation (or carrying infinitely).
This had been done before - see the paper "beautiful differentiation", for example. Or, a better way to say this is, this is how most higher order automatic differentiation algorithm work.

[keithalewis]

I am familiar with Conal Elliot's work and find it quite impressive, not just in AD. It is fun to write math papers but it is more rewarding to implement the math in software. The theory has been around for quite some time. Do you have any particular implementation in mind that I can download and run?

[MarisaKirisame]

That paper has runnable code.
Haskell's AD package also has forward mode AD tower.
http://www.fadbad.com/fadbad.html#General_introduction Also has higher order diff - it use forward over reverse but I assume it also has the simpler taylor series tower.

[keithalewis]

Haskell is not an option for me, but thanks for pointing this out. You also reminded me to take another look through www.Autodiff.org.

[MarisaKirisame]

If you had time, read the "evaluating derivative" book. I had seen numerous projects where ppl outside of AD start from scratch to build something, and after lots of work reinvent stuff already known. >90% of them reinvent stuff the book cover.