Relax type constraints to permit AbstractVectors

[oxinabox]

I think this is safe.
We do still convert everything to Vector{<:Real} via to_vec.
But if someone wants to bypass that by defining something like

FiniteDifferences.to_vec(x::CuVector{Float32}) = x, y->y

then this might make there lives easier.
It is stepping well off the beaten track, but I don't think we get anything from blocking them like this

[wesselb]

Sounds very sensible! If the failing CI tests are not related to this, then looks like we can just merge this.

Perhaps @willtebbutt would want to have a quick look too?

[oxinabox]

yeah CI failures are unrelated

[Keno]

I'm updating DAECompiler, which has a test here: https://github.com/CedarEDA/DAECompiler.jl/blob/main/test/sensitivity.jl#L120

which uses an immutable AbstractVector, which is unhappy with this code path, but worked fine before. I'll work around it, but flagging this, since @oxinabox wrote that test.