Scaling and balancing

[baggepinnen]

Hi there 👋

I was reading an interesting paper on scaling and balancing of optimal-control problems (arXiv PDF), and have used some of the techniques from it with great effect. While trying to implement these scaling procedures in InfiniteOpt, my naive attempt at directly performing the scaling of the optimization variable $\tilde x$ to obtain the unscaled variable $x$ was thwarted by things like x[1](1): MethodError: objects of type GenericAffExpr{Float64, GeneralVariableRef} are not callable (example below). The workaround for this is of course to instead formulate everything in terms of the optimization variable $\tilde x$, but this is both a bit tedious and also very error prone, it's easy to forget to apply the scaling somewhere, or to apply the inverse scaling. The scaling has to be applied in a lot of places, constraints, dynamics, initial guesses etc.

A few ideas:

• It would be very nice if, similar to Dymos, one in InfiniteOpt could associate scaling on the form $P_x \tilde x + q_x$ (i.e., an affine transformation) with variables and constraints and have this handled underneath the hood. In Dymos, this scaling is controlled by the ref and ref0 arguments to functions such as add_state, add_control etc. The paper linked above is cited in the Dymos docs, I think they have implemented more or less the kind of scaling proposed in the paper.
• If not automatically handled like in Dymos, being able to access a time point of an affinly transformed variable (GenericAffExpr) would be very handy, like x[1](1) below

With manually handled scaling, InfiniteOpt+Ipopt handily beats Dymos+Ipopt on the particular problem we are considering, without the scaling, Ipopt struggles to produce anything reasonable.

using InfiniteOpt, Ipopt
m = InfiniteModel(Ipopt.Optimizer);

@infinite_parameter(m, τ in [0, 1], num_supports = 50, derivative_method = OrthogonalCollocation(3)) # Time variable

nx = nu = 1
InfiniteOpt.@variables(m, begin
    # state variables
    (x̃[i = 1:nx], Infinite(τ)) # state variables
    (ũ[i = 1:nu], Infinite(τ)) # control variables
end)

Px = Diagonal([10])
Pu = Diagonal([2])

x = Px*x̃
u = Pu*ũ

@objective(m, Min, x[1](1))

julia> @objective(m, Min, x[1](1))
ERROR: MethodError: objects of type GenericAffExpr{Float64, GeneralVariableRef} are not callable

[pulsipher]

Hi @baggepinnen,

Scaling is certainly important for getting optimization problems that are well behaved.

Regarding your first idea: I think such a feature addition would be better suited for JuMP, since the utility of scaling variables is not unique to InfiniteOpt problems. If JuMP were to have this feature, then InfiniteOpt would inherit it. JuMP already does something similar with handling complex variables, so I imagine that extending this capability for variable scaling should be doable.

Regarding the 2nd idea: This seems doable, we would just need to expand our variable restriction syntax to support expressions. I have created an issue: #347.

[pulsipher]

I made a PR (#348), does this satisfy what you are looking for?

[baggepinnen]

Thanks for your quick turnaround! The functionality added in the PR looks very nice 🎉

Maybe introducing scaling in JuMP is something we can discuss at JuliaCon/JuMP dev this summer, if you and Oscar will be there

[pulsipher]

PR merged! I'll be at JuMP-dev, but note that JuMP-dev and JuliaCon are not co-located this year. Otherwise, I recommend creating an issue on JuMP to make your feature request.