Recommendation for surrogate optimization

[NAThompson]

One of the main uses of surrogate modeling is surrogate optimization. To wit, given an expensive optimization function $\Phi$, queried at a set of points ${p_i}_{i=0}^{n-1}$$, we construct a surrogate model $\Psi$ from these training points in order to compute $p := \mathrm{argmax} \Psi$, which we then use to query $\Phi$. Once $\Phi(p)$ is computed, we repeat this process with an updated surrogate model, in the hope that faster convergence is obtained than direct black box optimization on $\Phi$.

Naively, we should be able to pass any smt surrogate model into some black-box optimization function to complete this process. But surrogate models are not black boxes; each has exploitable structure we could use to quickly perform surrogate optimization; e.g., argmax and argmin could become member functions on each surrogate model class.

Does smt provide guidance on compatible optimizers for surrogate optimization? If not, would pull requests implementing this feature be accepted?

[relf]

Surrogate-based optimization is already present in SMT as the EGO algorithm in the applications directory. The related documentation is available here.

EGO works with kriging-based surrogates as it allows to leverage on their ability to give the variance information and thus use more powerful infill criteria (EI) than just the one based on the predicted output value (SBO).

SMT is primarily focused on providing surrogate models with analytic derivatives (hence the name of the initial paper). The point was indeed to allow the usage of such surrogates with any optimizer using gradients. Does that count as a guidance? I do not see what kind of compatibility guidance you expected.

Besides the performance of the surrogate optimization will actually depend on the original problem and the quality of the surrogate.