Refactor callable lookup mechanism #441

AdrianSosic · 2024-12-04T12:22:47Z

This PR refactors the callable lookup mechanism in the simulation module, fixing one of the major limitations of the module and preparing it for multi-target simulation.

In particular:

The position-based approach relying on the order of input arguments to the callable is replaced with a proper dataframe-based approach, similar to other user-facing functionality elsewhere. That is, the callable now expects a dataframe containing columns corresponding to the search space parameters and returns a dataframe containing columns corresponding to the targets of the objective.
A arrays_to_dataframes decorator utility has been added that allows to conveniently construct such callables from array-based ones
The custom_analytical example has been renamed to custom_blackbox and was rewritten from scratch, using the new approach and fixing the existing issues.
Redundant examples have been removed.
The filter_df function has been fixed such that it now correctly handles the edge case of an empty filter.

baybe/simulation/lookup.py

examples/Backtesting/botorch_analytical.py

baybe/utils/dataframe.py

baybe/simulation/lookup.py

examples/Backtesting/custom_analytical.py

benchmarks/domains/synthetic_2C1D_1C.py

examples/Basics/recommenders.py

baybe/simulation/lookup.py

CHANGELOG.md

AVHopp

I in general like the new approach. However, I am a bit puzzled by the amount of deleted examples.

baybe/simulation/lookup.py

examples/Backtesting/custom_analytical.py

baybe/simulation/lookup.py

baybe/utils/botorch_wrapper.py

examples/Basics/recommenders.py

baybe/simulation/lookup.py

benchmarks/domains/synthetic_2C1D_1C.py

examples/Backtesting/custom_analytical.py

The callable now operates with dataframe input/output, replacing the positional indexing approach with a clean label-based approach using parameter and target names.

Will be replaced with a proper one showing the difference between single target, desirability, and Pareto optimization.

The current version provides minimal additional insights compared to what is already presented in `custom_analytical.py`. Instead of just reiterating the same logic, it should rather compare different hybrid optimizers. For now, it is dropped because refactoring is not worth it.

AVHopp

Like it a lot. Some things could still be improved though :)

baybe/utils/dataframe.py

docs/userguide/simulation.md

examples/Backtesting/custom_analytical.py

examples/Constraints_Continuous/hybrid_space.py

examples/Searchspaces/continuous_space.py

examples/Transfer_Learning/backtesting.py

baybe/utils/dataframe.py

baybe/simulation/lookup.py

Co-authored-by: Martin Fitzner <17951239+Scienfitz@users.noreply.github.com>

AVHopp

Approve to unblock in case you want to get this done over the holidays

* Rename filter argument to to_keep * Make target dataframe positional-only * Revise docstring

AdrianSosic added the refactor label Dec 4, 2024

AdrianSosic self-assigned this Dec 4, 2024

AdrianSosic requested review from Scienfitz and AVHopp as code owners December 4, 2024 12:22

AdrianSosic force-pushed the refactor/lookup branch from 2ec5add to 531e029 Compare December 4, 2024 12:26

AdrianSosic marked this pull request as draft December 4, 2024 15:37

AdrianSosic marked this pull request as ready for review December 5, 2024 10:24

AdrianSosic commented Dec 5, 2024

View reviewed changes

baybe/simulation/lookup.py Outdated Show resolved Hide resolved

Scienfitz requested changes Dec 5, 2024

View reviewed changes

AdrianSosic commented Dec 5, 2024

View reviewed changes

CHANGELOG.md Show resolved Hide resolved

AVHopp reviewed Dec 9, 2024

View reviewed changes

AdrianSosic added 15 commits December 16, 2024 09:37

Remove unnecessary examples

cf718ba

Handle edge case of empty filter in dataframe utility

fd4cf9d

Refactor callable lookup mechanism

6033d8b

The callable now operates with dataframe input/output, replacing the positional indexing approach with a clean label-based approach using parameter and target names.

Implement decorator utility for creating lookup callables

05e63ef

Refactor custom analytical backtesting example

c05a71f

Update lookup mechanism in benchmark code

e100eda

Update CHANGELOG.md

2f8534c

Drop multi-target.py example

bc585c1

Will be replaced with a proper one showing the difference between single target, desirability, and Pareto optimization.

Remove outdated references

2f7dd1c

Remove botorch_function_wrapper

9596fbc

Explicitly convert target to objective

f976b55

Use dataframe-based lookup callable in user-facing code

4767692

Add back continuous search space example

e9b5db0

Simplify inplace column assignment

3bedbc3

AdrianSosic force-pushed the refactor/lookup branch from 814cb61 to 3bedbc3 Compare December 16, 2024 08:39

AdrianSosic added 3 commits December 16, 2024 13:00

Replace botorch_function_wrapper calls

2409f75

Add return type to decorator utility

f2a0157

Increase smoke test problem dimension

44bf262

AdrianSosic added 2 commits December 16, 2024 13:48

Fix sphinx references

0ce6171

Revise and extend lookup user guide

e6c12e4

AdrianSosic requested review from Scienfitz and AVHopp December 16, 2024 14:44

AdrianSosic added 3 commits December 16, 2024 15:48

Update CHANGELOG.md

9689e27

Fix example section heading

fbe3d69

Drop botorch_wrapper module from test configuration

10e3446

AVHopp reviewed Dec 17, 2024

View reviewed changes

Scienfitz reviewed Dec 19, 2024

View reviewed changes

baybe/utils/dataframe.py Show resolved Hide resolved

baybe/simulation/lookup.py Outdated Show resolved Hide resolved

Scienfitz approved these changes Dec 19, 2024

View reviewed changes

baybe/simulation/lookup.py Outdated Show resolved Hide resolved

AdrianSosic and others added 2 commits December 19, 2024 13:20

Use explicit value assignment for dataframe update

fe3a36d

Co-authored-by: Martin Fitzner <17951239+Scienfitz@users.noreply.github.com>

Remove unnecessary target from user guide example

71c9f74

AVHopp approved these changes Dec 19, 2024

View reviewed changes

AdrianSosic added 3 commits December 20, 2024 09:29

Apply minor text improvements

b64b0ce

Reference impute mode options via keyword link in admonition

b1d8373

Rewrite blackbox function docstring

f32207d

AdrianSosic force-pushed the refactor/lookup branch 2 times, most recently from 6a7d0ca to 21f611c Compare December 20, 2024 09:33

AdrianSosic added 4 commits December 20, 2024 10:35

Rename custom_analytical.py to custom_blackbox.py

6b201d4

Rename decorator utility

798a59a

Add admonition mentioning decorator utility

3d1c0eb

Refine filter_df logic

a6cc05a

* Rename filter argument to to_keep * Make target dataframe positional-only * Revise docstring

AdrianSosic force-pushed the refactor/lookup branch from 21f611c to a6cc05a Compare December 20, 2024 09:35

AdrianSosic merged commit fd78099 into main Dec 20, 2024
11 checks passed

AdrianSosic deleted the refactor/lookup branch December 20, 2024 09:58

AdrianSosic mentioned this pull request Dec 20, 2024

Reordering of parameters within simulate_scenarios? #454

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor callable lookup mechanism #441

Refactor callable lookup mechanism #441

AdrianSosic commented Dec 4, 2024 •

edited

Loading

AVHopp left a comment

AVHopp left a comment

AVHopp left a comment

Refactor callable lookup mechanism #441

Refactor callable lookup mechanism #441

Conversation

AdrianSosic commented Dec 4, 2024 • edited Loading

AVHopp left a comment

Choose a reason for hiding this comment

AVHopp left a comment

Choose a reason for hiding this comment

AVHopp left a comment

Choose a reason for hiding this comment

AdrianSosic commented Dec 4, 2024 •

edited

Loading