Skip to content

Commit

Permalink
Version 2.1.0 (#1108)
Browse files Browse the repository at this point in the history
* Hacking in the sum cost for MO

* Fix mypi

* Bugfix

* Make mypi happy

* Fix tests

* Make format

* Remove num_obj in MO optimizer

* A lot of restructuring

* Solve importings

* Solve importings

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Updated changelog

* Still fixing :)

* fix docstrings

* Change directory of tests

* Fix ambiguous variable

* Del newlines

* fix mypy

* mypy and doc fix

* Removed int type checking

* fix tests

* Always return List of incs

* fix path issue in example

* maint boing facade

* Better docstring

* Return values if no bounds passed

* Removed import

* Return only one incumbent

* Rename n_optimizers -> n_workers

* Update example

* Update docstring

* Remove legacy function

* Update to new signature

* Update changelog.md

* Format black

* Update test

* Please pre-commit

* Merge development

* Fix tests

* Fix typos

* Format black

* Updated text

* Increased version

* Improved readme

* SMAC logo 50%

* SMAC logo 50%

* Removed dev-2.0

* Added doc workflow for latest tag

* Fixing docs latest tag

* Fixing docs latest tag

* First steps new intensifier

* Make it work again

* MOVE random weight initialization to smbo ask method

* ADD enable parego in unittests

* Test issues

* FIX formatting

* Link to old documentation

* MOVE random weight initialization to smbo ask method

* ADD enable parego in unittests

* FIX formatting

* Fix unit tests

* Add annotation

* corrected log prob for tophat prior

* Removed configurations from callback

* Fixed tests

* Fix mypy

* Save docs as latest version too

* Enabled all tests again

* Make version an env

* Added last version to docs

* Global variable fix

* Added v before version number

* New Intensifier Implementation (#888)

- Completely reimplemented the intensifiers (including Successive Halving and Hyperband): All intensifiers support multi-fidelity, multi-objective and multi-threading by nature now.
- Expected behaviour for ask-and-tell interface ensured (also for Successive Halving).
- Continuing a run is now fully supported.
- Added more examples.
- Updated documentation based on new implementation.

* First benchmark steps

* Added a warning

* Set xi to 0.01

* Incorporated walltime in trajectory

* Working version of benchmark

* Changed requirements

* Changed xi back to 0.0 again

* No duplicate budgets

* Fixed bug which updated tracker wrongly

* Bugfix: Delete correct bracket+stage

* Fixed tests

* Added costs to the trajectory

* Changed initial design max ratio to 0.25

* Make format

* Added more details to intensifier

* Finished benchmark

* Added benchmark source files

* Fixed tests because of initial design changes

* Improved runhistory API

* Fixed typo (#894)

* Merge doc fixes

* Doc fixes (#893)

Co-authored-by: timruhkopf <timruhkopf@gmail.com>
Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>
Co-authored-by: dengdifan <difandeng@gmail.com>
Co-authored-by: Alexander Tornede <tornede@tnt.uni-hannover.de>
Co-authored-by: Sarah Krebs <s.krebs@ai.uni-hannover.de>

* Make format

* Satisfied mypy

* Enabled more tests

* Increased version

* Mypy

* Mypy

* Mypy

* Fixed critical bug when using highest budget for inc selection

* Added SH+HB comparison

* Stuff

* Updated report

* Fixed docstring

* Fixed image path

* Removed vscode folder

* Bugfixes

* Doc fixes

* Updated changelog

* Make isort

* Fix typos

* Removed new line

* Added alpha2 to docs selection

* update description

* Fix 531 correctly create integers in initial design

This PR fixes a bug in which integer hyperparameters would not
be created correctly in the initial design. This went
unnoticed, except in the case where it was part of a condition
and the function deactivate_inactive_hyperparameters was
called.

This PR fixes this bug by mapping the random float that was
sampled by, for example and LHD or Sobol sequence, to the
float representation of an integer by calling _transform and
_inverse_transform of the integer hyperparameter.

* Add contact paragraph to README.

* Increase days until stale issues are closed by stalebot (#904)

* Edit changelog

* Add issue number

* Closes #907

* Clarify origin of configurations (#908)

* Random forest speed up (#903)

Co-authored-by: René Sass <mail@renesass.de>

* Add: workflow to update pre-commit versions (#874)

Co-authored-by: René Sass <mail@renesass.de>

* Add Git-Flow Workflow Specification

* [DOC] acquisition function maximizers

* Add stale reminder email action

* Increase version number of ConfigSpace

* Add guideline for updating copyright notice years to CONTRIBUTING.md

* Update requirement for pyrfr (#920)

* Update pre-commit: isort,flake8 version numbers

* Format black

* Fix mypy

* Ac bench (#929)

* Add Branin AC Task

* Add ac optimization type

* Nicer plotting

* Add ac mode

* Update ac task

* Make new smac env name adjustable

* Increase python version for smac 1.4

* Update benchmark

* Update plotting

* Goodby comments

* Fix path

* Fix 1.4 wrapper

* Update benchmark results

---------

Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>

* [DOC] added documentation on the incumbent update

* Feature/add version to make publish (#931)

* Add version, correct package name and use bash in makefile

* Increase version

* Increase version in makefile, add places with versions to make publish

* Fix typo

* Transform integer feature names to strings (#930)

* Transform int features to str

* Fix pre-commit

* Update version to 2.0.0

* Update version to 2.0.0

* Update CHANGELOG.md

* Add information on deepcave to the readme

* Correct doc version

* fix(runhistory): set id of the sampled config (#951)

* fix: Set config_id of config in runhistory

* ci: Set dependabot to target development branch (#963)

* feat: Option preventing SMAC setting up logging (#947)

* Version 2.0.0 (#898)

* Update runhistory2epm.py

* pre_commit

* Updated changelog

* Fixed comment

* Updated changelog

* Some small improvements

* Removed trailing space

* Updated changelog

* Restricted pynisher

* Updated changelog

* Link JMLR paper

* Update bibtex entry

Before: arxiv paper, now journal article.

* Fixed citation in docs

* Fixed callback typos

* BOinG for mixed hyperparameter search space

* fix format

* maint

* maint documentation

* maint pre-commit

* fix workflow

* maint

* Add dependabot for workflow files

* Bump actions/setup-python from 2 to 4 (#849)

Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2 to 4.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2...v4)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump actions/checkout from 2 to 3 (#850)

Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* maint

* Updated stale bot

* Updated stale options again

* Fixed typing

* Increased version

* rename check_point_in_ss to check_subspace_point

* Make format

* Uncomment test procedure

* docstrings for boing facade

* avoid typing namespace everywhere in epm_configuration_chooser

* fix docstring in rh2epm_boing

* Limit on dask

* Trying to fix tests

* Replaced typing in all files

* docstring for boing subspace

* Changed feature requests to feature

* allow acq_optimizer_local_kwargs to be passed to boing subspace without acq_optimizer_local

* fix pre-commit

* Added any

* Update facade

* Fix typing
* Match arguments from pSMAC with SMAC4AC signature
* Add description of pSMAC to docstring
* Add option to pass facade class to pSMAC

* Add example for pSMAC

* Removed docstring in readme's example

* Corrected text

* move construct_gp_kernels to smac.epm_gpytorch.boing_kernels

* maint

* fix wrongly imported func name in tests

* Add reference for pSMAC #609

* Rename example file

* Added roadmap to readme

* Prettification

* Add result merger

* Reformat with black

* Delete unnecessary import

* Fix typing

* Inject first trajectory entry from file from first rundir

* Add test for ResultMerger

* Fix docstring

* Get runhistory and trajectory from pSMAC run

Via the result merger. Searchers rundirs and
merges found runhistories and calculates a
new trajectory.

* Get trajectory

* Add regex to requirements

* Fixed MO cache problem

* Fix MO tests

* Fixing MO in AbstractRacer

* Accepting ints

* Hacking in the sum cost for MO

* Fix mypi

* Bugfix

* Make mypi happy

* Fix tests

* Make format

* Remove num_obj in MO optimizer

* A lot of restructuring

* Solve importings

* Solve importings

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Updated changelog

* Still fixing :)

* fix docstrings

* Change directory of tests

* Fix ambiguous variable

* Del newlines

* fix mypy

* mypy and doc fix

* Removed int type checking

* fix tests

* Always return List of incs

* fix path issue in example

* maint boing facade

* Better docstring

* Return values if no bounds passed

* Removed import

* Return only one incumbent

* Rename n_optimizers -> n_workers

* Update example

* Update docstring

* Remove legacy function

* Update to new signature

* Update changelog.md

* Format black

* Update test

* Please pre-commit

* Merge development

* Fix tests

* Fix typos

* Format black

* Updated text

* Increased version

* Improved readme

* SMAC logo 50%

* SMAC logo 50%

* Removed dev-2.0

* Added doc workflow for latest tag

* Fixing docs latest tag

* Fixing docs latest tag

* First steps new intensifier

* Make it work again

* MOVE random weight initialization to smbo ask method

* ADD enable parego in unittests

* Test issues

* FIX formatting

* Link to old documentation

* MOVE random weight initialization to smbo ask method

* ADD enable parego in unittests

* FIX formatting

* Fix unit tests

* Add annotation

* corrected log prob for tophat prior

* Removed configurations from callback

* Fixed tests

* Fix mypy

* Save docs as latest version too

* Enabled all tests again

* Make version an env

* Added last version to docs

* Global variable fix

* Added v before version number

* New Intensifier Implementation (#888)

- Completely reimplemented the intensifiers (including Successive Halving and Hyperband): All intensifiers support multi-fidelity, multi-objective and multi-threading by nature now.
- Expected behaviour for ask-and-tell interface ensured (also for Successive Halving).
- Continuing a run is now fully supported.
- Added more examples.
- Updated documentation based on new implementation.

* First benchmark steps

* Added a warning

* Set xi to 0.01

* Incorporated walltime in trajectory

* Working version of benchmark

* Changed requirements

* Changed xi back to 0.0 again

* No duplicate budgets

* Fixed bug which updated tracker wrongly

* Bugfix: Delete correct bracket+stage

* Fixed tests

* Added costs to the trajectory

* Changed initial design max ratio to 0.25

* Make format

* Added more details to intensifier

* Finished benchmark

* Added benchmark source files

* Fixed tests because of initial design changes

* Improved runhistory API

* Fixed typo (#894)

* Merge doc fixes

* Doc fixes (#893)

Co-authored-by: timruhkopf <timruhkopf@gmail.com>
Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>
Co-authored-by: dengdifan <difandeng@gmail.com>
Co-authored-by: Alexander Tornede <tornede@tnt.uni-hannover.de>
Co-authored-by: Sarah Krebs <s.krebs@ai.uni-hannover.de>

* Make format

* Satisfied mypy

* Enabled more tests

* Increased version

* Mypy

* Mypy

* Mypy

* Fixed critical bug when using highest budget for inc selection

* Added SH+HB comparison

* Stuff

* Updated report

* Fixed docstring

* Fixed image path

* Removed vscode folder

* Bugfixes

* Doc fixes

* Updated changelog

* Make isort

* Fix typos

* Removed new line

* Added alpha2 to docs selection

* update description

* Fix 531 correctly create integers in initial design

This PR fixes a bug in which integer hyperparameters would not
be created correctly in the initial design. This went
unnoticed, except in the case where it was part of a condition
and the function deactivate_inactive_hyperparameters was
called.

This PR fixes this bug by mapping the random float that was
sampled by, for example and LHD or Sobol sequence, to the
float representation of an integer by calling _transform and
_inverse_transform of the integer hyperparameter.

* Add contact paragraph to README.

* Increase days until stale issues are closed by stalebot (#904)

* Edit changelog

* Add issue number

* Closes #907

* Clarify origin of configurations (#908)

* Random forest speed up (#903)

Co-authored-by: René Sass <mail@renesass.de>

* Add: workflow to update pre-commit versions (#874)

Co-authored-by: René Sass <mail@renesass.de>

* Add Git-Flow Workflow Specification

* [DOC] acquisition function maximizers

* Add stale reminder email action

* Increase version number of ConfigSpace

* Add guideline for updating copyright notice years to CONTRIBUTING.md

* Update requirement for pyrfr (#920)

* Update pre-commit: isort,flake8 version numbers

* Format black

* Fix mypy

* Ac bench (#929)

* Add Branin AC Task

* Add ac optimization type

* Nicer plotting

* Add ac mode

* Update ac task

* Make new smac env name adjustable

* Increase python version for smac 1.4

* Update benchmark

* Update plotting

* Goodby comments

* Fix path

* Fix 1.4 wrapper

* Update benchmark results

---------

Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>

* [DOC] added documentation on the incumbent update

* Feature/add version to make publish (#931)

* Add version, correct package name and use bash in makefile

* Increase version

* Increase version in makefile, add places with versions to make publish

* Fix typo

* Transform integer feature names to strings (#930)

* Transform int features to str

* Fix pre-commit

* Update version to 2.0.0

* Update version to 2.0.0

* Update CHANGELOG.md

* Add information on deepcave to the readme

* Correct doc version

---------

Co-authored-by: dengdifan <33290713+dengdifan@users.noreply.github.com>
Co-authored-by: dengdifan <difandeng@gmail.com>
Co-authored-by: Matthias Feurer <feurerm@informatik.uni-freiburg.de>
Co-authored-by: Carolin Benjamins <benjamin@tnt.uni-hannover.de>
Co-authored-by: eddiebergman <eddiebergmanhs@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>
Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com>
Co-authored-by: Katharina Eggensperger <eggenspk@informatik.uni-freiburg.de>
Co-authored-by: Eric Kalosa-Kenyon <ekquasar@appdynamics.com>
Co-authored-by: Pascal <66594767+PascalIversen@users.noreply.github.com>
Co-authored-by: helegraf <Graf.Helena@web.de>
Co-authored-by: timruhkopf <timruhkopf@gmail.com>
Co-authored-by: Alexander Tornede <tornede@tnt.uni-hannover.de>
Co-authored-by: Sarah Krebs <s.krebs@ai.uni-hannover.de>
Co-authored-by: Alexander Tornede <a.tornede@ai.uni-hannover.de>
Co-authored-by: Alexander Tornede <44395781+alexandertornede@users.noreply.github.com>
Co-authored-by: Sarah Krebs <30651149+sarah-krebs@users.noreply.github.com>
Co-authored-by: helegraf <h.graf@ai.uni-hannover.de>
Co-authored-by: Jeroen Rook <jrook94@gmail.com>
Co-authored-by: Tim Ruhkopf <timruhkopf@googlemail.com>

* feat: Option preventing SMAC setting up logging

* fix: Run isort

* Bump peter-evans/create-pull-request from 3 to 5 (#958)

Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 3 to 5.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/v3...v5)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump actions/setup-python from 2 to 4 (#945)

Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2 to 4.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2...v4)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* feat: Option preventing SMAC setting up logging

* fix: Run isort

* fix: Add to changelog

* feat: Option preventing SMAC setting up logging

* fix: Run isort

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: René Sass <mail@renesass.de>
Co-authored-by: dengdifan <33290713+dengdifan@users.noreply.github.com>
Co-authored-by: dengdifan <difandeng@gmail.com>
Co-authored-by: Matthias Feurer <feurerm@informatik.uni-freiburg.de>
Co-authored-by: Carolin Benjamins <benjamin@tnt.uni-hannover.de>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>
Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com>
Co-authored-by: Katharina Eggensperger <eggenspk@informatik.uni-freiburg.de>
Co-authored-by: Eric Kalosa-Kenyon <ekquasar@appdynamics.com>
Co-authored-by: Pascal <66594767+PascalIversen@users.noreply.github.com>
Co-authored-by: helegraf <Graf.Helena@web.de>
Co-authored-by: timruhkopf <timruhkopf@gmail.com>
Co-authored-by: Alexander Tornede <tornede@tnt.uni-hannover.de>
Co-authored-by: Sarah Krebs <s.krebs@ai.uni-hannover.de>
Co-authored-by: Alexander Tornede <a.tornede@ai.uni-hannover.de>
Co-authored-by: Alexander Tornede <44395781+alexandertornede@users.noreply.github.com>
Co-authored-by: Sarah Krebs <30651149+sarah-krebs@users.noreply.github.com>
Co-authored-by: helegraf <h.graf@ai.uni-hannover.de>
Co-authored-by: Jeroen Rook <jrook94@gmail.com>
Co-authored-by: Tim Ruhkopf <timruhkopf@googlemail.com>

* Citation update (#961)

* update CITATION.cff

* update order in CITATION.cff

* fix citation date

* fix citation

* maint

* Adjust hyperband configuration distribution across brackets

* Compute the actual differences between the isb keys. (#957)

* Compute the actual differences between the isb keys.

* Added change to CHANGELOG.md

* Adjust schedule for stale reminder

* fix(logging): Prevent automatic logging setup at init (#970)

* fix(logging): Prevent automatic logging setup at init

* fix(logging): Rely on `setup_logging` to handle arg

* Fix validate method of smbo, update docstring.

* Allow callbacks to be added to a specific index, make callback registering public

* Update CHANGELOG.md

* Fix broken references (#988)

* Adapt developer install instructions to include pre-commit installation (#994)

* [Feature] Initial Design now supports executing the default config provided by the user by means of the configspace he generated. (and is no longer ignored)

* Pipe dask_client through facade for exec on hpc (#983)

* Pipe dask_client through facade for exec on hpc

* Handle pynisher's memory arg

Before: Could only pass int via SMAC with the default unit B, but now tuple/list is possible

* Update CHANGELOG.md

* Make format

* No printi

---------

Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>

* [Bug-fix] Pr was failing due to mutable additional configs default

* [doc-fix] the count of initial design configs is computed slightly differently

* Feature/metadata callback (#999)

* Add example for using a callback to log run metadata to a file

* Add issue number in changelog

* Changelog formatting

* Simplify metadata example

* Simplify metadata example

* Put callback in smac directory to allow importing it in custom repos

* Move callback files to callback directory

* Edit changelog

* Correct metadata callback argument types

* Additional metadata arguments in example

* Set metadata default values in callback

* Edit comment to fix PR

* Documentation/dask client example (#1001)

* Update requirements (dask_jobqueue)

* Fix broken link

* Ignore parallel example

* Add parallelization example

* Remove imports, add comment abt dask client

* Add E's information

* Update docs

* Add comment about n_workers

---------

Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>

* Documentation/dask client example (#1001)

* Update requirements (dask_jobqueue)

* Fix broken link

* Ignore parallel example

* Add parallelization example

* Remove imports, add comment abt dask client

* Add E's information

* Update docs

* Add comment about n_workers

---------

Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>

* Feature/dask client (#1002)

* Add info message that scenario.n_workers is ignored

When passing a dask client,
directly set n_workers in client.

* Change info to warning

---------

Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>

* Update smac version (#1003)

* Update smac version

* Add hint on github release in make publish

* Update copyright year

* check if config in rh when storing state (#997)

* check if config in rh when storing state

* maint

* remove unnecessary changes

* Adapt developer install instructions to include pre-commit installation (#994)

* Pipe dask_client through facade for exec on hpc (#983)

* Pipe dask_client through facade for exec on hpc

* Handle pynisher's memory arg

Before: Could only pass int via SMAC with the default unit B, but now tuple/list is possible

* Update CHANGELOG.md

* Make format

* No printi

---------

Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>

* update changelog

* Increase tolerance for runhistory restoration test

---------

Co-authored-by: Sarah Segel <30651149+sarah-segel@users.noreply.github.com>
Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com>
Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>
Co-authored-by: Helena Graf <h.graf@ai.uni-hannover.de>

* Add a workaround to be able to pass a dataset via dask.scatter (#993)

* [feat] Support dask.scatter

* [fix] Add shared_data arg in abstract_facade

* Address the comments by Eddie

* [fix] Fix errors raised by Eddie

* [format] Apply black locally

* [fix] Get retries back to 16 as it was a mistake

* [doc] Update CHANGELOG

---------

Co-authored-by: helegraf <h.graf@ai.uni-hannover.de>
Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com>

* Adding Binder links for example (#976)

* binder

* maint

* fix dependencies

* maint docs

* Revert "maint docs"

This reverts commit 4fa1a0fae8dfa0b646e43273d50f72ddd76bf191.

* add requirement

---------

Co-authored-by: Difan Deng <deng@dengster.tnt.uni-hannover.de>

* Remove commit message from dependabot PRs (#1009)

* Bugfix/fix broken link (#1012)

* Remove commit message from dependabot PRs (#1010)

* rename check_point_in_ss to check_subspace_point

* Make format

* Uncomment test procedure

* docstrings for boing facade

* avoid typing namespace everywhere in epm_configuration_chooser

* fix docstring in rh2epm_boing

* Limit on dask

* Trying to fix tests

* Replaced typing in all files

* docstring for boing subspace

* Changed feature requests to feature

* allow acq_optimizer_local_kwargs to be passed to boing subspace without acq_optimizer_local

* fix pre-commit

* Added any

* Update facade

* Fix typing
* Match arguments from pSMAC with SMAC4AC signature
* Add description of pSMAC to docstring
* Add option to pass facade class to pSMAC

* Add example for pSMAC

* Removed docstring in readme's example

* Corrected text

* move construct_gp_kernels to smac.epm_gpytorch.boing_kernels

* maint

* fix wrongly imported func name in tests

* Add reference for pSMAC #609

* Rename example file

* Added roadmap to readme

* Prettification

* Add result merger

* Reformat with black

* Delete unnecessary import

* Fix typing

* Inject first trajectory entry from file from first rundir

* Add test for ResultMerger

* Fix docstring

* Get runhistory and trajectory from pSMAC run

Via the result merger. Searchers rundirs and
merges found runhistories and calculates a
new trajectory.

* Get trajectory

* Add regex to requirements

* Fixed MO cache problem

* Fix MO tests

* Fixing MO in AbstractRacer

* Accepting ints

* Hacking in the sum cost for MO

* Fix mypi

* Bugfix

* Make mypi happy

* Fix tests

* Make format

* Remove num_obj in MO optimizer

* A lot of restructuring

* Solve importings

* Solve importings

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Updated changelog

* Still fixing :)

* fix docstrings

* Change directory of tests

* Fix ambiguous variable

* Del newlines

* fix mypy

* mypy and doc fix

* Removed int type checking

* fix tests

* Always return List of incs

* fix path issue in example

* maint boing facade

* Better docstring

* Return values if no bounds passed

* Removed import

* Return only one incumbent

* Rename n_optimizers -> n_workers

* Update example

* Update docstring

* Remove legacy function

* Update to new signature

* Update changelog.md

* Format black

* Update test

* Please pre-commit

* Merge development

* Fix tests

* Fix typos

* Format black

* Updated text

* Increased version

* Improved readme

* SMAC logo 50%

* SMAC logo 50%

* Removed dev-2.0

* Added doc workflow for latest tag

* Fixing docs latest tag

* Fixing docs latest tag

* First steps new intensifier

* Make it work again

* MOVE random weight initialization to smbo ask method

* ADD enable parego in unittests

* Test issues

* FIX formatting

* Link to old documentation

* MOVE random weight initialization to smbo ask method

* ADD enable parego in unittests

* FIX formatting

* Fix unit tests

* Add annotation

* corrected log prob for tophat prior

* Removed configurations from callback

* Fixed tests

* Fix mypy

* Save docs as latest version too

* Enabled all tests again

* Make version an env

* Added last version to docs

* Global variable fix

* Added v before version number

* New Intensifier Implementation (#888)

- Completely reimplemented the intensifiers (including Successive Halving and Hyperband): All intensifiers support multi-fidelity, multi-objective and multi-threading by nature now.
- Expected behaviour for ask-and-tell interface ensured (also for Successive Halving).
- Continuing a run is now fully supported.
- Added more examples.
- Updated documentation based on new implementation.

* First benchmark steps

* Added a warning

* Set xi to 0.01

* Incorporated walltime in trajectory

* Working version of benchmark

* Changed requirements

* Changed xi back to 0.0 again

* No duplicate budgets

* Fixed bug which updated tracker wrongly

* Bugfix: Delete correct bracket+stage

* Fixed tests

* Added costs to the trajectory

* Changed initial design max ratio to 0.25

* Make format

* Added more details to intensifier

* Finished benchmark

* Added benchmark source files

* Fixed tests because of initial design changes

* Improved runhistory API

* Fixed typo (#894)

* Merge doc fixes

* Doc fixes (#893)

Co-authored-by: timruhkopf <timruhkopf@gmail.com>
Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>
Co-authored-by: dengdifan <difandeng@gmail.com>
Co-authored-by: Alexander Tornede <tornede@tnt.uni-hannover.de>
Co-authored-by: Sarah Krebs <s.krebs@ai.uni-hannover.de>

* Make format

* Satisfied mypy

* Enabled more tests

* Increased version

* Mypy

* Mypy

* Mypy

* Fixed critical bug when using highest budget for inc selection

* Added SH+HB comparison

* Stuff

* Updated report

* Fixed docstring

* Fixed image path

* Removed vscode folder

* Bugfixes

* Doc fixes

* Updated changelog

* Make isort

* Fix typos

* Removed new line

* Added alpha2 to docs selection

* update description

* Fix 531 correctly create integers in initial design

This PR fixes a bug in which integer hyperparameters would not
be created correctly in the initial design. This went
unnoticed, except in the case where it was part of a condition
and the function deactivate_inactive_hyperparameters was
called.

This PR fixes this bug by mapping the random float that was
sampled by, for example and LHD or Sobol sequence, to the
float representation of an integer by calling _transform and
_inverse_transform of the integer hyperparameter.

* Add contact paragraph to README.

* Increase days until stale issues are closed by stalebot (#904)

* Edit changelog

* Add issue number

* Closes #907

* Clarify origin of configurations (#908)

* Random forest speed up (#903)

Co-authored-by: René Sass <mail@renesass.de>

* Add: workflow to update pre-commit versions (#874)

Co-authored-by: René Sass <mail@renesass.de>

* Add Git-Flow Workflow Specification

* [DOC] acquisition function maximizers

* Add stale reminder email action

* Increase version number of ConfigSpace

* Add guideline for updating copyright notice years to CONTRIBUTING.md

* Update requirement for pyrfr (#920)

* Update pre-commit: isort,flake8 version numbers

* Format black

* Fix mypy

* Ac bench (#929)

* Add Branin AC Task

* Add ac optimization type

* Nicer plotting

* Add ac mode

* Update ac task

* Make new smac env name adjustable

* Increase python version for smac 1.4

* Update benchmark

* Update plotting

* Goodby comments

* Fix path

* Fix 1.4 wrapper

* Update benchmark results

---------

Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>

* [DOC] added documentation on the incumbent update

* Feature/add version to make publish (#931)

* Add version, correct package name and use bash in makefile

* Increase version

* Increase version in makefile, add places with versions to make publish

* Fix typo

* Transform integer feature names to strings (#930)

* Transform int features to str

* Fix pre-commit

* Update version to 2.0.0

* Update version to 2.0.0

* Update CHANGELOG.md

* Add information on deepcave to the readme

* Correct doc version

* fix(runhistory): set id of the sampled config (#951)

* fix: Set config_id of config in runhistory

* ci: Set dependabot to target development branch (#963)

* feat: Option preventing SMAC setting up logging (#947)

* Version 2.0.0 (#898)

* Update runhistory2epm.py

* pre_commit

* Updated changelog

* Fixed comment

* Updated changelog

* Some small improvements

* Removed trailing space

* Updated changelog

* Restricted pynisher

* Updated changelog

* Link JMLR paper

* Update bibtex entry

Before: arxiv paper, now journal article.

* Fixed citation in docs

* Fixed callback typos

* BOinG for mixed hyperparameter search space

* fix format

* maint

* maint documentation

* maint pre-commit

* fix workflow

* maint

* Add dependabot for workflow files

* Bump actions/setup-python from 2 to 4 (#849)

Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2 to 4.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2...v4)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump actions/checkout from 2 to 3 (#850)

Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* maint

* Updated stale bot

* Updated stale options again

* Fixed typing

* Increased version

* rename check_point_in_ss to check_subspace_point

* Make format

* Uncomment test procedure

* docstrings for boing facade

* avoid typing namespace everywhere in epm_configuration_chooser

* fix docstring in rh2epm_boing

* Limit on dask

* Trying to fix tests

* Replaced typing in all files

* docstring for boing subspace

* Changed feature requests to feature

* allow acq_optimizer_local_kwargs to be passed to boing subspace without acq_optimizer_local

* fix pre-commit

* Added any

* Update facade

* Fix typing
* Match arguments from pSMAC with SMAC4AC signature
* Add description of pSMAC to docstring
* Add option to pass facade class to pSMAC

* Add example for pSMAC

* Removed docstring in readme's example

* Corrected text

* move construct_gp_kernels to smac.epm_gpytorch.boing_kernels

* maint

* fix wrongly imported func name in tests

* Add reference for pSMAC #609

* Rename example file

* Added roadmap to readme

* Prettification

* Add result merger

* Reformat with black

* Delete unnecessary import

* Fix typing

* Inject first trajectory entry from file from first rundir

* Add test for ResultMerger

* Fix docstring

* Get runhistory and trajectory from pSMAC run

Via the result merger. Searchers rundirs and
merges found runhistories and calculates a
new trajectory.

* Get trajectory

* Add regex to requirements

* Fixed MO cache problem

* Fix MO tests

* Fixing MO in AbstractRacer

* Accepting ints

* Hacking in the sum cost for MO

* Fix mypi

* Bugfix

* Make mypi happy

* Fix tests

* Make format

* Remove num_obj in MO optimizer

* A lot of restructuring

* Solve importings

* Solve importings

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Updated changelog

* Still fixing :)

* fix docstrings

* Change directory of tests

* Fix ambiguous variable

* Del newlines

* fix mypy

* mypy and doc fix

* Removed int type checking

* fix tests

* Always return List of incs

* fix path issue in example

* maint boing facade

* Better docstring

* Return values if no bounds passed

* Removed import

* Return only one incumbent

* Rename n_optimizers -> n_workers

* Update example

* Update docstring

* Remove legacy function

* Update to new signature

* Update changelog.md

* Format black

* Update test

* Please pre-commit

* Merge development

* Fix tests

* Fix typos

* Format black

* Updated text

* Increased version

* Improved readme

* SMAC logo 50%

* SMAC logo 50%

* Removed dev-2.0

* Added doc workflow for latest tag

* Fixing docs latest tag

* Fixing docs latest tag

* First steps new intensifier

* Make it work again

* MOVE random weight initialization to smbo ask method

* ADD enable parego in unittests

* Test issues

* FIX formatting

* Link to old documentation

* MOVE random weight initialization to smbo ask method

* ADD enable parego in unittests

* FIX formatting

* Fix unit tests

* Add annotation

* corrected log prob for tophat prior

* Removed configurations from callback

* Fixed tests

* Fix mypy

* Save docs as latest version too

* Enabled all tests again

* Make version an env

* Added last version to docs

* Global variable fix

* Added v before version number

* New Intensifier Implementation (#888)

- Completely reimplemented the intensifiers (including Successive Halving and Hyperband): All intensifiers support multi-fidelity, multi-objective and multi-threading by nature now.
- Expected behaviour for ask-and-tell interface ensured (also for Successive Halving).
- Continuing a run is now fully supported.
- Added more examples.
- Updated documentation based on new implementation.

* First benchmark steps

* Added a warning

* Set xi to 0.01

* Incorporated walltime in trajectory

* Working version of benchmark

* Changed requirements

* Changed xi back to 0.0 again

* No duplicate budgets

* Fixed bug which updated tracker wrongly

* Bugfix: Delete correct bracket+stage

* Fixed tests

* Added costs to the trajectory

* Changed initial design max ratio to 0.25

* Make format

* Added more details to intensifier

* Finished benchmark

* Added benchmark source files

* Fixed tests because of initial design changes

* Improved runhistory API

* Fixed typo (#894)

* Merge doc fixes

* Doc fixes (#893)

Co-authored-by: timruhkopf <timruhkopf@gmail.com>
Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>
Co-authored-by: dengdifan <difandeng@gmail.com>
Co-authored-by: Alexander Tornede <tornede@tnt.uni-hannover.de>
Co-authored-by: Sarah Krebs <s.krebs@ai.uni-hannover.de>

* Make format

* Satisfied mypy

* Enabled more tests

* Increased version

* Mypy

* Mypy

* Mypy

* Fixed critical bug when using highest budget for inc selection

* Added SH+HB comparison

* Stuff

* Updated report

* Fixed docstring

* Fixed image path

* Removed vscode folder

* Bugfixes

* Doc fixes

* Updated changelog

* Make isort

* Fix typos

* Removed new line

* Added alpha2 to docs selection

* update description

* Fix 531 correctly create integers in initial design

This PR fixes a bug in which integer hyperparameters would not
be created correctly in the initial design. This went
unnoticed, except in the case where it was part of a condition
and the function deactivate_inactive_hyperparameters was
called.

This PR fixes this bug by mapping the random float that was
sampled by, for example and LHD or Sobol sequence, to the
float representation of an integer by calling _transform and
_inverse_transform of the integer hyperparameter.

* Add contact paragraph to README.

* Increase days until stale issues are closed by stalebot (#904)

* Edit changelog

* Add issue number

* Closes #907

* Clarify origin of configurations (#908)

* Random forest speed up (#903)

Co-authored-by: René Sass <mail@renesass.de>

* Add: workflow to update pre-commit versions (#874)

Co-authored-by: René Sass <mail@renesass.de>

* Add Git-Flow Workflow Specification

* [DOC] acquisition function maximizers

* Add stale reminder email action

* Increase version number of ConfigSpace

* Add guideline for updating copyright notice years to CONTRIBUTING.md

* Update requirement for pyrfr (#920)

* Update pre-commit: isort,flake8 version numbers

* Format black

* Fix mypy

* Ac bench (#929)

* Add Branin AC Task

* Add ac optimization type

* Nicer plotting

* Add ac mode

* Update ac task

* Make new smac env name adjustable

* Increase python version for smac 1.4

* Update benchmark

* Update plotting

* Goodby comments

* Fix path

* Fix 1.4 wrapper

* Update benchmark results

---------

Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>

* [DOC] added documentation on the incumbent update

* Feature/add version to make publish (#931)

* Add version, correct package name and use bash in makefile

* Increase version

* Increase version in makefile, add places with versions to make publish

* Fix typo

* Transform integer feature names to strings (#930)

* Transform int features to str

* Fix pre-commit

* Update version to 2.0.0

* Update version to 2.0.0

* Update CHANGELOG.md

* Add information on deepcave to the readme

* Correct doc version

---------

Co-authored-by: dengdifan <33290713+dengdifan@users.noreply.github.com>
Co-authored-by: dengdifan <difandeng@gmail.com>
Co-authored-by: Matthias Feurer <feurerm@informatik.uni-freiburg.de>
Co-authored-by: Carolin Benjamins <benjamin@tnt.uni-hannover.de>
Co-authored-by: eddiebergman <eddiebergmanhs@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>
Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com>
Co-authored-by: Katharina Eggensperger <eggenspk@informatik.uni-freiburg.de>
Co-authored-by: Eric Kalosa-Kenyon <ekquasar@appdynamics.com>
Co-authored-by: Pascal <66594767+PascalIversen@users.noreply.github.com>
Co-authored-by: helegraf <Graf.Helena@web.de>
Co-authored-by: timruhkopf <timruhkopf@gmail.com>
Co-authored-by: Alexander Tornede <tornede@tnt.uni-hannover.de>
Co-authored-by: Sarah Krebs <s.krebs@ai.uni-hannover.de>
Co-authored-by: Alexander Tornede <a.tornede@ai.uni-hannover.de>
Co-authored-by: Alexander Tornede <44395781+alexandertornede@users.noreply.github.com>
Co-authored-by: Sarah Krebs <30651149+sarah-krebs@users.noreply.github.com>
Co-authored-by: helegraf <h.graf@ai.uni-hannover.de>
Co-authored-by: Jeroen Rook <jrook94@gmail.com>
Co-authored-by: Tim Ruhkopf <timruhkopf@googlemail.com>

* feat: Option preventing SMAC setting up logging

* fix: Run isort

* Bump peter-evans/create-pull-request from 3 to 5 (#958)

Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 3 to 5.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/v3...v5)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump actions/setup-python from 2 to 4 (#945)

Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2 to 4.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2...v4)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* feat: Option preventing SMAC setting up logging

* fix: Run isort

* fix: Add to changelog

* feat: Option preventing SMAC setting up logging

* fix: Run isort

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: René Sass <mail@renesass.de>
Co-authored-by: dengdifan <33290713+dengdifan@users.noreply.github.com>
Co-authored-by: dengdifan <difandeng@gmail.com>
Co-authored-by: Matthias Feurer <feurerm@informatik.uni-freiburg.de>
Co-authored-by: Carolin Benjamins <benjamin@tnt.uni-hannover.de>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>
Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com>
Co-authored-by: Katharina Eggensperger <eggenspk@informatik.uni-freiburg.de>
Co-authored-by: Eric Kalosa-Kenyon <ekquasar@appdynamics.com>
Co-authored-by: Pascal <66594767+PascalIversen@users.noreply.github.com>
Co-authored-by: helegraf <Graf.Helena@web.de>
Co-authored-by: timruhkopf <timruhkopf@gmail.com>
Co-authored-by: Alexander Tornede <tornede@tnt.uni-hannover.de>
Co-authored-by: Sarah Krebs <s.krebs@ai.uni-hannover.de>
Co-authored-by: Alexander Tornede <a.tornede@ai.uni-hannover.de>
Co-authored-by: Alexander Tornede <44395781+alexandertornede@users.noreply.github.com>
Co-authored-by: Sarah Krebs <30651149+sarah-krebs@users.noreply.github.com>
Co-authored-by: helegraf <h.graf@ai.uni-hannover.de>
Co-authored-by: Jeroen Rook <jrook94@gmail.com>
Co-authored-by: Tim Ruhkopf <timruhkopf@googlemail.com>

* Citation update (#961)

* update CITATION.cff

* update order in CITATION.cff

* fix citation date

* fix citation

* maint

* Adjust hyperband configuration distribution across brackets

* Compute the actual differences between the isb keys. (#957)

* Compute the actual differences between the isb keys.

* Added change to CHANGELOG.md

* Adjust schedule for stale reminder

* fix(logging): Prevent automatic logging setup at init (#970)

* fix(logging): Prevent automatic logging setup at init

* fix(logging): Rely on `setup_logging` to handle arg

* Fix validate method of smbo, update docstring.

* Allow callbacks to be added to a specific index, make callback registering public

* Update CHANGELOG.md

* Fix broken references (#988)

* Adapt developer install instructions to include pre-commit installation (#994)

* [Feature] Initial Design now supports executing the default config provided by the user by means of the configspace he generated. (and is no longer ignored)

* Pipe dask_client through facade for exec on hpc (#983)

* Pipe dask_client through facade for exec on hpc

* Handle pynisher's memory arg

Before: Could only pass int via SMAC with the default unit B, but now tuple/list is possible

* Update CHANGELOG.md

* Make format

* No printi

---------

Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>

* [Bug-fix] Pr was failing due to mutable additional configs default

* [doc-fix] the count of initial design configs is computed slightly differently

* Feature/metadata callback (#999)

* Add example for using a callback to log run metadata to a file

* Add issue number in changelog

* Changelog formatting

* Simplify metadata example

* Simplify metadata example

* Put callback in smac directory to allow importing it in custom repos

* Move callback files to callback directory

* Edit changelog

* Correct metadata callback argument types

* Additional metadata arguments in example

* Set metadata default values in callback

* Edit comment to fix PR

* Documentation/dask client example (#1001)

* Update requirements (dask_jobqueue)

* Fix broken link

* Ignore parallel example

* Add parallelization example

* Remove imports, add comment abt dask client

* Add E's information

* Update docs

* Add comment about n_workers

---------

Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>

* Documentation/dask client example (#1001)

* Update requirements (dask_jobqueue)

* Fix broken link

* Ignore parallel example

* Add parallelization example

* Remove imports, add comment abt dask client

* Add E's information

* Update docs

* Add comment about n_workers

---------

Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>

* Feature/dask client (#1002)

* Add info message that scenario.n_workers is ignored

When passing a dask client,
directly set n_workers in client.

* Change info to warning

---------

Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>

* Update smac version (#1003)

* Update smac version

* Add hint on github release in make publish

* Update copyright year

* check if config in rh when storing state (#997)

* check if config in rh when storing state

* maint

* remove unnecessary changes

* Adapt developer install instructions to include pre-commit installation (#994)

* Pipe dask_client through facade for exec on hpc (#983)

* Pipe dask_client through facade for exec on hpc

* Handle pynisher's memory arg

Before: Could only pass int via SMAC with the default unit B, but now tuple/list is possible

* Update CHANGELOG.md

* Make format

* No printi

---------

Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>

* update changelog

* Increase tolerance for runhistory restoration test

---------

Co-authored-by: Sarah Segel <30651149+sarah-segel@users.noreply.github.com>
Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com>
Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>
Co-authored-by: Helena Graf <h.graf@ai.uni-hannover.de>

* Add a workaround to be able to pass a dataset via dask.scatter (#993)

* [feat] Support dask.scatter

* [fix] Add shared_data arg in abstract_facade

* Address the comments by Eddie

* [fix] Fix errors raised by Eddie

* [format] Apply black locally

* [fix] Get retries back to 16 as it was a mistake

* [doc] Update CHANGELOG

---------

Co-authored-by: helegraf <h.graf@ai.uni-hannover.de>
Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com>

* Adding Binder links for example (#976)

* binder

* maint

* fix dependencies

* maint docs

* Revert "maint docs"

This reverts commit 4fa1a0fae8dfa0b646e43273d50f72ddd76bf191.

* add requirement

---------

Co-authored-by: Difan Deng <deng@dengster.tnt.uni-hannover.de>

* Remove commit message from dependabot PRs (#1009)

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dengdifan <difandeng@gmail.com>
Co-authored-by: René Sass <mail@renesass.de>
Co-authored-by: Carolin Benjamins <benjamin@tnt.uni-hannover.de>
Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>
Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com>
Co-authored-by: Katharina Eggensperger <eggenspk@informatik.uni-freiburg.de>
Co-authored-by: Matthias Feurer <feurerm@informatik.uni-freiburg.de>
Co-authored-by: Eric Kalosa-Kenyon <ekquasar@appdynamics.com>
Co-authored-by: Pascal <66594767+PascalIversen@users.noreply.github.com>
Co-authored-by: helegraf <Graf.Helena@web.de>
Co-authored-by: timruhkopf <timruhkopf@gmail.com>
Co-authored-by: Alexander Tornede <tornede@tnt.uni-hannover.de>
Co-authored-by: Alexander Tornede <a.tornede@ai.uni-hannover.de>
Co-authored-by: Alexander Tornede <44395781+alexandertornede@users.noreply.github.com>
Co-authored-by: Sarah Krebs <30651149+sarah-krebs@users.noreply.github.com>
Co-authored-by: helegraf <h.graf@ai.uni-hannover.de>
Co-authored-by: Jeroen Rook <jrook94@gmail.com>
Co-authored-by: Eddie Bergman <eddiebergmanhs@gmail.com>
Co-authored-by: Tim Ruhkopf <timruhkopf@googlemail.com>
Co-authored-by: Difan Deng <33290713+dengdifan@users.noreply.github.com>
Co-authored-by: Aron Bahram <aron.bahram@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Shuhei Watanabe <47781922+nabenabe0928@users.noreply.github.com>
Co-authored-by: Difan Deng <deng@dengster.tnt.uni-hannover.de>

* Fix broken link

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dengdifan <difandeng@gmail.com>
Co-authored-by: René Sass <mail@renesass.de>
Co-authored-by: Carolin Benjamins <benjamin@tnt.uni-hannover.de>
Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>
Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com>
Co-authored-by: Katharina Eggensperger <eggenspk@informatik.uni-freiburg.de>
Co-authored-by: Matthias Feurer <feurerm@informatik.uni-freiburg.de>
Co-authored-by: Eric Kalosa-Kenyon <ekquasar@appdynamics.com>
Co-authored-by: Pascal <66594767+PascalIversen@users.noreply.github.com>
Co-authored-by: helegraf <Graf.Helena@web.de>
Co-authored-by: timruhkopf <timruhkopf@gmail.com>
Co-authored-by: Alexander Tornede <tornede@tnt.uni-hannover.de>
Co-authored-by: Alexander Tornede <a.tornede@ai.uni-hannover.de>
Co-authored-by: Alexander Tornede <44395781+alexandertornede@users.noreply.github.com>
Co-authored-by: Sarah Krebs <30651149+sarah-krebs@users.noreply.github.com>
Co-authored-by: helegraf <h.graf@ai.uni-hannover.de>
Co-authored-by: Jeroen Rook <jrook94@gmail.com>
Co-authored-by: Eddie Bergman <eddiebergmanhs@gmail.com>
Co-authored-by: Tim Ruhkopf <timruhkopf@googlemail.com>
Co-authored-by: Difan Deng <33290713+dengdifan@users.noreply.github.com>
Co-authored-by: Aron Bahram <aron.bahram@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Shuhei Watanabe <47781922+nabenabe0928@users.noreply.github.com>
Co-authored-by: Difan Deng <deng@dengster.tnt.uni-hannover.de>

* Drop torch requirements (#1015)

* Remove torch requirements

* Fix format

---------

Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>

* Remove leftover CLI entry points (#1014)

* Fix incumbent selection in case of SH+MO

* Update CHANGELOG.md

* Feature/GitHub actions pr draft (#1008)

Update the workflows only to trigger extensive tests for PRs that are not in the draft stage

---------

Co-authored-by: timruhkopf <timruhkopf@gmail.com>
Co-authored-by: Helena Graf <h.graf@ai.uni-hannover.de>

* Update stale report to include issues with recent interactions (#1025)

* Add recently updated issues to list of reminders.

* Add recently updated issues to list of reminders.

* Add recently updated issues to list of reminders.

* Raise error if resource limitation and parallelization is requested (#1023)

* Add warnings for parallelism doc

* Check for parallelization + pynisher

Does not work together, throw error.
Does not work bc Dask pickles jobs and
pynisher runs the target function in a process.

* Fix whitespaces

---------

Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>

* Raise an error for invalid scatter data (#1018)

* [enhance] Add an error for scatter data

* Update ChangeLog

---------

Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com>

* fix(configspace): shows all keys in print (#1043)

* Fix callback order (#1040)

Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>

* fix: dtypes (#1044)

* Fix(config_selector): Budget checking (#1039)

* Fix: target runner with partial func (#1045)

* Feature/improve documentation (#1028)

* Experimental instruction for installing SMAC in windows via WSL

* More detailed documentation regarding continuing runs

* Fixes

* Fixes

* Fixes

* Update docs/10_experimental.rst

Co-authored-by: Difan Deng <33290713+dengdifan@users.noreply.github.com>

* Fixes from PR

* Fixes from PR

* Fixes from PR

* Fixes from PR

* Fixes from PR

---------

Co-authored-by: Difan Deng <33290713+dengdifan@users.noreply.github.com>

* Handle configspace as dictionary in mlp example (#1057)

* Handle configspace as dictionary in mlp example

* Adapt sgd loss to newest scikit-learn version

* Add change to changelog

* Handle configspace as dictionary in parego example

* Correct get usage

* Raise version number

* Add missing period in docs.

* allow rf to impute OrdinalHyperparameter (#1065)

* allow rf to impute OrdinalHyperparameter

* Add missing period in docs.

* Add issue number in changelog.

---------

Co-authored-by: Sarah Krebs <s.krebs@ai.uni-hannover.de>

* fix dask_scheduler_file path (#1055)

* fix dask_scheduler_file path

* update change log

---------

Co-authored-by: dengdifan <difandeng@gmail.com>

* Fix config rejection for #1068 (#1069)

* racing: make sure config only deleted from rejected configs if among incumbents upon update

* test case for incumbent rejection

* update code comments

* added missing newline ahead of new test case

* update change log

---------

Co-authored-by: dengdifan <difandeng@gmail.com>
Co-authored-by: Difan Deng <33290713+dengdifan@users.noreply.github.com>

* fix(RandomForest): Always cast `seed` to `int` (#1084)

* fix(RandomForest): Always cast `seed` to `int`

In cases where seeds were generated from some numpy objects, sometimes
you'd get back an `np.integer`, which causes the program to crash when
communicating with `pyrfr` through `swig`. It seems that swig doesn't
know that it's an _int-like_ and so we explicitly cast it to `int`.

* Update CHANGELOG.md

---------

Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com>

* Bump actions/checkout from 3 to 4 (#1072)

* Bump actions/checkout from 3 to 4

Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update CHANGELOG.md

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: benjamc <c.benjamins@ai.uni-hannover.de>

* chore: log warn on condition when custom dask client is provided (#1071)

* Bump conda-incubator/setup-miniconda from 2 to 3 (#1087)

Bumps [conda-incubator/setup-miniconda](https://github.com/conda-incubator/setup-miniconda) from 2 to 3.
- [Release notes](https://github.com/conda-incubator/setup-miniconda/releases)
- [Changelog](https://github.com/conda-incubator/setup-miniconda/blob/main/CHANGELOG.md)
- [Commits](https://github.com/conda-incubator/setup-miniconda/compare/v2...v3)

---
updated-dependencies:
- dependency-name: conda-incubator/setup-miniconda
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Propagate the Scenario random seed to get_random_design (#1066)

* Propagate the Scenario's random seed to get_random_design

* add changelog line

---------

Co-authored-by: Sarah Segel <30651149+sarah-segel@users.noreply.github.com>

* [#1056] Add example on intensify for cross-validation. (#1061)

* Add example on intensify for cross-validation.

* Improve example

* Address Difan's comment

* Update CHANGELOG.md

* Change stale-reminder to recent-reminder (#1096)

* Change stale-reminder to recent-reminder (#1095) (#1099)

* Dont use mutable lists as default args (#1097)

* Don't use mutable default argument.

If this isn't respected, then additional_configs can be built up
overtime when multiple optimizers are created in a single process.

This can occur when the use_default_config option is set True and/or
when additional_configs are provided.

When different ConfigSpaces are provided this can result in an error due
to mismatched defaults, for instance.

In other cases it can result in non-deterministic behavior due to the
random ordering of the configs that get added (e.g., with different
parallel pytest runs).

- [ ] Cleanup other sources of this error as well elsewhere in the code.
- [ ] Add pylint checks for this issue.

* fixup some others mutable defaults

* add pylint checks

* fix more related pylint issues

* add changelog line

* Update pre-commit config

* Update pre-commit config

* Remove TODO formulation as an issue already exists for this

---------

Co-authored-by: Brian Kroth <bpkroth@microsoft.com>

* Bump actions/setup-python from 4 to 5 (#1089)

Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 5.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump peter-evans/create-pull-request from 5 to 6 (#1093)

Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 5 to 6.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/v5...v6)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* test: Fix upper bounds for `test_transformer()` and no random in `test_transformer_conditional()` (#1102)

* test: Don't rely on randomization for the test

* test: Include import

* test: Fix boundaries for catgoricals in test

* test: Missing import

* doc: Make todo not more clear about when to upgrade

* Change stale-reminder to recent-reminder (#1095) (#1107)

* Fix typo in docstring for `MultiFideltyFacade.get_initial_design` (#1104)


* Fix typo in docstring for `MultiFideltyFacade.get_initial_design`

The docstring specifies 0.1 is used as a default value for max_ratio,
however in code 0.25 is used.

* Bug/acquisition maximizer sampling (#1106)

* Adapt how n_points works in the config_selector.py

Also remove LocalAndSortedPriorRandomSearch by integrating it into LocalAndSortedRandomSearch

* Change retrain_after to 1 for BlackBOx Facade

* Fix bugs in local_and_random_search.py

* Edit CHANGELOG.md

* fix acq maximizer docstring

* call get config selector from super class

* Update random and local search to new functionality

* Update CHANGELOG.md

* Fix sampling of initial points for local search when less previous configs than n_points

* Re-format blackbo…
  • Loading branch information
1 parent 9aaa8e9 commit 937eb2c
Show file tree
Hide file tree
Showing 41 changed files with 552 additions and 310 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/citation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Check out a copy of the repository
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Check whether the citation metadata from CITATION.cff is valid
uses: citation-file-format/cffconvert-github-action@2.0.0
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/dist.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@ jobs:

steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Setup Python
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: "3.10"

Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Setup Python
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: "3.10"

Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/pre-commit-update.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,17 @@ jobs:
auto-update:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- uses: actions/setup-python@v4
- uses: actions/setup-python@v5

- uses: browniebroke/pre-commit-autoupdate-action@main

- name: Run updated pre-commit
run: |
pre-commit run --all-files
- uses: peter-evans/create-pull-request@v5
- uses: peter-evans/create-pull-request@v6
with:
token: ${{ secrets.GITHUB_TOKEN }}
branch: update/pre-commit-hooks
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/pre-commit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,19 +27,20 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
submodules: recursive

- name: Setup Python 3.10
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: "3.10"

- name: Install pre-commit
run: |
pip install pre-commit
pre-commit install
pip install pylint
- name: Run pre-commit
run: |
Expand Down
12 changes: 6 additions & 6 deletions .github/workflows/pytest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,10 +54,10 @@ jobs:

steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Setup Python
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

Expand Down Expand Up @@ -108,10 +108,10 @@ jobs:

steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Conda install
uses: conda-incubator/setup-miniconda@v2
uses: conda-incubator/setup-miniconda@v3
with:
auto-update-conda: true
python-version: ${{ matrix.python-version }}
Expand Down Expand Up @@ -148,10 +148,10 @@ jobs:

steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Setup Python
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

Expand Down
13 changes: 13 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,16 @@ repos:
name: flake8 smac
files: smac
exclude: "scripts|tests"

- repo: local
hooks:
- id: pylint
name: pylint
entry: pylint
language: system
types: [python]
args:
[
"-rn", # Only display messages
"-sn", # Don't display the score
]
30 changes: 29 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,37 @@
# 2.1.0

## Improvements
- Change the surrogate model to be retrained after every iteration by default in the case of blackbox optimization
(#1106).
- Integrate `LocalAndSortedPriorRandomSearch` functionality into `LocalAndSortedRandomSearch` (#1106).
- Change the way the `LocalAndSortedRandomSearch` works such that the incumbent always is a starting point and that
random configurations are sampled as the basis of the local search, not in addition (#1106).

## Bugfixes
- Fix path for dask scheduler file (#1055).
- Add OrdinalHyperparameter for random forest imputer (#1065).
- Don't use mutable default argument (#1067).
- Propagate the Scenario random seed to `get_random_design` (#1066).
- Configurations that fail to become incumbents will be added to the rejected lists (#1069).
- SMAC RandomForest doesn't crash when `np.integer` used, i.e. as generated from a `np.random.RandomState` (#1084).
- Fix the handling of n_points/ challengers in the acquisition maximizers, such that this number now functions as the
number of points that are sampled from the acquisition function to find the next challengers. Now also doesn't
restrict the config selector to n_retrain many points for finding the max, and instead uses the defaults that are
defined via facades/ scenarios (#1106).

## Misc
- ci: Update action version (#1072).

## Minor
- When a custom dask client is provided, emit the warning that the `n_workers` parameter is ignored only if it deviates from its default value, `1` ([#1071](https://github.com/automl/SMAC3/pull/1071)).

# 2.0.2

## Improvements
- Add an error when we get an empty dict data_to_scatter so that we can avoid an internal error caused in Dask precautiously
- Add an error when we get an empty dict data_to_scatter so that we can avoid an internal error caused in Dask precautiously.
- Add experimental instruction for installing SMAC in Windows via a WSL.
- More detailed documentation regarding continuing runs.
- Add a new example that demonstrates the use of intensification to speed up cross-validation for machine learning.

## Bugfixes
- Fix bug in the incumbent selection in the case that multi-fidelity is combined with multi-objective (#1019).
Expand Down
2 changes: 1 addition & 1 deletion CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ date-released: "2016-08-17"
url: "https://automl.github.io/SMAC3/master/index.html"
repository-code: "https://github.com/automl/SMAC3"

version: "2.0.2"
version: "2.1.0"

type: "software"
keywords:
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ SHELL := /bin/bash

NAME := SMAC3
PACKAGE_NAME := smac
VERSION := 2.0.2
VERSION := 2.1.0

DIR := "${CURDIR}"
SOURCE_DIR := ${PACKAGE_NAME}
Expand Down
2 changes: 1 addition & 1 deletion benchmark/src/wrappers/v20.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@


class Version20(Wrapper):
supported_versions: list[str] = ["2.0.2"]
supported_versions: list[str] = ["2.1.0"]

def __init__(self, task: Task, seed: int) -> None:
super().__init__(task, seed)
Expand Down
127 changes: 127 additions & 0 deletions examples/4_advanced_optimizer/4_intensify_crossvalidation.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
"""
Speeding up Cross-Validation with Intensification
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
An example of optimizing a simple support vector machine on the digits dataset. In contrast to the
[simple example](examples/1_basics/2_svm_cv.py), in which all cross-validation folds are executed
at once, we use the intensification mechanism described in the original
[SMAC paper](https://link.springer.com/chapter/10.1007/978-3-642-25566-3_40) as also demonstrated
by [Auto-WEKA](https://dl.acm.org/doi/10.1145/2487575.2487629). This mechanism allows us to
terminate the evaluation of a configuration if after a certain number of folds, the configuration
is found to be worse than the incumbent configuration. This is especially useful if the evaluation
of a configuration is expensive, e.g., if we have to train a neural network or if we have to
evaluate the configuration on a large dataset.
"""
__copyright__ = "Copyright 2023, AutoML.org Freiburg-Hannover"
__license__ = "3-clause BSD"

N_FOLDS = 10 # Global variable that determines the number of folds

from ConfigSpace import Configuration, ConfigurationSpace, Float
from sklearn import datasets, svm
from sklearn.model_selection import StratifiedKFold

from smac import HyperparameterOptimizationFacade, Scenario
from smac.intensifier import Intensifier

# We load the digits dataset, a small-scale 10-class digit recognition dataset
X, y = datasets.load_digits(return_X_y=True)


class SVM:
@property
def configspace(self) -> ConfigurationSpace:
# Build Configuration Space which defines all parameters and their ranges
cs = ConfigurationSpace(seed=0)

# First we create our hyperparameters
C = Float("C", (2 ** - 5, 2 ** 15), default=1.0, log=True)
gamma = Float("gamma", (2 ** -15, 2 ** 3), default=1.0, log=True)

# Add hyperparameters to our configspace
cs.add_hyperparameters([C, gamma])

return cs

def train(self, config: Configuration, instance: str, seed: int = 0) -> float:
"""Creates a SVM based on a configuration and evaluate on the given fold of the digits dataset
Parameters
----------
config: Configuration
The configuration to train the SVM.
instance: str
The name of the instance this configuration should be evaluated on. This is always of type
string by definition. In our case we cast to int, but this could also be the filename of a
problem instance to be loaded.
seed: int
The seed used for this call.
"""
instance = int(instance)
classifier = svm.SVC(**config, random_state=seed)
splitter = StratifiedKFold(n_splits=N_FOLDS, shuffle=True, random_state=seed)
for k, (train_idx, test_idx) in enumerate(splitter.split(X=X, y=y)):
if k != instance:
continue
else:
train_X = X[train_idx]
train_y = y[train_idx]
test_X = X[test_idx]
test_y = y[test_idx]
classifier.fit(train_X, train_y)
cost = 1 - classifier.score(test_X, test_y)

return cost


if __name__ == "__main__":
classifier = SVM()

# Next, we create an object, holding general information about the run
scenario = Scenario(
classifier.configspace,
n_trials=50, # We want to run max 50 trials (combination of config and instances in the case of
# deterministic=True. In the case of deterministic=False, this would be the
# combination of instances, seeds and configs). The number of distinct configurations
# evaluated by SMAC will be lower than this number because some of the configurations
# will be executed on more than one instance (CV fold).
instances=[f"{i}" for i in range(N_FOLDS)], # Specify all instances by their name (as a string)
instance_features={f"{i}": [i] for i in range(N_FOLDS)}, # breaks SMAC
deterministic=True # To simplify the problem we make SMAC believe that we have a deterministic
# optimization problem.

)

# We want to run the facade's default initial design, but we want to change the number
# of initial configs to 5.
initial_design = HyperparameterOptimizationFacade.get_initial_design(scenario, n_configs=5)

# Now we use SMAC to find the best hyperparameters
smac = HyperparameterOptimizationFacade(
scenario,
classifier.train,
initial_design=initial_design,
overwrite=True, # If the run exists, we overwrite it; alternatively, we can continue from last state
# The next line defines the intensifier, i.e., the module that governs the selection of
# instance-seed pairs. Since we set deterministic to True above, it only governs the instance in
# this example. Technically, it is not necessary to create the intensifier as a user, but it is
# necessary to do so because we change the argument max_config_calls (the number of instance-seed pairs
# per configuration to try) to the number of cross-validation folds, while the default would be 3.
intensifier=Intensifier(scenario=scenario, max_config_calls=N_FOLDS, seed=0)
)

incumbent = smac.optimize()

# Get cost of default configuration
default_cost = smac.validate(classifier.configspace.get_default_configuration())
print(f"Default cost: {default_cost}")

# Let's calculate the cost of the incumbent
incumbent_cost = smac.validate(incumbent)
print(f"Incumbent cost: {incumbent_cost}")

# Let's see how many configurations we have evaluated. If this number is higher than 5, we have looked
# at more configurations than would have been possible with regular cross-validation, where the number
# of configurations would be determined by the number of trials divided by the number of folds (50 / 10).
runhistory = smac.runhistory
print(f"Number of evaluated configurations: {len(runhistory.config_ids)}")
5 changes: 5 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,11 @@ add-ignore = [ # http://www.pydocstyle.org/en/stable/error_codes.html
"D415", # First line should end with a period, question mark, or exclamation point
]

[tool.pylint."messages control"]
# FIXME: This is to do a staged introduction of pylint checks for just a single class of problems initially (#1067).
disable = ["all"]
enable = ["dangerous-default-value"]

[tool.mypy]
python_version = "3.9"
show_error_codes = true
Expand Down
1 change: 1 addition & 0 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ def read_file(filepath: str) -> str:
"pydocstyle",
"flake8",
"pre-commit",
"pylint",
],
}

Expand Down
2 changes: 1 addition & 1 deletion smac/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
Copyright {datetime.date.today().strftime('%Y')}, Marius Lindauer, Katharina Eggensperger,
Matthias Feurer, André Biedenkapp, Difan Deng, Carolin Benjamins, Tim Ruhkopf, René Sass
and Frank Hutter"""
version = "2.0.2"
version = "2.1.0"


try:
Expand Down
2 changes: 0 additions & 2 deletions smac/acquisition/maximizer/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
)
from smac.acquisition.maximizer.differential_evolution import DifferentialEvolution
from smac.acquisition.maximizer.local_and_random_search import (
LocalAndSortedPriorRandomSearch,
LocalAndSortedRandomSearch,
)
from smac.acquisition.maximizer.local_search import LocalSearch
Expand All @@ -13,7 +12,6 @@
"AbstractAcquisitionMaximizer",
"DifferentialEvolution",
"LocalAndSortedRandomSearch",
"LocalAndSortedPriorRandomSearch",
"LocalSearch",
"RandomSearch",
]
Loading

0 comments on commit 937eb2c

Please sign in to comment.