Refactor out policy evaluation to separate class #364
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
boomanaiden154
force-pushed
the
users/boomanaiden154/blackbox-evaluator
branch
2 times, most recently
from
5cdcfbf
to
ad552a2
Compare
mtrofin
reviewed
Sep 13, 2024
boomanaiden154
force-pushed
the
users/boomanaiden154/blackbox-evaluator
branch
from
ad552a2
to
d565f23
Compare
mtrofin
approved these changes
Sep 13, 2024
boomanaiden154
changed the base branch from
users/boomanaident154/whole-corpus-sampler
to
main
This patch refactors out policy evaluation in blackbox_learner to a new BlackboxEvaluator class so that we can change some details of how things are collected, particularly with regards to sampling and how samples are held.
This patch makes two primary changes to the CI config. It restricts the CI running on push events to only run on the main branch. This prevents the CI from running twice on PRs from branches in the main repository, which are needed for stacked PRs. Additionally, this PR sets the permissions at the top of the file, which is a general best practice for Github workflows for security reasons. Additionally, bump a couple of the actions that were using deprecated NodeJS versions so we do not get the warning anymore.
This patch deletes es_worker.py, which is not used anywhere, even for testing. An exact copy of the class is in blackbox_learner_test.py for testing purposes.
This patch replaces load_corpus_element with the (presumably) correct function name, load_module_spec, which is defined directly below. This seems to have been an oversight during a refactoring at some point.
Pytype complains that attribute errors are not reenabled after a temporary disable. This patch fixes that be reenabling them so that it stops throwing the warning while also potentially improving coverage of the code if more is added in the future.
boomanaiden154
force-pushed
the
users/boomanaiden154/blackbox-evaluator
branch
from
ea3bb06
to
993b996
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This patch refactors out policy evaluation in blackbox_learner to a new BlackboxEvaluator class so that we can change some details of how things are collected, particularly with regards to sampling and how samples are held.