You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The pyutilib.testdriver should allow specifying multiple sets of options for a solver. It would then run a "full-factorial" suite of tests. For example:
This is on the wishlist because this capability has lots of difficult implications, including how it gets implemented, how fractional specifications are managed (in the 'tests' block), and how tests get named. If this was spelled out in more detail, I'd be willing to look into this. However, this might be a nice extension for John or Dave to dig into to help familiarize themselves with this code.
It is an open question how to consistently name the individual tests.
The text was updated successfully, but these errors were encountered:
The pyutilib.testdriver YAML input specification could support a section that allows us to specify a subset of the full factorial set of tests. For example:
testsuites:
suite_a:
solvers:
solver_a:
solver_b:
problems:
problem_1:
problem_2:
tests:
- solver_a_problem_1
- solver_b_problem_2
would only run solver_a on problem_1 and solver_b on problem_2. Additionally,
testsuites:
suite_a:
solvers:
solver_a:
solver_b:
solver_c:
problems:
problem_1:
problem_2:
tests: */4
would run 1/4 of the full-factorial design (distributed in some statistically meaningful manner).
The pyutilib.testdriver should allow specifying multiple sets of options for a solver. It would then run a "full-factorial" suite of tests. For example:
would specify 14 tests.
This is on the wishlist because this capability has lots of difficult implications, including how it gets implemented, how fractional specifications are managed (in the 'tests' block), and how tests get named. If this was spelled out in more detail, I'd be willing to look into this. However, this might be a nice extension for John or Dave to dig into to help familiarize themselves with this code.
It is an open question how to consistently name the individual tests.
The text was updated successfully, but these errors were encountered: