-
-
Notifications
You must be signed in to change notification settings - Fork 143
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Invalid estimation procedure ids #1249
Comments
Hi @ArturDev42, Thank you very much for raising this issue. Yes, such lists are available, but very hidden and only accessible by using the API: As you can see, these estimation procedures vary quite a bit from the live to the test server. To ultimately improve the situation, do you think it would make sense to rather create an API call that returns the estimation procedures? Then one can look up the current definitions dynamically. And to answer your question, no, unfortunately, this is not possible. |
Description
On https://www.openml.org/search?type=measure&measure_type=estimation_procedure I can see the different estimation procedure ids. When creating a new task on the test server as follows
with for example
estimation_procedure_id=11
which according to the list corresponds to '10% Holdout set', the following error is thrown:If instead I use for example
estimation_procedure_id=25
, I receive the error as described in openml/OpenML#1190. My used version are the same as mentioned in openml/OpenML#1190.Improvement of docs
I think it would be great to show the list of possible estimation procedures in a tutorial somewhere. I noticed that such a list exists from the comment mentioned in https://openml.github.io/openml-python/develop/examples/30_extended/task_manual_iteration_tutorial.html but I only by chance noticed that I need to click on 'Show list' to see the procedure ids. Would also be great to be able to query that information via the API.
Is it also possible to manually create new estimation procedures?
Thanks!
The text was updated successfully, but these errors were encountered: