You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
While OpenML invests a lot of effort into providing high-quality datasets and benchmarks, this is not always visible. This is an initial summary of a short brainstorm on how to further improve the way OpenML is perceived in the community, also compared to other platforms. The goal of this thread is to discuss this further and decide on concrete actions. We would be very happy to help anyone in the community to pick up specific ideas to work on, and we warmly welcome any other ideas or feedback.
Website
Many other platforms have a number of nice features that make it more easily accessible:
Trending datasets (based on novelty and likes), which is an inviting way for people to discover new interesting datasets. OpenML could easily add something like this to the website.
Popular topics: OpenML could also add this. We are currently working on learning the topics automatically, but we should also show them more clearly on the website.
Bring back liking of datasets,tasks,... Currently, this is not implemented on the new website.
Dataset explorer (quick view of the data table, should be easy to do in Dash)
Code snippets on how to load datasets and reproduce runs
Data quality
OpenML datasets are generally high quality but not always perceived as such. Ways to improve this:
Do a big push, with help from students, in improving the dataset descriptions and meta-data
Have a 'human verified' badge for datasets that where checked by a human.
Have a clear (human-verified) synthetic / real badge (not just a tag)
Add features on the website to allow users to improve data quality. E.g. improve descriptions, fix metadata. Give users a reward for doing this, e.g. with a reputation/activity score as in Stack Overflow
Create data quality bots that automatically create a data quality report for datasets and post that on the website.
Dataset discussion boards on the website
Make an official 'Editor' role: people who we trust to change datasets and that have more access to do so.
Add 'datasheets' with structured metadata
Add a 'Dataset review' feature: structured reviews of datasets
List original dataset paper, and/or list datasets in which the dataset is used.
Community visibility
Build trust together with the community:
Add testimonials on the website from well-known machine learning people praising OpenML or highlighting use cases
Join forces with Kaggle/HuggingFace. E.g. map OpenML datasets to equivalents on Kaggle. Link to Kaggle and ask Kaggle to link back to OpenML, e.g. to get more data, see benchmarks,... Find a way to discover matching datasets.
Do an open dataset study (maybe together with Kaggle, HuggingFace) that reviews datasets wrt quality, realism, signal for benchmarking
More and better curated OpenML benchmarking suites, create more excitement about them
Auto-generate notebooks and link them to OpenML datasets, e.g. running a standard dataset analysis or benchmark, so that people can more easily reproduce them.
Allow people to contribute their own notebooks, or link to Kaggle kernels
Make an overview of papers that use OpenML, allow users to add their papers to this list. Make it nice and organized.
Make it more clear that OpenML runs are reproducible, e.g. by generating code/notebooks to do this.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
While OpenML invests a lot of effort into providing high-quality datasets and benchmarks, this is not always visible. This is an initial summary of a short brainstorm on how to further improve the way OpenML is perceived in the community, also compared to other platforms. The goal of this thread is to discuss this further and decide on concrete actions. We would be very happy to help anyone in the community to pick up specific ideas to work on, and we warmly welcome any other ideas or feedback.
Website
Many other platforms have a number of nice features that make it more easily accessible:
Data quality
OpenML datasets are generally high quality but not always perceived as such. Ways to improve this:
Community visibility
Build trust together with the community:
Beta Was this translation helpful? Give feedback.
All reactions