Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Data Version Control (DVC) for tracking mapping experiments #159

Open
kursatyurt opened this issue Nov 18, 2022 · 1 comment
Open

Add Data Version Control (DVC) for tracking mapping experiments #159

kursatyurt opened this issue Nov 18, 2022 · 1 comment

Comments

@kursatyurt
Copy link
Collaborator

ASTE is used to measure numerical and performance metrics of preCICE.

One of the goals is having a reproducible experiment environment and allow everybody to share their results. It can be also used in development phase to measure how the changes affect metrics.

DVC is open-source system for data management. It is mainly designed for Machine Learning, however applicable to cases where huge amount of data plays role.

https://github.com/kursatyurt/aste/tree/dvc_v2 has a reference usage of DVC for ASTE experiments and looking for your feedback to how to introduce DVC to ASTE pipeline.

The DVC pipeline which is similar to github action pipelines can be seen in https://github.com/kursatyurt/aste/blob/dvc_v2/dvc.yaml

We are looking for your feedback 🥰

@fsimonis
Copy link
Member

The plotting capabilities in DVC are really minimal.
One needs to specify X and Y for each series.
So we need a custom config for each experiment based on the test series.

For each plot, we need:

  • a dvc configuration specifying mesh width (X) and series (Ys)
  • a CSV with mesh width;NN;NP;TSP

We can easily generate these from the result file after gathering results from the test runs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants