Should dvc metrics diff with --precision X shows differences < X?

[courentin]

We have a lot of metrics in my team, quite often we use the output of dvc metrics diff --precision 2 in PR.

However, we have a lot of micro changes that are inferior to the desired precision, which pollutes the output with a lot of insignificant diffs like that:

Path	Metric	Old	New	Change
metrics.json	F1_CORRECT_conf	0.008	0.008	-0.0

So, my question is should dvc metrics diff with --precision X shows differences < X?

[karajan1001]

Using - for unchanged values instead of 0.0?

The same with dvc exp diff.

[courentin]

I'd say not showing them in the diff since we don't consider them as changed if their diff is inferior to the precision specified, what do you think?

[dberenbaum]

I had thought of --precision as purely to truncate outputs until now, but you make a good case that it might be a core part of deciding which metrics to return. It's an interesting scenario since I can see users who may want it either way (maybe it's important to see that a metric changed to validate that two commits don't have identical results, even if it's fine for that change to show as 0.0).

I have a bias towards keeping it as is because:

• We are likely to keep moving in the direction of dvc exp diff where we combine diffs of different files, and --precision might not always make sense (for diffs that aren't comparing scalar numeric values).
• Including more metrics seems like a good default since users can always grep or otherwise filter down the results as needed.