Getting visualizations from saved report taking very long

[gnomesanta]

Environment details

• SDMetrics version: 0.11.1
• Python version: 3.10.6
• Operating System: Ubuntu 22.04.1

Problem description

Hi, I already have a saved QualityReport after generating my synthetic data. The QualityReport pkl file is about 40MB.

I am trying to extract the visualization for Column Pair Trends. But the function is taking very long to get the visualization. I am not sure if this is normal, or if it might be an issue with the method?

What I already tried

x = report.get_visualization('Column Pair Trends')
x.write_image('test1.png')

[npatki]

Hi @gnomesanta, Could you describe a little more about the dataset you're using? Is it single table or multi table? How many columns are there in each table?

The Column Pair Trends property evaluates the correlation across every pair of columns in a table. So if a table has a lot of columns -- say 500 -- then you're looking at 250K pairwise comparisons. This would explain the size of the report and the time taken for visualization.

(Rest assured that the report is not saving the actual data. Only the scores.)