Add metric for sequence length similarity

[fealho]

CU-86b2evqnr, Resolve #638.

[sdv-team]

Task linked: CU-86b2evqnr SDMetrics - Add metric for sequence length similarity #638

[R-Palazzo]

Looks good!
I just want to check if we should use actual or normalized counts.
Currently, we use non-normalized counts, so the metric score depends on the size of the real and synthetic data; for instance:

real_data = pd.Series(['id1', 'id1', 'id2', 'id2', 'id2', 'id3'] * 2)
synthetic_data = pd.Series(['id4', 'id4', 'id5', 'id6', 'id6', 'id6'])

score = SequenceLengthSimilarity.compute(real_data, synthetic_data)

print(score)

We will print 0.33, while in both real and synthetic data, we have a sequence that is half; one is one-third, and the last is one-sixth of the data size. Let me know if it makes sense. @Neha, do you have any thoughts on it?

[npatki]

the metric score depends on the size of the real and synthetic data

As it should! The metric is called SequenceLengthSimilarity, so of course the length of the sequences matter.

In this example, score1 should not be equal to score2:

real_data1 = pd.Series(['id1', 'id1', 'id2', 'id2', 'id2', 'id3'])
real_data2 = pd.Series(['id1', 'id1', 'id2', 'id2', 'id2', 'id3']*2)
synthetic_data = pd.Series(['id4', 'id4', 'id5', 'id6', 'id6', 'id6'])

score1 = SequenceLengthSimilarity.compute(real_data1, synthetic_data)
score2 = SequenceLengthSimilarity.compute(real_data2, synthetic_data)

This is because the SequenceLengthSimilarity metric is computing the length of each sequence and then using KSComplement on the resulting distributions.

• Length of sequences in real_data1 is: [2, 3, 1]
• Length of sequences in real_data2 is: [4, 6, 2]
• Length of sequences in synthetic_data: [2, 1, 3]

As a result, score1 should be 1.0 while score2 should be 0.33333. Normalizing would defeat the purpose of comparing lengths.

@amontanez24 could you also review if you get the chance?

[amontanez24]

This lgtm!