Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix KSComplement instability for constant float values #654

Merged
merged 9 commits into from
Nov 27, 2024
Merged

Fix KSComplement instability for constant float values #654

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
97538ad
Add kscomplement fix
fealho Nov 15, 2024
9296f8b
Merge branch 'main' into issue-652-kscomplement
fealho Nov 19, 2024
955783e
Merge branch 'main' into issue-652-kscomplement
fealho Nov 19, 2024
3a7e024
Update tests
fealho Nov 20, 2024
276252b
Fix tests
fealho Nov 25, 2024
031febf
Feedback
fealho Nov 25, 2024
3700419
Docstring
fealho Nov 25, 2024
e7d49ea
Merge branch 'main' into issue-652-kscomplement
fealho Nov 25, 2024
2e752dd
Merge branch 'main' into issue-652-kscomplement
fealho Nov 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions sdmetrics/single_column/statistical/kscomplement.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@
from sdmetrics.single_column.base import SingleColumnMetric
from sdmetrics.utils import is_datetime

MAX_DECIMALS = sys.float_info.dig - 1


class KSComplement(SingleColumnMetric):
"""Kolmogorov-Smirnov statistic based metric.
Expand Down Expand Up @@ -59,9 +61,8 @@ def compute(real_data, synthetic_data):
real_data = pd.to_numeric(real_data)
synthetic_data = pd.to_numeric(synthetic_data)

max_decimals = sys.float_info.dig - 1
real_data = real_data.round(max_decimals)
synthetic_data = synthetic_data.round(max_decimals)
real_data = real_data.round(MAX_DECIMALS)
synthetic_data = synthetic_data.round(MAX_DECIMALS)

try:
statistic, _ = ks_2samp(real_data, synthetic_data)
Expand Down
5 changes: 5 additions & 0 deletions tests/integration/single_column/statistical/test_kscomplement.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,12 @@ def test_bad(array_like):


def test_one_float_value():
# Setup
fealho marked this conversation as resolved.
Show resolved Hide resolved
real = pd.Series([0.3 - 0.2])
synth = pd.Series([0.2 - 0.1])

# Run
output = KSComplement.compute(real, synth)

# Assert
assert output == 1
Loading