Duplicate entries in 1993

[neagle]

This problem report was submitted by @gurujeet-ngc via email:

AGAGD shows a big spike of over 3000 games on a day in 1993. This is in error and a result of games from the Go congress being doubly counted. See analysis and recommended action below.

The doubling occurred on the reporting for the Go Congress where games were reported as occurring on 8/7/1993. The total games for tournament cong19930807a (1540) equals the sum of the games for cong19930807b (778) and cong19930807c (762).

Looking at just one player’s games, Pin_Player_1, it is clear that the 9 games played in ‘a’ reappear in ‘b’ or ‘c’. This is true for all other players as well.

Recommendation: delete tournament cong19930807 and its games from the database as they are erroneous duplicates. Additionally there is a tournament cong19930808 with a much smaller number of games that all appear to duplicate some of the games played in the previous tournaments. It is possible that these were self-pair games but should be investigated further to see if they are duplicates also.

[vash3g]

These should not be deleted directly from the database. they should be removed using ratings tools at best.

[neagle]

What are the problems / risks associated with deleting them from the database?

What would be your recommendation as to how to remove them with the ratings tools?

[gurujeet-ngc]

I lost Steve’s original message. Can you forward – thanks. From: Nate ***@***.***> Sent: Monday, August 9, 2021 11:35 AM To: ***@***.***> Cc: Gurujeet ***@***.***>; ***@***.***> Subject: Re: [usgo/agagd] Duplicate entries in 1993 (#217) What are the problems / risks associated with deleting them from the database? What would be your recommendation as to how to remove them with the ratings tools? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#217 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AQV5GIIZRVY3BBNU5FEQS6LT37YTXANCNFSM5B2F4VCQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email>.

[michaelhiiva]

If we were just to use the ratings tools for this and use the revert function, would that not require the original [tournament_name].in file? This would be based on the current documentation for our aga-ratings tools.

[gurujeet-ngc]

I think it can just be done with a few SQL delete statements. Identify which of the tables have the duplicated tournaments and games. I don’t have the schema, but that should be pretty straightforward. From: Michael ***@***.***> Sent: Monday, October 4, 2021 7:01 PM To: ***@***.***> Cc: Gurujeet ***@***.***>; ***@***.***> Subject: Re: [usgo/agagd] Duplicate entries in 1993 (#217) If we were just to use the ratings tools for this and use the revert function, would that not require the original [tournament_name].in file? This would be based on the current documentation for our aga-ratings tools. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#217 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AQV5GIJPUT3MX6M54Z5KSJLUFIW3BANCNFSM5B2F4VCQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[vash3g]

How do we know this is in error? Is this based on a large spike or historical research?

This should not be done directly though SQL commands. We have an archive of over 4000 old tournaments. At a minimum this should be testes with the ratings tools before any other attempts are made.

[gurujeet-ngc]

The error was clearly documented with the submission report. From: Steve ***@***.***> Sent: Monday, October 4, 2021 9:51 PM To: ***@***.***> Cc: Gurujeet ***@***.***>; ***@***.***> Subject: Re: [usgo/agagd] Duplicate entries in 1993 (#217) How do we know this is in error? Is this based on a large spike or historical research? This should not be done directly though SQL commands. We have an archive of over 4000 old tournaments. At a minimum this should be testes with the ratings tools before any other attempts are made. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#217 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AQV5GIOIJZHGAACMF6X5MITUFJKZTANCNFSM5B2F4VCQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[michaelhiiva]

@gurujeet-ngc The schema is available at: https://github.com/usgo/usgo-sql-schemas/blob/main/schema.sql.

[neagle]

Yup, it's very clear from the evidence in the submission that this is an error. We just need to examine what the most appropriate way of fixing this is. @vash3g, you've mentioned that this should be fixed through the ratings tools and not by directly modifying the DB -- that makes intuitive sense, but what are the problems or issues with directly modifying the data, out of curiosity?