Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

regulations.gov duplicates with different posted_date #17

Open
judgelord opened this issue Mar 24, 2021 · 0 comments
Open

regulations.gov duplicates with different posted_date #17

judgelord opened this issue Mar 24, 2021 · 0 comments
Assignees
@judgelord
Labels
bug

Comments

@judgelord
Copy link
Owner

Possibly due to files being updated between my database updates, there are some duplicates.

Check for them by adding a count by comment_url, which should be unique in the comments table.

I am deduping by grouping by everything except posted_date and slicing the max posted date.
@judgelord judgelord self-assigned this Mar 24, 2021
judgelord added a commit that referenced this issue Mar 24, 2021
@judgelord
#17 dedupe cfpb db
1421392
judgelord added a commit that referenced this issue Mar 26, 2021
@judgelord
slice max comment start and end date where comment periods changed (c…
a9a0f7a 
…ausing duplicate comments) #17
judgelord added a commit that referenced this issue Mar 27, 2021
@judgelord
dedupe regulations.gov federal register crosswalk #17
04881fb
@judgelord judgelord added the bug label Jul 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@judgelord judgelord

Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@judgelord