Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resolve OpenCorporates ID to LEI mapping issues in Open Ownership Register #32

Closed
StephenAbbott opened this issue Jun 21, 2024 · 1 comment
Assignees

Comments

@StephenAbbott
Copy link
Member

Following work on #8, we have been made aware of mapping issues in the OpenCorporates to LEI mapping files. These have been examined and resolved by the OpenCorporates and GLEIF data quality teams now, so the issues should be resolved in the OpenCorporates bulk data files which Open Ownership receives each month.

This ticket is for us to take remedial action to replace all of the LEIs in our BODS datasets with the correct ones mapped to OpenCorporates IDs.

@tiredpixel tiredpixel moved this from Todo to In Progress in Open Ownership Register and BODS pipelines Jun 25, 2024
tiredpixel added a commit that referenced this issue Jun 25, 2024
tiredpixel added a commit that referenced this issue Jun 28, 2024
OC repositories don't have metadata or a concept of latest, so this is
unnecessary.
tiredpixel added a commit that referenced this issue Jun 28, 2024
@tiredpixel
Copy link
Contributor

tiredpixel commented Jun 28, 2024

Migration Plan for LEI Flush

  • stop Ingester PSC streamer in Heroku
  • stop Transformer PSC streamer in Heroku
  • delete add_ids index in Elasticsearch
  • delete alt_names index in Elasticsearch (not strictly necessary for LEI flush)
  • delete companies index in Elasticsearch (not strictly necessary for LEI flush)
  • create add_ids index in Elasticsearch
  • create alt_names index in Elasticsearch (not strictly necessary for LEI flush)
  • create companies index in Elasticsearch (not strictly necessary for LEI flush)
  • deploy latest code container LEI changes
  • run Ingester OC to fetch latest LEIs for add_ids index
  • add new LEIs to SK
  • add new LEIs to DK
  • add new LEIs to PSC
  • check LEIs in Elasticsearch for LEI scheme name (2) additions
  • remove old LEIs from SK
  • remove old LEIs from DK
  • remove old LEIs from PSC
  • check LEIs in Elasticsearch for LEI scheme name original removals
  • start Ingester PSC streamer in Heroku
  • start Transformer PSC streamer in Heroku
  • start running rest of monthly bulk data import
  • check Register entity page IDs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants