Handle combined opinions in columbia merger #3799
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🔍 Existing Issues For ReviewYour pull request is modifying functions with the following pre-existing issues: 📄 File: cl/corpus_importer/management/commands/columbia_merge.py
Did you find this useful? React with a 👍 or 👎 |
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
quevon24
changed the title
…lumbia-merger # Conflicts: # cl/corpus_importer/management/commands/columbia_merge.py
for more information, see https://pre-commit.ci
# Conflicts: # cl/corpus_importer/management/commands/columbia_merge.py
update comments
# Conflicts: # cl/corpus_importer/utils.py
|
For this PR the change is minimal and avoids conflicts when trying to match combined opinions from cl when the xml has multiple opinions. The only thing that could be improved is the FILED_TAGS list which contains the valid text strings to identify a date as date_filed |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This small change excludes combined opinions when we have multiple opinions in the file and a combined opinion in cl, this is useful we try to map and merge the opinions in the columbia merger.
We need to exclude those opinions because we have splitted opinions in the columbia dataset and some of the matched clusters have combined opinions.
I also added a new exception type to catch the issues with strings in volume number, issue mentioned in #3824