Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MINOR: Fix Table constraint relationships for SoftDeleted entities. #19129

Merged
merged 2 commits into from
Dec 19, 2024

Conversation

IceS2
Copy link
Contributor

@IceS2 IceS2 commented Dec 18, 2024

Currently Soft Delete and Constraints do not fare well together and on different scenarios it errors.

An example is our E2E MySQL Pipeline where we:

  1. Ingest all the Tables
  2. Ingest Tables with filters, which prompts to soft delete a couple tables
  3. Ingest Tables with other filters, which prompts to restore a couple tables

Some of those tables have different Foreign Key constraints and the process is currently erroring for a couple tables due to two different methods being called:

  1. storeEntity (On soft delete)

Here the addConstraintRelationship gets called and searches for the Foreign Key table, which was already soft deleted, raising an exception.

  1. updateTableConstraints (On restore entity)

Here the validateTableConstraints gets called and validates the Foreign Key table, which is still soft deleted, raising an exception.

This PR fixes the issues by changing the filter to look also for soft deleted tables both within addConstraintRelationship and validateTableConstraints. I think it makes sense to keep the constraint if there's a soft deleted table in Collate, but if we don't want this or for other weird corner cases we might need to rethink how we are dealing with this flow.

Type of change:

  • Bug fix

Checklist:

  • I have read the CONTRIBUTING document.
  • My PR title is Fixes <issue-number>: <short explanation>
  • I have commented on my code, particularly in hard-to-understand areas.
  • For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

Copy link
Contributor

The Java checkstyle failed.

Please run mvn spotless:apply in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Java code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

harshach
harshach previously approved these changes Dec 18, 2024
Copy link

@pmbrull pmbrull merged commit fa36c70 into main Dec 19, 2024
14 of 19 checks passed
@pmbrull pmbrull deleted the fix-er-constraints-on-soft-delete branch December 19, 2024 09:20
harshach pushed a commit that referenced this pull request Dec 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Ingestion safe to test Add this label to run secure Github workflows on PRs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants