Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spline support for expand operation #814

Open
liujiawen opened this issue Apr 19, 2023 · 0 comments
Open

Spline support for expand operation #814

liujiawen opened this issue Apr 19, 2023 · 0 comments
Labels

Comments

@liujiawen
Copy link

Background

The Spline agent does not support column-level lineage tracking in expand operations. So, if my sql contains "grouping", or "with cube", the column lineage will be lost after expand.

Feature

Add OperationNodeBuilder to deal with spark expand operation.

Example [Optional]

image

Proposed Solution [Optional]

Solution Ideas

  1. I add a ExpandNodeBuilder class in package za.co.absa.spline.harvester.builder.plan, which has almost the same code as UnionNodeBuilder
  2. I modified the class OperationNodeBuilderFactory, to deal with expand operators.
@wajda wajda transferred this issue from AbsaOSS/spline Jul 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: New
Development

No branches or pull requests

1 participant