Integrate spark-lineage with data in S3 #16095
Closed
mykola-yesypchuk-inflection
started this conversation in
General
Replies: 2 comments 1 reply
-
|
@ulixius9 can you chime in here |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
hey @mykola-yesypchuk-inflection, just a couple of days back we added the support for s3 container lineage with spark agent as part of this pr: open-metadata/openmetadata-spark-agent#17 we will soon be releasing this fix, once that's out you would be able to see the lineage between tables and containers 🙏 |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Great news. Thanks! |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hello.
I am looking for the data lineage solution and trying to integrate OpenMetadata with my data pipeline.
Currently I checking how spark-lineage and s3 storage are working together, but it seems not working. After my pipeline run I can see that UI get "spark pipeline entry" but empty lineage threads inside it.
Any suggestion?
Beta Was this translation helpful? Give feedback.
All reactions