-
Notifications
You must be signed in to change notification settings - Fork 996
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fixes 16652: add GCS storage service #16917
Fixes 16652: add GCS storage service #16917
Conversation
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
The Python checkstyle failed. Please run You can install the pre-commit hooks with |
Hi @MChamberlin thanks for the PR. Could you please check the python checkstyle? a |
openmetadata-service/src/main/resources/json/data/testConnections/storage/gcs.json
Show resolved
Hide resolved
Phylum OSS Supply Chain Risk Analysis - SUCCESSThe Phylum risk analysis is complete and has passed the active policy. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great PR, thanks @MChamberlin
Quality Gate passed for 'open-metadata-ui'Issues Measures |
Quality Gate passed for 'open-metadata-ingestion'Issues Measures |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR
Describe your changes:
Fixes 16652.
Add GCS (Google Cloud Storage) storage service connection. Documentation for this service (and for ADLS) already existed, but the actual implementation did not and the service is marked as unsupported in the UI code.
I assume this documentation mismatch is an oversight. The ADLS documentation should probably be removed because that connector implementation does not exist either.
This PR adds an implementation for the GCS storage service connector. It tries to follow the general pattern of the S3 connector. It supports multiple project ids and bucket metrics via the
google-cloud-monitoring
package.Connector UI screenshots:
Screenshot of storage container with structured data (parsed from manifest file):
Type of change:
Checklist:
Fixes <issue-number>: <short explanation>