Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added new connector: source-twilio #1365

Closed
wants to merge 3 commits into from

Conversation

Luishfs
Copy link
Collaborator

@Luishfs Luishfs commented Mar 13, 2024

Description:

(Describe the high level scope of new or changed features)

Notes for reviewers:

(anything that might help someone review this PR)


This change is Reviewable

Luishfs added 2 commits March 13, 2024 13:01
Setup for base image snapshot collection

Added initial snapshots

copying the base image behaviour

NIT: quick fix so the pull wont conflict with poetry
Remote Repo URL: git@github.com:airbytehq/airbyte.git
Source name: 8c862a8
Source Commit ID: 8c862a8013150cfee238c379e66ff9e84550da6b
Source Repo Prefix: airbyte-integrations/connectors/source-twilio/
Import Path: source-twilio/
License Type: MIT
License Path: airbyte-integrations/connectors/source-twilio/metadata.yaml

git-merge-subpath: 8c862a8013150cfee238c379e66ff9e84550da6b airbyte-integrations/connectors/source-twilio source-twilio
@Luishfs Luishfs requested a review from jgraettinger March 13, 2024 16:38
@Luishfs Luishfs self-assigned this Mar 13, 2024
@Luishfs Luishfs force-pushed the backup/source-twilio branch 3 times, most recently from b7cf98a to 39655da Compare March 22, 2024 12:57
@Luishfs Luishfs requested review from williamhbaker and removed request for jgraettinger March 22, 2024 12:58
NOTE: a small amount of fixups were necessary
because this connector is still MIT

Added to CI
@Luishfs Luishfs force-pushed the backup/source-twilio branch from 39655da to 6fd0254 Compare March 22, 2024 13:34
@Luishfs Luishfs linked an issue Mar 26, 2024 that may be closed by this pull request
@williamhbaker
Copy link
Member

There are a couple more recent commits to source-twilio from airbyte that seem useful, and the connector is still MIT licensed. Any reason not to pick a more recent commit?

@Luishfs
Copy link
Collaborator Author

Luishfs commented Mar 26, 2024

There are a couple more recent commits to source-twilio from airbyte that seem useful, and the connector is still MIT licensed. Any reason not to pick a more recent commit?

@williamhbaker I believe they were added last week, so i missed then on the import :) . Is our image in sync with those changes? If so i can force push the new import and roll all changes again

@williamhbaker
Copy link
Member

There are a couple more recent commits to source-twilio from airbyte that seem useful, and the connector is still MIT licensed. Any reason not to pick a more recent commit?

@williamhbaker I believe they were added last week, so i missed then on the import :) . Is our image in sync with those changes? If so i can force push the new import and roll all changes again

Ah I see. Our strategy has been to import the latest we coudl that is still MIT licensed, so it would be good to update our import since we have the opportunity to do so.

As far as our image being in sync with those changes, assuming you are referring to the connector running on ATF I'd guess it isn't. But still, we want to get the latest imported version we can, understanding that it may be a bit more work for us to make it backward compatible, because the latest updates to airbyte tend to be useful and backward compatible out of the box.

Copy link
Member

@williamhbaker williamhbaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need snapshot tests comparing the connector output pre and post conversion if at all possible, and also in the PR description describe how you tested the migration from the old version of the connector (pre-CDK) to the new version built from this branch.

See other inline comments as well

@@ -880,19 +953,13 @@
]
}
},
"required": [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The collection key needs to be required, or materializations will have a bad time materializing the collection generally.

@@ -1004,19 +1085,13 @@
]
}
},
"required": [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Collection key should be required, see above

@@ -1354,10 +1459,6 @@
]
}
},
"required": [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Collection keys should be required, see above, and assume this applies for all other collection keys as well, since I'm going to stop noting this for each individual collection as I continue reading through the snapshot 😄

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, going to update this

"type": [
"null",
"string"
]
},
"price": {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's pretty hard to comprehend what is actually changing here, and if its alright or not. Can you comment on what is changing with these discovery outputs? A link to a jsondiff would be helpful; see also here

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that was due to all changes from the commits and the ATF disparity with the version i imported.
Just checked it, and the source-twillio image has been updated yesterday, so im going to pull the current commit and start again. Hopefully we will remove most of this noise

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will also provide the jsondiff link.

@Luishfs
Copy link
Collaborator Author

Luishfs commented Apr 9, 2024

As mentioned earlier, im removing this PR and moving it here, the new branch is created inside estuary/connectors and all other comments were already adressed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants