Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

source-facebook-marketing: support multiple account ids #2162

Merged
merged 2 commits into from
Nov 22, 2024

Conversation

Alex-Bair
Copy link
Contributor

@Alex-Bair Alex-Bair commented Nov 22, 2024

Description:

The main change in this PR is supporting multiple Facebook Marketing account IDs in a single connector. Multiple account IDs can be provided as comma separated values in the account_ids field. This change is backwards compatible with older configs that only have an account_id field.

The account ID is used in multiple locations throughout the connector, and many of the changes are for threading the account ID through all streams.

Supporting more than one account ID in the connector also means that streams need to retain a cursor per account ID. The various state management tools (get_updated_state, stream_slices, the state property, etc.) have been updated to retain a cursor value for each account ID.

Closes #2110

Workflow steps:

(How does one use this feature, and how has it changed)

Documentation links affected:

The source-facebook-marketing documentation should be updated to reflect that multiple account IDs are supported. The docs update PR will be linked here once it's ready.

Notes for reviewers:

Tested on a local stack with a single account ID and a recent start date (01NOV2024). Confirmed that:

  • The same number of records are retrieved for all standard streams before and after these changes.
  • Existing tasks switch over to use account_ids without error.
  • Existing tasks' incremental streams pick up where they left off (i.e. cursor values are preserved through the switchover).

Spec and discover snapshot changes are expected due to adding support for multiple account IDs & updating the documentation URLs.


This change is Reviewable

@Alex-Bair Alex-Bair added change:unplanned Unplanned change, useful for things like doc updates change:planned This is a planned change and removed change:unplanned Unplanned change, useful for things like doc updates labels Nov 22, 2024
Copy link
Member

@williamhbaker williamhbaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Alex-Bair Alex-Bair force-pushed the bair/facebook-marketing-account-ids branch from 724bbf4 to beb2ff1 Compare November 22, 2024 16:44
@Alex-Bair Alex-Bair merged commit f9e3e79 into main Nov 22, 2024
62 of 78 checks passed
@Alex-Bair Alex-Bair deleted the bair/facebook-marketing-account-ids branch November 22, 2024 16:50
@Alex-Bair Alex-Bair changed the title Bair/facebook marketing account ids source-facebook-marketing: support multiple account ids Nov 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
change:planned This is a planned change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

source-facebook-marketing: Allow for multiple customer IDs
2 participants