v1.1.0 dbt_salesforce_source

PR #48 includes the following updates:

🚨 Breaking Change 🚨

• Added logic to support user-specified scenarios where the Fivetran Salesforce connector syncs column names using the original Salesforce API naming convention. For example, while Fivetran typically provides the column as created_date, some users might choose to receive it as CreatedDate according to the API naming. This update ensures the package is automatically compatible with both naming conventions.
  • Specifically, the package now performs a COALESCE, preferring the original Salesforce API naming. If the original naming is not present, the Fivetran version is used instead.
  • Renamed columns are now explicitly cast to prevent conflicts during the COALESCE.
  • ❗This change is considered breaking since the resulting column types may differ from prior versions of this package.

Under the Hood

• Added the following macros to support the mentioned bug fix logic:
  • add_renamed_columns: Determines the original names for each column and adds them to the list generated within each get_*_columns macro. By default, this macro processes column names by removing underscores and capitalizing each part that follows an underscore. This ensures all necessary columns are available for use in the coalesce_rename macro. Additionally, this macro tags each column with its renamed version to maintain tracking.
  • column_list_to_dict: Converts the list of dictionaries generated by the get_*_columns macros into a dictionary of dictionaries for use in the coalesce_rename macro. This conversion is necessary so that each column dictionary entry can be accessed by a key, rather than iterating through a list.
  • coalesce_rename: Utilizes the dictionary generated by column_list_to_dict to coalesce a column with its renamed counterpart, producing the final column. This macro also allows for the passing of a custom renamed spelling, datatype, and alias as arguments to override default values.
• Added validation test to ensure the final column names generated before and after this update remain the same.

Full Changelog: v1.0.1...v1.1.0

v1.0.1 dbt_salesforce_source

PR #44 includes the following updates:

Bug Fix

• Adds the missing version: 2 tag to the src_salesforce_history.yml file.

Under the Hood

• Cleans up duplicate seed configurations in integration_tests/dbt_project.yml.
• Updates the pull request templates.
• Includes auto-releaser GitHub Actions workflow to automate future releases.

Full Changelog: v1.0.0...v1.0.1

v1.0.0 dbt_salesforce_source

📣 THIS IS A MAJOR PACKAGE RELEASE! 📣 More details below.

PR #40 includes the following updates:

🚨 Breaking Change 🚨:

• We have removed all tmp models and will use the fivetran_utils.fill_staging_column macro to compare directly to our source models in your schemas.

🚀 Feature Updates 🚀 :

• We have added history mode models in the models/history folder to support Fivetran's history mode feature. This will allow customers to utilize the Fivetran history mode feature, which records every version of each record in the source table from the moment this mode is activated in the equivalent tables.

• These models are disabled by default due to their size, so you will need to set the below variable configurations for each of the individual models you want to utilize in your dbt_project.yml.

vars:
  salesforce__[history_model]_enabled: true ##Ex: salesforce__account_history_enabled: true          

• IMPORTANT: All fields in your Salesforce history mode connector that are being synced are being included in the end models. To change which fields are brought in via end models, you will need to update the fields you are bringing in via your history mode connector in Fivetran and then run a dbt run --full-refresh. See the DECISIONLOG for more details.

• Here are the history staging models that were added:

Model added	Description
stg_salesforce__account_history	Represents historical records of individual accounts, which are organizations or people involved with your business (such as customers, competitors, and partners).
stg_salesforce__contact_history	Represents the historical record of contacts, which are people associated with an account.
stg_salesforce__opportunity_history	Represents historical records of opportunities, which are sales or pending deals.

• All history models are incremental due to the volume of data being ingested.

• We support the option to pull from both your standard Salesforce and History Mode connectors simultaneously from their specific database/schemas. We also support pulling from just your History Mode connector on its own and bypassing the standard connector on its own. See more detailed instructions in the README.

• Salesforce History Mode models can contain a multitude of rows if you bring in all historical data, so we've introduced the flexibility to set first date filters to bring in only the historical data you need. More details can be found in the README.

🔎 Under The Hood 🔎

• Salesforce standard models have been moved into the salesforce folder to clearly delineate between the two sets of models.
• We have deprecated the using_[source]_history_mode_active_records variables. The introduction of the new history mode capabilities in this package made these variables redundant.

v0.7.0 dbt_salesforce_source

🚨 Breaking Changes 🚨:

PR #39 includes the following breaking changes:

• Updates the old passthrough column methodology to allow for aliasing and/or transformations of any field names brought in. This is useful, for example, if you wish to bring in fields across different Salesforce objects that may have the same names and wish to alias them to avoid confusion, particularly if any of the objects are joined together downstream.

• In addition we have added the salesforce__ prefix to all the passthrough variables. Please be sure to update these accordingly if you are leveraging the variables in your existing project:

Old	New
account_pass_through_columns	salesforce__account_pass_through_columns
contact_pass_through_columns	salesforce__contact_pass_through_columns
event_pass_through_columns	salesforce__event_pass_through_columns
lead_pass_through_columns	salesforce__lead_pass_through_columns
opportunity_pass_through_columns	salesforce__opportunity_pass_through_columns
opportunity_line_item_pass_through_columns	salesforce__opportunity_line_item_pass_through_columns
order_pass_through_columns	salesforce__order_pass_through_columns
product_2_pass_through_columns	salesforce__product_2_pass_through_columns
task_pass_through_columns	salesforce__task_pass_through_columns
user_role_pass_through_columns	salesforce__user_role_pass_through_columns
user_pass_through_columns	salesforce__user_pass_through_columns

Under the Hood:

• Incorporated the new fivetran_utils.drop_schemas_automation macro into the end of each Buildkite integration test job.
• Updated the pull request templates.

dbt_salesforce_source v0.6.0

🚨 Breaking Changes 🚨:

PR #34 includes the following breaking changes:

• Dispatch update for dbt-utils to dbt-core cross-db macros migration. Specifically {{ dbt_utils.<macro> }} have been updated to {{ dbt.<macro> }} for the below macros:
  • any_value
  • bool_or
  • cast_bool_to_text
  • concat
  • date_trunc
  • dateadd
  • datediff
  • escape_single_quotes
  • except
  • hash
  • intersect
  • last_day
  • length
  • listagg
  • position
  • replace
  • right
  • safe_cast
  • split_part
  • string_literal
  • type_bigint
  • type_float
  • type_int
  • type_numeric
  • type_string
  • type_timestamp
  • array_append
  • array_concat
  • array_construct
• For current_timestamp and current_timestamp_in_utc macros, the dispatch AND the macro names have been updated to the below, respectively:
  • dbt.current_timestamp_backcompat
  • dbt.current_timestamp_in_utc_backcompat
• Dependencies on fivetran/fivetran_utils have been upgraded, previously [">=0.3.0", "<0.4.0"] now [">=0.4.0", "<0.5.0"].

Features

• Addition of the fivetran_formula_model table within the src_salesforce.yml file. This source may be leveraged when using the dbt_salesforce_formula_utils package.

dbt_salesforce_source v0.5.1

It's a good day today. Some fixes are here!

🐞 Bug fixes

PR #33 incorporates the following updates:

• Updated src_salesforce.yml to accomdate Snowflake users that make have issues with the orders source. See updated README for details.
• Updated typo in config for stg_salesforce__task_tmp.sql.

dbt_salesforce_source v0.5.0

🎉 Salesforce Package Updates 🎉

We are updating the Salesforce package! To improve its utility, the changes include the following:

Features

• Bringing additional tables (Account, Contact, Event, Task, Lead, Order, Product_2) to create a new Contact Enhanced and Daily Activity model as well as updating the Opportunity Enhanced model. (#30)
• Allowing formula fields to be added as passthrough columns. We added integration with the Salesforce Formula package by embedding the macro outputs as part of our staging models so that your custom formula fields can be included. (#30)
• Standardization updates (#25):
  • Updated formatting in our sql files.
  • The README has been updated to reflect our rehaul of our documentation style to make it more straightforward.
  • Added identifier variable to each source to allow for more end-user customization on which table to pull from.

dbt_salesforce_source 0.4.2

Happy Thursday!

This release of the dbt_salesforce_source package includes the following updates:

Fixes

• Casts the created_date and closed_date fields within the stg_salesforce__opportunity model to a timestamp using the dbt_utils.type_timestamp() macro. This is needed for Redshift users that see these fields being synced as timestamptz as the downstream date functions do not work with the timestamptz datatype.

dbt_salesforce_source 0.4.1

This release of the dbt_salesforce_source package includes the following updates:

Features

• Support for Databricks compatibility! (#20)
• Add feature for disabling the user_role table. (#22)

Contributors

• drernie (#20)

dbt_salesforce_source v0.4.0

🎉 dbt v1.0.0 Compatibility 🎉

🚨 Breaking Changes 🚨

• Adjusts the require-dbt-version to now be within the range [">=1.0.0", "<2.0.0"]. Additionally, the package has been updated for dbt v1.0.0 compatibility. If you are using a dbt version <1.0.0, you will need to upgrade in order to leverage the latest version of the package.
  • For help upgrading your package, I recommend reviewing this GitHub repo's Release Notes on what changes have been implemented since your last upgrade.
  • For help upgrading your dbt project to dbt v1.0.0, I recommend reviewing dbt-labs upgrading to 1.0.0 docs for more details on what changes must be made.
• Upgrades the package dependency to refer to the latest dbt_fivetran_utils. The latest dbt_fivetran_utils package also has a dependency on dbt_utils [">=0.8.0", "<0.9.0"].
  • Please note, if you are installing a version of dbt_utils in your packages.yml that is not in the range above then you will encounter a package dependency error.