Bug/renamed columns dynamic

[fivetran-catfritz]

PR Overview

This PR will address the following Issue/Feature:

This PR will result in the following new package version:

Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:

PR Checklist

Basic Validation

Please acknowledge that you have successfully performed the following commands locally:

• dbt run –full-refresh && dbt test
• dbt run (if incremental models are present) && dbt test

Before marking this PR as "ready for review" the following have been applied:

• The appropriate issue has been linked, tagged, and properly assigned
• All necessary documentation and version upgrades have been applied
• docs were regenerated (unless this PR does not include any code or yml updates)
• BuildKite integration tests are passing
• Detailed validation steps have been provided below

Detailed Validation

Please share any and all of your validation steps:

If you had to summarize this PR in an emoji, which would it be?

💃

[fivetran-joemarkiewicz]

@fivetran-catfritz just one small suggestion and a more open ended question I have when doing a high level review of this PR. Let me know your thoughts!

[fivetran-joemarkiewicz]

Suggested change

	{# Use renamed_column_name value if it provided in the get_columns macro #}
	{# Use renamed_column_name value if it's provided in the get_columns macro #}

[fivetran-joemarkiewicz]

The only downside I see here is that maintenance will become difficult and it will be hard for us and the community to sift through the macros to understand what is occurring.

Let's bring this up during standup. I am leaning more towards manually coalescing the fields in these staging models to make it more clear what is happening. These macros are super powerful, but we need to consider not making things so dynamic that maintaining the code is a challenge. I feel we ran into this same issue with some of the HubSpot staging models and they are now quite difficult to decipher and triage.

[fivetran-catfritz]

I was thinking along the same lines as well, though I was thinking the macro could help with maintenance, but the answer might be a hybrid of my two methods if it's too nuanced. Because of all the coalescing, I was trying to avoid manually providing info like the datatype that already exists in the column macro, so we'd only have to maintain the columns and datatypes in one place, though I could see how that reduces transparency. Maybe a comment pointing to the macro could help, but yeah let's discuss at standup.