Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-1658905 Limit SnowflakeSinkConnectorConfig responsibilities #925

Merged
merged 1 commit into from
Sep 18, 2024

Conversation

sfc-gh-mbobowski
Copy link
Contributor

@sfc-gh-mbobowski sfc-gh-mbobowski commented Sep 17, 2024

Overview

SNOW-1658905 Initial config refactor

SnowflakeSinkConnectorConfig class has a lot of responsibilities:

  • holding config keys and default values
  • applying default parameters values to the given config
  • storing config validators
  • serving config definition in a format required by KC framework

This PR extracts some parts of this class to dedicated files. There are still many other things to be fixed around config but this is a 1 hour of work that gives some effect.

Pre-review checklist

  • This change should be part of a Behavior Change Release. See go/behavior-change.
  • This change has passed Merge gate tests
  • Snowpipe Changes
  • Snowpipe Streaming Changes
  • This change is TEST-ONLY
  • This change is README/Javadocs only
  • This change is protected by a config parameter <PARAMETER_NAME> eg snowflake.ingestion.method.
    • Yes - Added end to end and Unit Tests.
    • No - Suggest why it is not param protected
  • Is his change protected by parameter <PARAMETER_NAME> on the server side?
    • The parameter/feature is not yet active in production (partial rollout or PrPr, see Changes for Unreleased Features and Fixes).
    • If there is an issue, it can be safely mitigated by turning the parameter off. This is also verified by a test (See go/ppp).

@sfc-gh-mbobowski sfc-gh-mbobowski requested a review from a team as a code owner September 17, 2024 12:26
@sfc-gh-mbobowski sfc-gh-mbobowski changed the title SNOW-1658905 Extract ConnectorConfigDefinition from the main config class SNOW-1658905 Limit SnowflakeSInkConnectorConfig responsibilities Sep 17, 2024
@sfc-gh-mbobowski sfc-gh-mbobowski changed the title SNOW-1658905 Limit SnowflakeSInkConnectorConfig responsibilities SNOW-1658905 Limit SnowflakeSinkConnectorConfig responsibilities Sep 17, 2024
@Override
public void ensureValid(String name, Object value) {
assert value instanceof String;
final String strValue = (String) value;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we assert on instance of string?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's copied. Even if it's wrong I don't want to change anything within this PR.

public TopicToTableValidator() {}

public void ensureValid(String name, Object value) {
String s = (String) value;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above :)

Copy link
Contributor

@sfc-gh-gjachimko sfc-gh-gjachimko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

// validate REST API is called
@Override
public void ensureValid(String name, Object value) {
assert value instanceof String;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the use of assert is a legacy part that we should get rid of. Since we cannot enforce that they are enabled at runtime, we assume they can be disabled.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but not in the same PR as refactor.

@sfc-gh-mbobowski sfc-gh-mbobowski merged commit cea8f1f into master Sep 18, 2024
80 checks passed
@sfc-gh-mbobowski sfc-gh-mbobowski deleted the mbobowski-SNOW-1658905-refactor branch September 18, 2024 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants