Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-1571459 parse jbdc properties #909

Merged
merged 11 commits into from
Aug 13, 2024

Conversation

sfc-gh-bzabek
Copy link
Contributor

@sfc-gh-bzabek sfc-gh-bzabek commented Aug 9, 2024

Overview

SNOW-1571459 parse properties
SNOW-1571515 detect overrides

Add new property: "snowflake.jdbc.map": "isInsecureMode : true, notYetExistingProp : true"

This will allow user to set custom snowflake jdbc properties and to handle properties that will be created for JDBC Driver in the future.
The map is not allowed to override other jdbc properties like: snowflake.database.name, snowflake.schema,name, snowflake.private.key etc. nor jvm.proxy.xxx. In such a case the error is thrown.
It is also not designed to replace them.

Properties passed to a map are not validated, for example "foo: bar" is allowed. In such a case we rely on Jdbc Driver behaviour.

Pre-review checklist

  • This change should be part of a Behavior Change Release. See go/behavior-change.
  • This change has passed Merge gate tests
  • Snowpipe Changes
  • Snowpipe Streaming Changes
  • This change is TEST-ONLY
  • This change is README/Javadocs only
  • This change is protected by a config parameter <PARAMETER_NAME> eg snowflake.ingestion.method.
    • Yes - Added end to end and Unit Tests.
    • No - Suggest why it is not param protected
  • Is his change protected by parameter <PARAMETER_NAME> on the server side?
    • The parameter/feature is not yet active in production (partial rollout or PrPr, see Changes for Unreleased Features and Fixes).
    • If there is an issue, it can be safely mitigated by turning the parameter off. This is also verified by a test (See go/ppp).

@sfc-gh-bzabek sfc-gh-bzabek requested a review from a team as a code owner August 9, 2024 11:19
/** All jdbc properties including proxyProperties */
private final Properties properties;
/** Proxy related properties */
private final Properties proxyProperties;
Copy link
Contributor Author

@sfc-gh-bzabek sfc-gh-bzabek Aug 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a special use case for proxyProperties

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

    this.internalStage =
        new SnowflakeInternalStage(
            (SnowflakeConnectionV1) this.conn, credentialExpireTimeMillis, proxyProperties);

if (proxyProperties != null && !proxyProperties.isEmpty()) {
Properties combinedProperties =
mergeProxyAndConnectionProperties(this.prop, this.proxyProperties);
if (!proxyProperties.isEmpty()) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nullability is handled in JdbcProperties create().

this.connectorName = conf.get(Utils.NAME);
this.ingestionMethodConfig = IngestionMethodConfig.determineIngestionMethod(conf);
this.prop = InternalUtils.createProperties(conf, this.url, ingestionMethodConfig);

Copy link
Contributor Author

@sfc-gh-bzabek sfc-gh-bzabek Aug 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking about parsing it inside JdbcProperties but it would also require ingestionMethodConfig and url. Hence I am parsing outside of JdbcProperties. I think it's simpler.

}

public Properties getProperties() {
Properties copy = new Properties();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason for making a copy here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super anti bug safety? I can remove it. I would not insist on keeping that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reconsidered and removed the copy.


public class JdbcPropertiesTest {

@Test
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add new tests using JUnit5 (import from junit.jupiter package)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in both classes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@@ -134,4 +136,18 @@ public void testResultSize() throws SQLException {
resultSet = new MockResultSetForSizeTest(100);
assert InternalUtils.resultSize(resultSet) == 100;
}

@Test
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not mandatory, but you can give a try to migrate this file to JUnit5 as well. Maybe it can be done in 10 minutes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, only @Test needed migration.

@sfc-gh-bzabek sfc-gh-bzabek merged commit 3dabfc0 into master Aug 13, 2024
80 checks passed
@sfc-gh-bzabek sfc-gh-bzabek deleted the bzabek-SNOW-1571459-parse-jbdc-properties branch August 13, 2024 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants