[Spark] Native test TestPrestoSparkNativeJoinQueries.testBucketedInnerJoin fails in CI

[czentgr]

The CircleCI tests only tests a subset of the Spark native E2E tests.

It only tests these three test classes (56 total tests):

TESTCLASSES=TestPrestoSparkExpressionCompiler,TestPrestoSparkNativeJoinQueries,TestPrestoSparkSqlFunctions

Meanwhile, when migrating to github actions a larger number of tests is run revealing test issues.

The following set of test classes is run:

TESTCLASSES = TestPrestoSparkExpressionCompiler,TestPrestoSparkNativeBitwiseFunctionQueries,TestPrestoSparkNativeAggregations,TestPrestoSparkNativeTpchConnectorQueries,TestPrestoSparkNativeSimpleQueries,TestPrestoSparkSqlFunctions,TestPrestoSparkNativeJoinQueries,TestPrestoSparkNativeWindowQueries,TestPrestoSparkNativeTpchQueries,TestPrestoSparkNativeArrayFunctionQueries,TestPrestoSparkNativeGeneralQueries

TestPrestoSparkNativeJoinQueries.testBucketedInnerJoin fails:

2024-12-12T20:21:19.2582110Z [ERROR] Tests run: 9, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 83.079 s <<< FAILURE! - in com.facebook.presto.spark.TestPrestoSparkNativeJoinQueries
2024-12-12T20:21:19.2586127Z [ERROR] com.facebook.presto.spark.TestPrestoSparkNativeJoinQueries.testBucketedInnerJoin[Session{queryId=20241212_202001_00000_4djsh, transactionId=Optional.empty, user=user, source=test, catalog=hive, schema=tpch, timeZoneKey=Pacific/Apia, locale=en, remoteUserAddress=address, userAgent=agent, clientTags=[], resourceEstimates=ResourceEstimates{executionTime=Optional.empty, cpuTime=Optional.empty, peakMemory=Optional.empty, peakTaskMemory=Optional.empty}, startTime=1734034801106}](1)  Time elapsed: 5.509 s  <<< FAILURE!
2024-12-12T20:21:19.2590369Z java.lang.AssertionError: Execution of 'actual' query failed: SELECT b.name, c.name FROM customer_bucketed b, customer c WHERE b.name=c.name
2024-12-12T20:21:19.2591392Z 	at org.testng.Assert.fail(Assert.java:98)
2024-12-12T20:21:19.2592154Z 	at com.facebook.presto.tests.QueryAssertions.assertQuery(QueryAssertions.java:178)
2024-12-12T20:21:19.2593160Z 	at com.facebook.presto.tests.QueryAssertions.assertQuery(QueryAssertions.java:106)
2024-12-12T20:21:19.2594376Z 	at com.facebook.presto.tests.AbstractTestQueryFramework.assertQuery(AbstractTestQueryFramework.java:169)
2024-12-12T20:21:19.2595984Z 	at com.facebook.presto.nativeworker.AbstractTestNativeJoinQueries.testBucketedInnerJoin(AbstractTestNativeJoinQueries.java:61)
2024-12-12T20:21:19.2597609Z 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2024-12-12T20:21:19.2598774Z 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2024-12-12T20:21:19.2600145Z 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2024-12-12T20:21:19.2601214Z 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
2024-12-12T20:21:19.2602249Z 	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:135)
2024-12-12T20:21:19.2603507Z 	at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:673)
2024-12-12T20:21:19.2604728Z 	at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:220)
2024-12-12T20:21:19.2605786Z 	at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:50)
2024-12-12T20:21:19.2606872Z 	at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:945)
2024-12-12T20:21:19.2607936Z 	at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:193)
2024-12-12T20:21:19.2609093Z 	at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146)
2024-12-12T20:21:19.2610218Z 	at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
2024-12-12T20:21:19.2611472Z 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
2024-12-12T20:21:19.2612648Z 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
2024-12-12T20:21:19.2613496Z 	at java.base/java.lang.Thread.run(Thread.java:829)
2024-12-12T20:21:19.2614040Z Caused by: java.lang.NullPointerException
2024-12-12T20:21:19.2615083Z 	at com.facebook.presto.spark.util.PrestoSparkFailureUtils.toPrestoSparkFailure(PrestoSparkFailureUtils.java:59)
2024-12-12T20:21:19.2616756Z 	at com.facebook.presto.spark.execution.AbstractPrestoSparkQueryExecution.execute(AbstractPrestoSparkQueryExecution.java:395)
2024-12-12T20:21:19.2618593Z 	at com.facebook.presto.spark.PrestoSparkQueryRunner.executeWithStrategies(PrestoSparkQueryRunner.java:548)
2024-12-12T20:21:19.2619959Z 	at com.facebook.presto.spark.PrestoSparkQueryRunner.execute(PrestoSparkQueryRunner.java:522)
2024-12-12T20:21:19.2621085Z 	at com.facebook.presto.tests.QueryAssertions.assertQuery(QueryAssertions.java:175)
2024-12-12T20:21:19.2621805Z 	... 18 more
2024-12-12T20:21:19.2621984Z 
2024-12-12T20:21:19.9747647Z [INFO] Running com.facebook.presto.spark.TestPrestoSparkNativeSimpleQueries
2024-12-12T20:21:20.3392104Z 2024-12-12T14:21:20.244-0600	INFO	TestNG-test=Surefire test-1	com.facebook.airlift.log.Logging	Logging to stderr
2024-12-12T20:21:21.8718469Z WARNING: An illegal reflective access operation has occurred
2024-12-12T20:21:21.8720547Z WARNING: Illegal reflective access by org.apache.hadoop.fs.HadoopExtendedFileSystemCache (file:/root/.m2/repository/com/facebook/presto/presto-hdfs-core/0.291-SNAPSHOT/presto-hdfs-core-0.291-SNAPSHOT.jar) to field java.lang.reflect.Field.modifiers
2024-12-12T20:21:21.8722760Z WARNING: Please consider reporting this to the maintainers of org.apache.hadoop.fs.HadoopExtendedFileSystemCache
2024-12-12T20:21:21.8724479Z WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
2024-12-12T20:21:21.8725430Z WARNING: All illegal access operations will be denied in a future release
2024-12-12T20:21:21.9602325Z 2024-12-12T14:21:21.883-0600	WARN	TestNG-test=Surefire test-1	org.apache.hadoop.util.NativeCodeLoader	Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

See:
https://github.com/czentgr/presto/actions/runs/12303617334/job/34339523632

Your Environment

N/A

Expected Behavior

Test succeeds.

Current Behavior

Test fails.

Possible Solution

Steps to Reproduce

Screenshots (if appropriate)

Context

[czentgr]

Good run: https://github.com/prestodb/presto/actions/runs/12307010027/job/34350124877?pr=24236