Support for latest Spark 3.4.4 in 3.4 series.

[pratham76]

Could you let us know if there are any plans to support latest spark 3.4.4 in 3.4.x series?
@zhouyuan could you please help here? Thanks

cc: @majetideepak @deepashreeraghu

[zhouyuan]

Hi @pratham76 Yes, we tried to bump the version a month ago when Spark-344 released, at that time it's pending on iceberg release of one commit(apache/iceberg@e3bbcac)
Will try to verify again (#7789)

CC @weiting-chen

thanks, -yuan

[zhouyuan]

@pratham76

• Iceberg dropped JDK8 support starting from 1.7.0 release, see below discussion thread for more details: https://lists.apache.org/thread/xd4sjd0cx010qcv9qmlt89zdx4s1cyp4

• Gluten JDK8 support is still required for the current release cycle. We may upgrade JDK8 in the 1.4 release

I think we could support Spark-344 in the main branch, but may require some refactoring on CI to run Spark unit test with iceberg only on JDK11+

[pratham76]

thanks @zhouyuan, does that imply we would remove JDK8 support for spark 3.4.x in main branch ?

[FelixYBW]

Let's move to JDK17 starting from Spark3.4 then, including 4.0. In future once we drop 3.3 and 3.2 support, we remove JDK8 as well.

[FelixYBW]

For customer it's already the trend.

@zhli1142015 @surnaik you already upgrade to JDK17 in your product, right?

[weiting-chen]

Gluten 1.3.0 will be released on Jan. 2025 soon, the support can only be discussed in next release(1.4.0)
Spark 3.4.4 support can be included in Gluten 2025 roadmap and I have added it, please check below link
#8226