Support python 312

[swheaton]

Type

• Bug fix (non-breaking change which fixes an issue): Fixes Support Python 3.12 #6433
• New feature (non-breaking change which adds functionality). Resolves #
• Breaking change (fix or feature that would cause existing functionality to not work as expected) Resolves #

Motivation and Context

Closes #6433
Add support for python 3.12

Checklist:

• I have run python util/check_style.py --apply to apply Open3D code style
  to my code.
• This PR changes Open3D behavior or adds new functionality.
  • Both C++ (Doxygen) and Python (Sphinx / Google style) documentation is
    updated accordingly.
  • I have added or updated C++ and / or Python unit tests OR included test
    results (e.g. screenshots or numbers) here.
• I will follow up and update the code if CI fails.
• For fork PRs, I have selected Allow edits from maintainers.

Description

• Follows template of Add support for python3.11 #6288 to add python3.12 support

• TensorFlow updated to v2.16.1

• PyTorch updated to v2.2.0

• CUDA updated to 12.1

[swheaton]

Here is the start of my PR - although the checks do not pass, just opening it up for initial comment.

The current issue I'm seeing in CI is that I don't think there is a single version of scipy for requirements_test.txt that works with both python 3.12 and 3.8 -- current 1.10.1 requires Python <3.12, >=3.8, while the next release 1.11.1 requires Python <3.13, >=3.9.

Python 3.8 goes EOL in October of this year.

Here's a few options I see:

• (don't recommend) Remove python 3.8 support despite it not being EOL for several more months
• Wait to support python 3.12 (hold this PR) until 3.8 becomes EOL
• (don't recommend) Don't pin packages for tests
• Have 2 sets of test requirements: requirements_test_py312.txt that is used for python 3.12 instead of requirements_test.txt

I'm sure there may be other compatibility issues with packages but the CI just hasn't gotten far enough to see them yet.
@ssheorey

[ssheorey]

Here is the start of my PR - although the checks do not pass, just opening it up for initial comment.

The current issue I'm seeing in CI is that I don't think there is a single version of scipy for requirements_test.txt that works with both python 3.12 and 3.8 -- current 1.10.1 requires Python <3.12, >=3.8, while the next release 1.11.1 requires Python <3.13, >=3.9.

Python 3.8 goes EOL in October of this year.

Here's a few options I see:

* (don't recommend) Remove python 3.8 support despite it not being EOL for several more months

* Wait to support python 3.12 (hold this PR) until 3.8 becomes EOL

* (don't recommend) Don't pin packages for tests

* Have 2 sets of test requirements: `requirements_test_py312.txt` that is used for python 3.12 instead of `requirements_test.txt`

I'm sure there may be other compatibility issues with packages but the CI just hasn't gotten far enough to see them yet. @ssheorey

Hi @swheaton thanks for contributing Python 3.12 support! See the requirements.txt file format docs for specifying different package versions based on different Python versions:

https://pip.pypa.io/en/stable/reference/requirements-file-format/

[swheaton]

Hi @swheaton thanks for contributing Python 3.12 support! See the requirements.txt file format docs for specifying different package versions based on different Python versions:

https://pip.pypa.io/en/stable/reference/requirements-file-format/

Ah thanks, learned something today!

[swheaton]

It goes further but looks like there are some C API changes in 3.12 that affect open3d. Googling around, this appears to be a common issue with python 3.12.
https://github.com/swheaton/Open3D/actions/runs/8423001029/job/23063629915#step:4:3856

This seems like a related S/O post (with no answer), since python/pyproject.toml also has jupyter related requirements in nit.

Seems related to cython/cython#5238, but I'm not sure what dependency to bump to get any available fixes there

[ssheorey]

TensorFlow updated to v2.16.1
PyTorch updated to v2.2.0
CUDA updated to 11.8

[swheaton]

Note to self, also need to update open3d-ml repo, similar to isl-org/Open3D-ML#619
Since this repo always checks out the ML repo's main branch, I'm not sure the order these things are supposed to happen.

[johnthagen]

Thanks for working on this! Python 3.12 support requests for Open3D will likely increase once Ubuntu 24.04 LTS is released on April 25 (in two weeks).

This is because Ubuntu 24.04 ships with Python 3.12 installed by default

• https://launchpad.net/ubuntu/noble/+source/python3.12

[jtkbong]

Hi, my engineering team is moving all our projects to Python 3.12 and Open3D is one of the projects that isn't supported by Python 3.12 yet. I am wondering if support for 3.12 will come soon, or should we consider going down 1 version and use Python 3.11 instead. Thanks!

[jweaston]

Will support for 3.12 have to wait until the 0.19 release or will 3.12 support be available for 0.18 once this PR is completed?

[johnthagen]

@jweaston Very likely 3.12 support will only be for the 0.19 release and not back ported to 0.18.

This is based on observing how new Python support has been added past and is also much less work for those maintaining Open3D.

[apockill]

@johnthagen thank you for the updates- is there an estimate for when 0.19 might drop? It would help people plan their upgrade paths for Ubuntu 24.04.

[johnthagen]

@apockill I am only an occasional contributor to Open3D. @ssheorey would know more about release plans, but you can track the issues the team has assigned to the 0.19 milestone here: https://github.com/isl-org/Open3D/milestone/14

[ssheorey]

Hi @benjaminum can you take a look at the python ml_ops test failures:
They are mostly like this:

../python/test/ml_ops/test_cconv.py:103: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../python/test/ml_ops/mltest.py:95: in run_op
    ans = fn(*args, **kwargs)
../../../../../.pyenv/versions/3.12.2/envs/o3d-312/lib/python3.12/site-packages/keras/src/utils/traceback_utils.py:122: in error_handler
    raise e.with_traceback(filtered_tb) from None
../../../../../.pyenv/versions/3.12.2/envs/o3d-312/lib/python3.12/site-packages/open3d/ml/tf/python/layers/neighbor_search.py:131: in call
    table = ops.build_spatial_hash_table(
../../../../../.pyenv/versions/3.12.2/envs/o3d-312/lib/python3.12/site-packages/open3d/ml/tf/python/ops/ops.py:183: in build_spatial_hash_table
    return _lib.open3d_build_spatial_hash_table(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

points = <tf.Tensor: shape=(231, 3), dtype=float32, numpy=
array([[0.        , 0.        , 0.        ],
       [0.        , 0. ...1.        ],
       [9.        , 1.8       , 2.        ],
       [9.        , 1.8       , 2.3333333 ]], dtype=float32)>
radius = <tf.Tensor: shape=(), dtype=float64, numpy=0.51>, points_row_splits = <tf.Tensor: shape=(2,), dtype=int64, numpy=array([  0, 231])>
hash_table_size_factor = 0.015625, max_hash_table_size = 33554432, name = None

>   ???
E   tensorflow.python.framework.errors_impl.InvalidArgumentError: Exception encountered when calling FixedRadiusSearch.call().
E   
E   cannot compute Open3DBuildSpatialHashTable as input #1(zero-based) was expected to be a float tensor but is a double tensor [Op:Open3DBuildSpatialHashTable] name: 
E   
E   Arguments received by FixedRadiusSearch.call():
E     • points=tf.Tensor(shape=(231, 3), dtype=float32)
E     • queries=tf.Tensor(shape=(5, 3), dtype=float32)
E     • radius=tf.Tensor(shape=(), dtype=float64)
E     • points_row_splits=None
E     • queries_row_splits=None
E     • hash_table_size_factor=0.015625
E     • hash_table=None

<string>:2239: InvalidArgumentError
________________ test_cconv_gradient[float32-filter_size2-5-3-False-True-False-True-False-ball_to_cube_volume_preserving-linear_border-ml0] ________________

[dyollb]

looking forward to open3d being available in 3.12

[ssheorey]

[Needs minor cleanup before merge]

[ssheorey]

Needs corresponding update to Python 3.12 in Open3D-ML for some CI checks (macOS Python test) to pass.
isl-org/Open3D-ML#657

[ssheorey]

Thanks everyone, especially @swheaton @johnthagen for the help!

[swheaton]

Thanks for taking my PR to the finish line, apologies I wasn't able to finish it myself.
Just glad we will have py312 support soon!