linux-64 - rebuild Christmas 2024 - switch to rattler-build #229

traversaro · 2024-12-17T23:31:51Z

Just an attempt, heavily based on:

…e, perception

It was superseded by moveit/moveit2#2399 .

…in.patch The patch was integrated upstream in moveit/moveit2#2395 .

Patch upstreamed in https://github.com/moveit/moveit2/pull/2895/files .

…eason is not added

Heavely based on RoboStack/ros-jazzy#5

traversaro · 2024-12-18T00:11:20Z

The build fails with:

 ╭─ Running build for recipe: ros-humble-ament-package-0.14.0-np126py311h15c07e3_7
 │
 │ ╭─ Fetching source code
 │ │ Fetching source from git repo: https://github.com/ros2-gbp/ament_package-release.git
 │ │ Fetching repository from https://github.com/ros2-gbp/ament_package-release.git at release/humble/ament_package/0.14.0-4 into /home/runner/work/ros-humble/r
 │ │ os-humble/output/src_cache/ament_package-release.git
 │ │
 │ ╰─────────────────── (took 0 seconds)
 │
 ╰─────────────────── (took 0 seconds)
Error: 
  × Failed to run git command: failed to checkout FETCH_HEAD: error: pathspec
  │ 'release/humble/ament_package/0.14.0-4' did not match any file(s) known to
  │ git
  │

the corresponding recipe is :

# yaml-language-server: $schema=https://raw.githubusercontent.com/prefix-dev/recipe-format/main/schema.json

package:
  name: ros-humble-ackermann-msgs
  version: 2.0.2
source:
  git: https://github.com/ros2-gbp/ackermann_msgs-release.git
  rev: release/humble/ackermann_msgs/2.0.2-3
  target_directory: ros-humble-ackermann-msgs/src/work

build:
  number: 7
about:
  homepage: https://www.ros.org/
  license: BSD-3-Clause
  summary: |
    Robot Operating System

changing rev: to tag: seems to solve the problem.

traversaro · 2024-12-18T00:31:06Z

New problem:

 │ │ $SRC_DIR/conda_build.sh: line 10: build_ament_python.sh: command not found

for some reason vinca creates the build_ament_python.sh script without the executable permissions, but I need to go to bed now. :)

wolfv · 2024-12-18T11:33:04Z

for some reason vinca creates the build_ament_python.sh script without the executable permissions, but I need to go to bed now. :)

This is the first time I hear about this... You should be able to use either without extension (rattler-build picks it automatically, ie: script: foo) or with extension (${{ "foo.sh" if unix else "foo.bat" }}).

traversaro · 2024-12-18T11:46:22Z

for some reason vinca creates the build_ament_python.sh script without the executable permissions, but I need to go to bed now. :)

This is the first time I hear about this... You should be able to use either without extension (rattler-build picks it automatically, ie: script: foo) or with extension (${{ "foo.sh" if unix else "foo.bat" }}).

In the latter case it seems it was having both problem in locating the script (I had to add $RECIPE_DIR explicitly) and in running if there was not executable permissions. I added fixes/workaround for both in vinca at the moment, see 7bb5dfb .

With that the build goes on, and we encounter the first actual build problem:

2024-12-18T11:35:48.9129704Z  │ │ -- Detecting CXX compile features - done
2024-12-18T11:35:48.9231736Z  │ │ -- Found ament_cmake: 1.3.11 ($PREFIX/share/ament_cmake/cmake)
2024-12-18T11:35:49.0253604Z  │ │ -- Found Python3: $PREFIX/bin/python (found version "3.11.11") found components: Interpreter
2024-12-18T11:35:49.1631421Z  │ │ -- Found rosidl_default_generators: 1.2.0 ($PREFIX/share/rosidl_default_generators/cmake)
2024-12-18T11:35:49.1654897Z  │ │ -- Using all available rosidl_typesupport_c: rosidl_typesupport_fastrtps_c;rosidl_typesupport_introspection_c
2024-12-18T11:35:49.1710013Z  │ │ -- Found rosidl_adapter: 3.1.6 ($PREFIX/share/rosidl_adapter/cmake)
2024-12-18T11:35:49.2092726Z  │ │ -- Using all available rosidl_typesupport_cpp: rosidl_typesupport_fastrtps_cpp;rosidl_typesupport_introspection_cpp
2024-12-18T11:35:49.6083400Z  │ │ CMake Error at $PREFIX/share/rosidl_generator_c/cmake/rosidl_generator_c_generate_interfaces.cmake:69 (message):
2024-12-18T11:35:49.6084407Z  │ │ -- Configuring incomplete, errors occurred!
2024-12-18T11:35:49.6085363Z  │ │   Target dependency
2024-12-18T11:35:49.6087018Z  │ │   'lib/python3.11/site-packages/rosidl_generator_c/__init__.py' does not
2024-12-18T11:35:49.6088066Z  │ │   exist
2024-12-18T11:35:49.6089801Z  │ │ Call Stack (most recent call first):
2024-12-18T11:35:49.6091264Z  │ │   $PREFIX/share/ament_cmake_core/cmake/core/ament_execute_extensions.cmake:48 (include)
2024-12-18T11:35:49.6092682Z  │ │   $PREFIX/share/rosidl_cmake/cmake/rosidl_generate_interfaces.cmake:286 (ament_execute_extensions)
2024-12-18T11:35:49.6093638Z  │ │   CMakeLists.txt:29 (rosidl_generate_interfaces)
2024-12-18T11:35:49.6239446Z  │ │ ninja: error: loading 'build.ninja': No such file or directory
2024-12-18T11:35:49.6257756Z  │ │ × error Script failed with status 1
2024-12-18T11:35:49.6259613Z  │ │ × error Work directory: '/home/runner/work/ros-humble/ros-humble/output/bld/rattler-build_ros-humble-lifecycle-msgs_1734521405/work'
2024-12-18T11:35:49.6261039Z  │ │ × error To debug the build, run it manually in the work directory (execute the `./conda_build.sh` or `conda_build.bat` script)

that sounds related to #139 .

traversaro · 2024-12-18T11:55:19Z

that sounds related to #139 .

Indeed the required change was added in the rattler-build branch of vinca in RoboStack/vinca@55cda6e , so I cherry-picked the changes from #139 in this PR.

wolfv · 2024-12-18T11:55:28Z

I see ... that's a curious issue. We actually delay the jinja evaluation of the script these days so that you can do things like pip install ${{ PREFIX }} but that does have the drawback that we are not properly loading the file contents in that case! Sorry!

traversaro · 2024-12-18T12:14:31Z

I see ... that's a curious issue. We actually delay the jinja evaluation of the script these days so that you can do things like pip install ${{ PREFIX }} but that does have the drawback that we are not properly loading the file contents in that case! Sorry!

No problem, being explicit on the location of the build scripts seems to be working fine.

traversaro · 2024-12-18T12:28:27Z

Now it lasted ~26 minutes, and now it fails with:

2024-12-18T12:24:16.5535609Z  │ │   to work with policies introduced by <max> or earlier.
2024-12-18T12:24:16.6059410Z  │ │ -- The C compiler identification is GNU 14.2.0
2024-12-18T12:24:16.6881602Z  │ │ -- The CXX compiler identification is GNU 14.2.0
2024-12-18T12:24:16.6997625Z  │ │ -- Detecting C compiler ABI info
2024-12-18T12:24:16.7578873Z  │ │ -- Detecting C compiler ABI info - done
2024-12-18T12:24:16.7821877Z  │ │ -- Check for working C compiler: $BUILD_PREFIX/bin/x86_64-conda-linux-gnu-cc - skipped
2024-12-18T12:24:16.7824943Z  │ │ -- Detecting C compile features
2024-12-18T12:24:16.7830347Z  │ │ -- Detecting C compile features - done
2024-12-18T12:24:16.7957820Z  │ │ -- Detecting CXX compiler ABI info
2024-12-18T12:24:16.8675073Z  │ │ -- Detecting CXX compiler ABI info - done
2024-12-18T12:24:16.8936336Z  │ │ -- Check for working CXX compiler: $BUILD_PREFIX/bin/x86_64-conda-linux-gnu-c++ - skipped
2024-12-18T12:24:16.8939772Z  │ │ -- Detecting CXX compile features
2024-12-18T12:24:16.8948262Z  │ │ -- Detecting CXX compile features - done
2024-12-18T12:24:16.9046451Z  │ │ -- Found eigen3_cmake_module: 0.1.1 ($PREFIX/share/eigen3_cmake_module/cmake)
2024-12-18T12:24:16.9072554Z  │ │ -- Found Eigen3: TRUE (found version "3.4.0")
2024-12-18T12:24:16.9095186Z  │ │ -- Found ament_cmake: 1.3.11 ($PREFIX/share/ament_cmake/cmake)
2024-12-18T12:24:17.0065906Z  │ │ -- Found Python3: $PREFIX/bin/python (found version "3.11.11") found components: Interpreter
2024-12-18T12:24:17.1483576Z  │ │ -- Found console_bridge_vendor: 1.4.1 ($PREFIX/share/console_bridge_vendor/cmake)
2024-12-18T12:24:17.1494911Z  │ │ -- Found eigen_stl_containers: 1.0.0 ($PREFIX/share/eigen_stl_containers/cmake)
2024-12-18T12:24:17.1524980Z  │ │ -- Found geometry_msgs: 4.2.4 ($PREFIX/share/geometry_msgs/cmake)
2024-12-18T12:24:17.1879067Z  │ │ -- Found rosidl_generator_c: 3.1.6 ($PREFIX/share/rosidl_generator_c/cmake)
2024-12-18T12:24:17.1940311Z  │ │ -- Found rosidl_adapter: 3.1.6 ($PREFIX/share/rosidl_adapter/cmake)
2024-12-18T12:24:17.2047688Z  │ │ -- Found rosidl_generator_cpp: 3.1.6 ($PREFIX/share/rosidl_generator_cpp/cmake)
2024-12-18T12:24:17.2224452Z  │ │ -- Using all available rosidl_typesupport_c: rosidl_typesupport_fastrtps_c;rosidl_typesupport_introspection_c
2024-12-18T12:24:17.2464672Z  │ │ -- Using all available rosidl_typesupport_cpp: rosidl_typesupport_fastrtps_cpp;rosidl_typesupport_introspection_cpp
2024-12-18T12:24:17.3439726Z  │ │ CMake Error at CMakeLists.txt:48 (find_package):
2024-12-18T12:24:17.3442406Z  │ │ -- Configuring incomplete, errors occurred!
2024-12-18T12:24:17.3444935Z  │ │   Could not find a configuration file for package "octomap" that is
2024-12-18T12:24:17.3447349Z  │ │   compatible with requested version range "1.9.7...<1.10.0".
2024-12-18T12:24:17.3451490Z  │ │   The following configuration files were considered but not accepted:
2024-12-18T12:24:17.3455376Z  │ │     $PREFIX/share/octomap/octomap-config.cmake, version: 1.10.0
2024-12-18T12:24:17.3570547Z  │ │ ninja: error: loading 'build.ninja': No such file or directory
2024-12-18T12:24:17.3586296Z  │ │ × error Script failed with status 1
2024-12-18T12:24:17.3588159Z  │ │ × error Work directory: '/home/runner/work/ros-humble/ros-humble/output/bld/rattler-build_ros-humble-geometric-shapes_1734523091/work'
2024-12-18T12:24:17.3589777Z  │ │ × error To debug the build, run it manually in the work directory (execute the `./conda_build.sh` or `conda_build.bat` script)
2024-12-18T12:24:17.3590766Z  │ │
2024-12-18T12:24:17.3591141Z  │ ╰─────────────────── (took 1 second)

geometric_shapes do not like octomap 1.10 ?

traversaro · 2024-12-18T12:43:20Z

Ok, the relevant code is https://github.com/moveit/geometric_shapes/blob/8bf32729ca6f217168d54b118c8b60602dfa25a6/CMakeLists.txt#L46 (see moveit/geometric_shapes#241 for more details):

# Enforce the system package version on Ubuntu jammy and noble which is also used by libfcl-dev
# The version is fixed to prevent ABI conflicts with ros-octomap
find_package(octomap 1.9.7...<1.10.0 REQUIRED)

basically on apt packages that were two ABI incompatible versions of octomap, one from upstream Ubuntu and one from ROS, that is exactly what I always feared also for robostack w.r.t. to incompatibilities between conda-forge and robostack (see RoboStack/robostack.github.io#17). The issue is tracked in ros/rosdistro#41622 it is marked as solved but I think it is still there for Jazzy. In a nutshell, I think we can patch the workaround away. However, we may want to double check that we use octomap 1.10.0 consistently across the build.

fyi @EzraBrooks as geometric_shapes is moveit-related and I guess you were interested in a rebuild in ros-humble at some point.

EzraBrooks · 2024-12-18T17:33:58Z

Thanks for continuing to work on this, sorry I disappeared from the conversation - got reprioritized onto other things for the moment.

Appreciate the tag. Octomap is something that's on my back burner to address MoveIt's direct dependency on.. eventually.

traversaro · 2024-12-18T19:32:08Z

New failure is in rviz censored Ogre:

gnu
2024-12-18T14:30:56.0717600Z  │ │ -- CMAKE_PREFIX_PATH: $SRC_DIR/build/ogredeps;$SRC_DIR/build/ogre-v1.12.13-prefix/src/ogre-v1.12.13-build/Dependencies;$SRC_DIR/build/ogre-v1.12.13-prefix/
2024-12-18T14:30:56.0719054Z  │ │ src/ogre-v1.12.13/Dependencies;$SRC_DIR/build/ogre-v1.12.13-prefix/src/ogre-v1.12.13-build/../Dependencies;$SRC_DIR/build/ogre-v1.12.13-prefix/src/ogre-v1.
2024-12-18T14:30:56.0719715Z  │ │ 12.13/../Dependencies;/usr/local;/usr/lib/x86_64-linux-gnu
2024-12-18T14:30:56.0720279Z  │ │ -- Found FREETYPE: $PREFIX/lib/libfreetype.so
2024-12-18T14:30:56.0720990Z  │ │ -- Could NOT find X11 (missing: X11_X11_INCLUDE_PATH) 
2024-12-18T14:30:56.0722858Z  │ │ -- Found OpenGL: $BUILD_PREFIX/x86_64-conda-linux-gnu/sysroot/usr/lib/libGL.so
2024-12-18T14:30:56.0724705Z  │ │ -- Looking for Cg...
2024-12-18T14:30:56.0726644Z  │ │ -- Cg_PREFIX_PATH changed.
2024-12-18T14:30:56.0728591Z  │ │

traversaro · 2024-12-18T19:32:50Z

We probably just need to add the missing dependency (probably before it was satisfied transitively).

traversaro · 2024-12-18T20:13:28Z

Indeed it complains that it can't find X11/X.h header (see https://github.com/Kitware/CMake/blob/bacfc7c313a7cdcb15bc584fdb6350493aa84858/Modules/FindX11.cmake#L158) that indeed is not part of xorg-libx11: https://conda-metadata-app.streamlit.app/?q=conda-forge%2Flinux-64%2Fxorg-libx11-1.8.10-h4f16b4b_1.conda . X11/X.h is found in xorg-xorgproto (I searched for it via https://conda-metadata-app.streamlit.app/Search_by_file_path).

traversaro · 2024-12-18T20:41:45Z

For the second time the build failed with error:

2024-12-18T20:23:57.6753322Z Error: 
2024-12-18T20:23:57.6754227Z   × Failed to resolve dependencies: HTTP status server error (524 <unknown
2024-12-18T20:23:57.6754924Z   │ status code>) for url (https://conda.anaconda.org/robostack-staging/linux-
2024-12-18T20:23:57.6755400Z   │ 64/repodata.json.zst)
2024-12-18T20:23:57.6755866Z   ╰─▶ HTTP status server error (524 <unknown status code>) for url (https://
2024-12-18T20:23:57.6803741Z       conda.anaconda.org/robostack-staging/linux-64/repodata.json.zst)
2024-12-18T20:23:57.6804121Z

it is trivial to restart it, but I guess this may be related to conda/infrastructure#906 .

wolfv · 2024-12-18T22:01:08Z

Damn, I think we should retry more, even on weirder status codes ...

traversaro · 2024-12-18T22:23:32Z

By the way, I have no idea why we pin libpqxx, but probably it make sense to change the pin to 7.9, see https://github.com/conda-forge/libpqxx-feedstock/blob/b0440cf6f893c788c1c8f1bd4c3eb1bb60c26b7e/recipe/meta.yaml#L2-L15 .

wolfv · 2024-12-18T22:30:52Z

conda/infrastructure#1081 btw

sonelu and others added 30 commits August 26, 2024 19:15

build 7 - ros_core - linux_64

dbb99f9

build 7 - ros_base - linux_64

7593258

build 7 - perception - linux_64

f596530

cleanup of duplicate packages in the description of ros_core, ros_bas…

734b28a

…e, perception

build 7 - simulation - linux_64

a6a5412

build 7 - desktop - linux_64

a64192b

build 7 - desktop_full - linux_64

b446058

all packs above --- end of package support ---

bd6755d

Sync with conda-forge pinnings

6055eb1

Sync with conda-forge pinnings

83ce4cc

Bump version, fix string

90bb656

Bump mutex

d83318e

Cleanup

eb89cd1

Try fix mutex package not found

aa6f3e0

Try fix

41d1430

Fix?

d64ae82

Update testpr.yml

5e12c1e

Test

04d0664

Try fix

9c8ee17

Try fix

b3c6d63

Fixup

c07eda1

cleanup comments

ab2582c

cleanup comments

a5073b4

Fix backward_ros patch

352119d

Delete patch/ros-humble-moveit-core.patch

4e59004

It was superseded by moveit/moveit2#2399 .

Delete patch/ros-humble-moveit-resources-prbt-ikfast-manipulator-plug…

3a8ee6c

…in.patch The patch was integrated upstream in moveit/moveit2#2395 .

Delete patch/ros-humble-moveit-simple-controller-manager.patch

0742b27

Patch upstreamed in https://github.com/moveit/moveit2/pull/2895/files .

Explicitly add ros_workspace to packages_select_by_deps as for some r…

6dfd619

…eason is not added

Update conda_build_config.yaml

f32a8cb

Initial attempt to switch to rattler-build

cab2e61

Heavely based on RoboStack/ros-jazzy#5

traversaro added 4 commits December 18, 2024 00:33

merge latest changes in sonelu's linux-64 branches

c37f0ae

Use commit hash for vinca version

1eb787c

Fix compatibility with latest rattler-build

26f955f

Fix compatibility with rattler-build

4c0a3b2

Fix for rattler-build 0.32

3fcb6fd

Bump vinca version for rattler-build 0.32 support

7bb5dfb

Remove patches that assume that PYTHON_INSTALL_DIR is an absolute path

164a49d

Remove conflict leftover

8560574

traversaro mentioned this pull request Dec 18, 2024

Remove patches that assume that PYTHON_INSTALL_DIR is an absolute path #139

Closed

traversaro mentioned this pull request Dec 18, 2024

Linux_64 - rebuild September 2024 #206

Open

Refresh additional recipes and add geometric-shapes patch

4509e27

Add missing xorg-xorgproto package on libx11 dependency

87b901e

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

linux-64 - rebuild Christmas 2024 - switch to rattler-build #229

linux-64 - rebuild Christmas 2024 - switch to rattler-build #229

traversaro commented Dec 17, 2024

traversaro commented Dec 18, 2024

traversaro commented Dec 18, 2024

wolfv commented Dec 18, 2024

traversaro commented Dec 18, 2024

traversaro commented Dec 18, 2024

wolfv commented Dec 18, 2024

traversaro commented Dec 18, 2024

traversaro commented Dec 18, 2024

traversaro commented Dec 18, 2024

EzraBrooks commented Dec 18, 2024

traversaro commented Dec 18, 2024

traversaro commented Dec 18, 2024

traversaro commented Dec 18, 2024

traversaro commented Dec 18, 2024

wolfv commented Dec 18, 2024

traversaro commented Dec 18, 2024

wolfv commented Dec 18, 2024

linux-64 - rebuild Christmas 2024 - switch to rattler-build #229

Are you sure you want to change the base?

linux-64 - rebuild Christmas 2024 - switch to rattler-build #229

Conversation

traversaro commented Dec 17, 2024

traversaro commented Dec 18, 2024

traversaro commented Dec 18, 2024

wolfv commented Dec 18, 2024

traversaro commented Dec 18, 2024

traversaro commented Dec 18, 2024

wolfv commented Dec 18, 2024

traversaro commented Dec 18, 2024

traversaro commented Dec 18, 2024

traversaro commented Dec 18, 2024

EzraBrooks commented Dec 18, 2024

traversaro commented Dec 18, 2024

traversaro commented Dec 18, 2024

traversaro commented Dec 18, 2024

traversaro commented Dec 18, 2024

wolfv commented Dec 18, 2024

traversaro commented Dec 18, 2024

wolfv commented Dec 18, 2024