[WIP] [gazebo10] Add new port #8178

traversaro · 2019-09-15T10:55:33Z

Add new port for the Gazebo simulator and its libraries.
Fix #8014 .

Currently a WIP, as it is needs #8177 and #8136 to be merged before.

Furthermore, at the moment the part of handling executables is a draft, as I am not sure how about the following points:

The gazebo and related executables is a tool for C++ projects in the sense that is a launcher for simulations that contain C++ plugins developed against the Gazebo C++ API . For this reason, on Windows to launch Gazebo plugins compiled in Debug, you need to have the Gazebo executables compiled in Debug, and for plugins compiled in Release, you need the Release executables. As far as I saw, all the ports that install tools only install the release version, as they are typically tools only used for code generation or similar build duties.
What is the correct strategy to adopt in this case? Install the tools under tools/gazebo10/debug ?
Note that there is an ongoing discussion on this in Is there a good reason why the vcpkg cmake toolchain is not adding (/debug)/bin to PATH? #7826, if the tools could be installed in bin and in debug/bin there would be no problem at all. Perhaps @seanyen you had already some plan on this for using vcpkg for generating Chocolatey packages for ROS?
How is tools/executables installation meant to be handled on x64-osx and x64-linux triplets? Some ports I saw identify executables by grepping all the files that end in .exe in the bin directory (see for example

vcpkg/ports/pcl/portfile.cmake

Line 62 in ecfc714

file(GLOB EXEFILES_RELEASE ${CURRENT_PACKAGES_DIR}/bin/*.exe)

) but this is not going to work on *nix triplets.

Rastaban · 2019-09-16T20:03:23Z

I recommend using draft PRs for WIP https://github.blog/2019-02-14-introducing-draft-pull-requests/. It helps up know when you feel it is ready for checkin.

traversaro · 2019-09-19T21:42:59Z

I recommend using draft PRs for WIP https://github.blog/2019-02-14-introducing-draft-pull-requests/. It helps up know when you feel it is ready for checkin.

Thanks, I actually wanted to run some CI on some early version of the port to collect preliminary info, and I was not sure if CI was active for Draft PRs. ~~However, now the port kind of stalled as a non-trivial patch is necessary to compile Gazebo against Ogre 1.12, so I think we can put the PR in draft mode indeed.~~ Actually apparently it is not possible to switch a PR to be a "Draft" once it has been created, see https://github.community/t5/How-to-use-Git-and-GitHub/Feature-Request-Switch-from-ready-to-draft-in-pull-requests/td-p/19107 .

vicroms · 2019-10-08T17:48:00Z

What is the correct strategy to adopt in this case? Install the tools under tools/gazebo10/debug ?

Yes, that would be the correct strategy. We're not in the business of shipping applications with vcpkg so while we allow ports to build executable files, we also require those executables to be moved to a tools/<port>/ directory. You're free to structure things inside tools/<port> however you want, so for this case I would suggest to have a folder for each configuration: tools/gazebo/debug/ and tools/gazebo/release/.

How is tools/executables installation meant to be handled on x64-osx and x64-linux triplets? Some ports I saw identify executables by grepping all the files that end in .exe in the bin directory (see for example

vcpkg/ports/pcl/portfile.cmake

Line 62 in ecfc714

file(GLOB EXEFILES_RELEASE ${CURRENT_PACKAGES_DIR}/bin/*.exe)

) but this is not going to work on *nix triplets.

You're correct, that doesn't work on non-Windows triplets. The correct way to handle is to move the binaries by name, for example:

set(EXECUTABLE_SUFFIX)
if (VCPKG_TARGET_IS_WINDOWS)
  set(EXECUTABLE_SUFFIX ".exe")
endif()

if(NOT DEFINED VCPKG_BUILD_TYPE OR VCPKG_BUILD_TYPE STREQUAL "debug")
  file(COPY "program1${EXECUTABLE_SUFFIX}" DESTINATION "tools/gazebo/debug")
  # other output files to be moved
...

vicroms · 2019-10-08T17:48:38Z

/azp run

traversaro · 2019-10-08T19:27:27Z

Thanks for the reply. Unfortunately the port is still incomplete, as the required amount of patches turned out to be non-negligible, I finally fixed compilation (almost all the changes has been submitted upstream: https://bitbucket.org/osrf/gazebo/pull-requests/) but several post-build vcpkg checks are still failing.

traversaro · 2019-10-30T15:15:27Z

As of b132fee, the port is ready and compiles successfully (at least on my machine). Proper generation of Release and Debug setup.bat is still missing, however during the testing even with properly modified setup.bat, the gzserver.exe tool does not start correctly but just hangs with the [Msg] Waiting for master message:

$ ./gzserver.exe  --verbose
[Msg] Waiting for master.
Gazebo multi-robot simulator, version 10.1.0
Copyright (C) 2012 Open Source Robotics Foundation.
Released under the Apache 2 License.
http://gazebosim.org

I am not sure when I will be able to debug this. I remember experiencing this problem also in the past, @seanyen perhaps do you have some hint on what could be the problem given your experience working with Gazebo on Windows?

stephenjust · 2019-11-12T06:20:18Z

@traversaro Thank you for putting this together, I've had such a headache trying to build Gazebo for Windows on my own.

I've managed to get both gzclient and gzserver running on my machine with the following "setup.bat" script run beforehand:

@REM Add paths to Windows. Run this script prior to running gzserver.exe
@REM and gzclient.ext on a Windows machine.
@REM Meant to be run from inside {installdir}
@set build_type=Release
@if not "%1"=="" set build_type=%1
@echo Configuring for build type %build_type%

@REM Need absolute paths in order to run anything inside install dir
@set gz_root_path=%~dp0
@set gz_build_path=%gz_root_path%tools\gazebo10\%build_type%
@set gz_resource_path=%gz_root_path%share\gazebo-10
@pushd %~dp0
@set deps_path=%CD%\bin
@popd
@echo - script path is %gz_root_path%
@echo - build path is %gz_build_path%
@echo - resource path is %gz_resource_path%
@echo - dependencies path is %deps_path%

@set HOME=%HOMEDRIVE%%HOMEPATH%

@set GAZEBO_MODEL_PATH=%gz_resource_path%models
@set GAZEBO_PLUGINS_PATH=%gz_root_path%bin\gazebo-10\plugins
@set GAZEBO_RESOURCE_PATH=%gz_resource_path%;%gz_root_path%Media
@set OGRE_RESOURCE_PATH=%gz_root_path%bin

@set PATH=%gz_build_path%;%PATH%

There appears to still be some problems with certain resources - particularly shaders with / in the path, and most image files (image codecs are now a plugin you need to include in latest Ogre).

Thank you for the work so far though!

traversaro · 2019-11-12T07:56:31Z

@traversaro Thank you for putting this together, I've had such a headache trying to build Gazebo for Windows on my own.

I've managed to get both gzclient and gzserver running on my machine with the following "setup.bat" script run beforehand:

That is great, thanks a lot for working on this. Eventually, can I take some parts of your setup.bat script to include it in the port?

There appears to still be some problems with certain resources - particularly shaders with / in the path, and most image files (image codecs are now a plugin you need to include in latest Ogre).

Have you tried to add %gz_root_path%/share/gazebo-10 to GAZEBO_RESOURCE_PATH? At a first glance, it seems that is missing w.r.t. to the official setup.bat (see https://bitbucket.org/osrf/gazebo/src/a9352854ba1da2d2ddbad60e43f5bcbc7484e23e/CMakeLists.txt#lines-161 and https://bitbucket.org/osrf/gazebo/src/a9352854ba1da2d2ddbad60e43f5bcbc7484e23e/cmake)./setup.bat.in Sorry, I saw later that that is already included in the script. For which reason you also added there %gz_root_path%Media ?

traversaro · 2019-11-12T08:00:04Z

Have you tried to run your script for launching Gazebo in Debug mode? I think that in that case, also OGRE_RESOURCE_PATH and GAZEBO_PLUGINS_PATH need to be modified to use the Debug version of the plugins.

stephenjust · 2019-11-12T08:13:27Z

Yes, feel free to use parts of the setup.bat in the port. I had derived it from win_addpath.bat

I added %gz_root_path%Media to the resource path but I wasn't sure if it was needed or not, it didn't seem to make any visible difference.

I also haven't tried the debug build, I agree it probably won't work as is.

Preliminary version, not ready for merge.

traversaro · 2019-11-18T20:34:34Z

@stephenjust I saw a few related PR in Gazebo (https://bitbucket.org/osrf/gazebo/pull-requests/3150/fix-texture-loading-on-ogre-111-112/diff and https://bitbucket.org/osrf/gazebo/pull-requests/3151), do you think it make sense to add them as patches to the port?
Btw, if you are able to reliable get to work the port by installing a suitable setup.bat or win_addpath.bat, feel free to open a new improved PR to add Gazebo in vcpkg. I do not have a lot of time to work on it, so if you are able to get it merged in the main vcpkg repo, it would be great!

JackBoosY · 2020-01-14T04:28:00Z

/azp run

JackBoosY · 2020-01-16T07:23:23Z

ports/gazebo10/gazebo-3131.patch

+   Ogre::String programName = this->baseName + "VP_" +
+                              Ogre::StringConverter::toString(_permutation);
+
+-#if OGRE_DEBUG_MODE


Why not force OGRE_DEBUG_MODE to 0 in CMakeLists.txt?

Because OGRE_DEBUG_MODE is already defned to 1 in OGRE headers if OGRE is compiled in Debug, so forcing it to 0 could create problems in the compilation of OGRE headers. See the related upstream PR : https://bitbucket.org/osrf/gazebo/pull-requests/3131 .

JackBoosY · 2020-01-16T07:24:48Z

ports/gazebo10/portfile.cmake

+file(INSTALL ${SOURCE_PATH}/LICENSE DESTINATION ${CURRENT_PACKAGES_DIR}/share/gazebo10 RENAME copyright)
+
+# Post-build test for cmake libraries
+vcpkg_test_cmake(PACKAGE_NAME gazebo)


vcpkg_test_cmake is deprecated, please remove this line.

JackBoosY · 2020-01-16T07:25:37Z

ports/gazebo10/portfile.cmake

+
+# Remove debug files
+file(REMOVE_RECURSE ${CURRENT_PACKAGES_DIR}/debug/include 
+                    ${CURRENT_PACKAGES_DIR}/debug/lib/cmake


${CURRENT_PACKAGES_DIR}/debug/lib/cmake should be deleted by vcpkg_fixup_cmake_targets automatically.

Good catch, thanks!

JackBoosY · 2020-01-16T07:26:06Z

ports/gazebo10/portfile.cmake

+  set(EXECUTABLE_SUFFIX ".exe")
+endif()
+
+if(NOT DEFINED VCPKG_BUILD_TYPE OR VCPKG_BUILD_TYPE STREQUAL "release")


I prefer to use patch to fix the installation path with tools.

Just to understand, what is the rational to prefer a patch for this change that will never be upstreamed, as the installation of executables in tools/<port> is a vcpkg peculiarity?
Do you have a good example port that can be used as an example for a patch that fixes tool installation?
The current strategy was proposed by @vicroms in #8178 (comment) .

JackBoosY · 2020-01-16T07:27:43Z

This PR contains too many patches, maybe we can wait for upstream to accept them and use the new version?

traversaro

Some replies.

traversaro · 2020-01-16T08:23:00Z

This PR contains too many patches, maybe we can wait for upstream to accept them and use the new version?

I think this is indeed a good idea, as Gazebo 11 that should contain all the proposed patch should be released at the end of January, and then we can add a gazebo11 port. Furthermore, as mentioned in #8178 (comment) I am not able to use the existing port, so we would need at least someone that is able to correctly use the port to actually merge it.

traversaro · 2020-03-16T21:30:20Z

As of b132fee, the port is ready and compiles successfully (at least on my machine). Proper generation of Release and Debug setup.bat is still missing, however during the testing even with properly modified setup.bat, the gzserver.exe tool does not start correctly but just hangs with the [Msg] Waiting for master message:
$ ./gzserver.exe  --verbose
[Msg] Waiting for master.
Gazebo multi-robot simulator, version 10.1.0
Copyright (C) 2012 Open Source Robotics Foundation.
Released under the Apache 2 License.
http://gazebosim.org
I am not sure when I will be able to debug this. I remember experiencing this problem also in the past, @seanyen perhaps do you have some hint on what could be the problem given your experience working with Gazebo on Windows?

For future reference, I had this problem also with Gazebo compiled outside of vcpkg because I had a rather old Windows 10 version (1709). By updating to Windows 10 1903, the problem was fixed.

Rastaban assigned vicroms Sep 16, 2019

vicroms added the wip label Sep 18, 2019

traversaro closed this Sep 19, 2019

traversaro reopened this Sep 19, 2019

traversaro force-pushed the addGazebo10 branch from 0914f4f to b132fee Compare October 30, 2019 15:10

[gazebo10] Add new port

e7836e5

Preliminary version, not ready for merge.

traversaro force-pushed the addGazebo10 branch from 91a661d to e7836e5 Compare November 18, 2019 20:02

PhoebeHui assigned JackBoosY Jan 14, 2020

JackBoosY suggested changes Jan 16, 2020

View reviewed changes

traversaro commented Jan 16, 2020

View reviewed changes

traversaro closed this Jan 16, 2020

traversaro mentioned this pull request Jan 16, 2020

[New Port Request] Gazebo #8014

Closed

This was referenced Apr 20, 2020

Terrain rendering does not support shadows and texture blending with Ogre >= 1.11 gazebosim/gazebo-classic#2686

Open

Gazebo 11 does not run correctly with Ogre 1.12 from conda-forge gazebosim/gazebo-classic#2700

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] [gazebo10] Add new port #8178

[WIP] [gazebo10] Add new port #8178

traversaro commented Sep 15, 2019 •

edited

Loading

Rastaban commented Sep 16, 2019 •

edited

Loading

traversaro commented Sep 19, 2019 •

edited

Loading

vicroms commented Oct 8, 2019

vicroms commented Oct 8, 2019

traversaro commented Oct 8, 2019

traversaro commented Oct 30, 2019

stephenjust commented Nov 12, 2019 •

edited

Loading

traversaro commented Nov 12, 2019 •

edited

Loading

traversaro commented Nov 12, 2019

stephenjust commented Nov 12, 2019

traversaro commented Nov 18, 2019

JackBoosY commented Jan 14, 2020

JackBoosY Jan 16, 2020

traversaro Jan 16, 2020 •

edited

Loading

JackBoosY Jan 16, 2020

traversaro Jan 16, 2020

JackBoosY Jan 16, 2020

traversaro Jan 16, 2020

JackBoosY Jan 16, 2020

traversaro Jan 16, 2020

JackBoosY commented Jan 16, 2020

traversaro left a comment

traversaro commented Jan 16, 2020

traversaro commented Mar 16, 2020

[WIP] [gazebo10] Add new port #8178

[WIP] [gazebo10] Add new port #8178

Conversation

traversaro commented Sep 15, 2019 • edited Loading

Rastaban commented Sep 16, 2019 • edited Loading

traversaro commented Sep 19, 2019 • edited Loading

vicroms commented Oct 8, 2019

vicroms commented Oct 8, 2019

traversaro commented Oct 8, 2019

traversaro commented Oct 30, 2019

stephenjust commented Nov 12, 2019 • edited Loading

traversaro commented Nov 12, 2019 • edited Loading

traversaro commented Nov 12, 2019

stephenjust commented Nov 12, 2019

traversaro commented Nov 18, 2019

JackBoosY commented Jan 14, 2020

JackBoosY Jan 16, 2020

Choose a reason for hiding this comment

traversaro Jan 16, 2020 • edited Loading

Choose a reason for hiding this comment

JackBoosY Jan 16, 2020

Choose a reason for hiding this comment

traversaro Jan 16, 2020

Choose a reason for hiding this comment

JackBoosY Jan 16, 2020

Choose a reason for hiding this comment

traversaro Jan 16, 2020

Choose a reason for hiding this comment

JackBoosY Jan 16, 2020

Choose a reason for hiding this comment

traversaro Jan 16, 2020

Choose a reason for hiding this comment

JackBoosY commented Jan 16, 2020

traversaro left a comment

Choose a reason for hiding this comment

traversaro commented Jan 16, 2020

traversaro commented Mar 16, 2020

traversaro commented Sep 15, 2019 •

edited

Loading

Rastaban commented Sep 16, 2019 •

edited

Loading

traversaro commented Sep 19, 2019 •

edited

Loading

stephenjust commented Nov 12, 2019 •

edited

Loading

traversaro commented Nov 12, 2019 •

edited

Loading

traversaro Jan 16, 2020 •

edited

Loading