From 5ad8c8a79abd4faa2b39f86d80882b1d7410483e Mon Sep 17 00:00:00 2001 From: Gareth Morgan Date: Fri, 24 Mar 2023 16:09:17 -0400 Subject: [PATCH] Release v23.03 (#74) * Added libXt as a pre-required library on Linux MaterialX (MaterialXRenderHw) requires libXt to build. Ubuntu 20.04 may have it installed with system installation. To be safe, I added it in the doc so users can build MaterialX without the missing libXt error. * OGSMOD-2637: Fixed deployHdAurora of building and running usdview * OGSMOD-2642: Improved error message on missing Windows SDK * Fix typo in README.md * Fix CMake error message * Disable failed cases that cause CI/CD errors (#53) * Disable failed cases * Minor fixes in documentation * Fix typos * OGSMOD-2658: Refactored the externals to install to individual prefix location Each and every external library will be installed to its own installation path. This allows an easy override with _ROOT for users to use their own external libraries. NOTE: this commit breaks the Aurora build * OGSMOD-2679: Upgraded Aurora build scripts to support multi-config build in Visual Studio - Updated cmake files to work with the new externals layout - An externals config file is auto-generated by installExternals.py for easy configuration with 'cmake -S . -B Build'. - Created patches for OpenImageIO and USD to support multi-config build. - Created customized find modules of USD, TBB and OpenSubdiv to support multi-config build. * OGSMOD-2679: Updated findNRD/NRI to support multi-config build * OGSMOD-2679: Improved of the build scripts and the end message of the external script * OGSMOD-2680: Fixed the Linux build with changes of the multi-config support - Fixed the cmake files to build on Linux - Preset the patch file to always have LF line-ending to fix the patching errors. - Updated the default externals config * OGSMOD-2681: Updated README.md to sync with the changes of the Aurora build scripts * Windows pipeline for Aurora Build/Test in CI/CD (#55) * Windows pipeline for Aurora Build/Test in CI/CD * Move build instructions to a separate file * Removed RelWithDebInfo from the build variants of installExternals.py * Added error checking on missing the externals config file Existing users who ran installExternals.py do not have the externals config file. The old externals installation is not compatible to the current build scripts. This error reminds users to re-run the install script. * OGSMOD-2803: Fixed the build error with ENABLE_HGI_BACKEND=ON on Windows * OGSMOD-2704: Built usdview by installExternals.py - Updated externals script to build USD with python - Built usdview as well - Updated cmake scripts to build Aurora with the updated USD build - Deployed usdview (and other USD executables) with HdAurora after HdAurora is built. Batch codes to build Aurora/externals and run usdview: set BUILD_TYPE=Release set BUILD_DIR=Build.release set EXTERNALS_DIR=%HOME%\AuroraExternals.release python Scripts\installExternals.py --build-variant %BUILD_TYPE% "%EXTERNALS_DIR%" cmake -S . -B %BUILD_DIR% cmake --build %BUILD_DIR% --config %BUILD_TYPE% set PYTHONPATH=%cd%\%BUILD_DIR%\bin\%BUILD_TYPE%\python set PATH=%PATH%;%cd%\%BUILD_DIR%\bin\%BUILD_TYPE% usdview model.usd Note usdview only works with the release build of USD. Aurora builds the debug build USD with the 'd' suffixed to the dlls. Since usdview hardcodes the loaded dlls without suffix 'd', it cannot load the needed dlls. * OGSMOD-2859: Support building on Ubuntu 22.04 With gcc 11.3.0 clang 14.0.0 We need boost 1.78 to build on newer compiler. The script to build Aurora: ``` BUILD_TYPE=Release BUILD_DIR=Build.$BUILD_TYPE EXTERNALS_DIR=AuroraExternals.$BUILD_TYPE CC=gcc CXX=g++ compiler=gcc python Scripts/installExternals.py -v --build-variant $BUILD_TYPE "$EXTERNALS_DIR" CC=clang CXX=clang++ compiler=clang cmake -S . -B $BUILD_DIR -D CMAKE_BUILD_TYPE=$BUILD_TYPE -D ENABLE_HGI_BACKEND=ON CC=clang CXX=clang++ compiler=clang cmake --build $BUILD_DIR ``` * OGSMOD-2875: Fixed the HdAurora build on system with multiple installed Pythons * OGSMOD-2858: updated the doc on how to run usdview within the Aurora build tree * Fix CI/CD build errors (#64) We get type conversion warnings treated as errors. Its because we get the master version of STB in installExternals: and there was a change in STB that introduced the warning, hence the problem So we disable the warning 4244 to fix the error Co-authored-by: svc_ogs_td * OGSMOD-2616 : Windows pipeline for Aurora simple deploy in CI/CD (#63) * Windows pipeline for Aurora simple deploy in CI/CD * OGSMOD-3009: upgrade zlib to 1.2.13 (security patch) * OGSMOD-2990: Clone STB using an explicit SHA This commit fixed the issue of "changing" master HEAD of STB. * OGSMOD-3083: Picking the correct python version from the USD/boost This commit solve the potential version mismatching issue of python between Aurora and USD/boost. * Merge from main->dev to remove conflicts (#73) Correct dev history by merging main. * Add prerequisites to build documentation (#68) * Add prerequisites to build documentation * Fix typo * Add Linux * Add dependency checking to installExternals.py (#75) * Fix typo * Add versioning * Remove spew * Add comments * Use hash not URL Co-authored-by: Andy Shiue Co-authored-by: Mauricio Vives Co-authored-by: Liwei Zhan Co-authored-by: Andy Shiue Co-authored-by: svc_ogs_td --- CMakeLists.txt | 32 ++- Doc/Build.md | 34 ++++ Doc/HdAurora.md | 28 ++- Jenkinsfile | 5 + Libraries/Aurora/CMakeLists.txt | 16 +- Libraries/Aurora/Source/AssetManager.cpp | 5 + Libraries/HdAurora/CMakeLists.txt | 40 +++- Scripts/cmake/modules/Findpxr.cmake | 101 +++++++--- Scripts/installExternals.py | 238 +++++++++++++++++++---- Scripts/setUSDEnv.bat | 8 + Tests/Helpers/AuroraTestHelpers.cpp | 6 + 11 files changed, 433 insertions(+), 80 deletions(-) create mode 100644 Scripts/setUSDEnv.bat diff --git a/CMakeLists.txt b/CMakeLists.txt index e28aa71..fcf5e6f 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -12,10 +12,25 @@ project(Aurora VERSION 0.0.0.1) set(AURORA_ROOT_DIR ${PROJECT_SOURCE_DIR}) +# install path if the user did not explicitly specify one. +if(CMAKE_INSTALL_PREFIX_INITIALIZED_TO_DEFAULT) + set(CMAKE_INSTALL_PREFIX "${PROJECT_BINARY_DIR}/Installed" CACHE PATH "..." FORCE) +endif() + +# Set the default install directories +set(INSTALL_BIN "$/bin") +set(INSTALL_LIB "$/lib") +set(INSTALL_INC "$/include") + # Set the default output directories set(RUNTIME_OUTPUT_DIR "${PROJECT_BINARY_DIR}/bin/$") set(LIBRARY_OUTPUT_DIR "${PROJECT_BINARY_DIR}/lib/$") +add_custom_target(MakeRuntimeDir ALL + COMMAND ${CMAKE_COMMAND} -E make_directory ${RUNTIME_OUTPUT_DIR} +) +set_property(TARGET MakeRuntimeDir PROPERTY FOLDER "Deployment") + # Set scripts folder. set(SCRIPTS_DIR ${AURORA_ROOT_DIR}/Scripts) @@ -25,6 +40,10 @@ list(APPEND CMAKE_MODULE_PATH "${SCRIPTS_DIR}/cmake/modules" ) +option(ENABLE_TESTS "Build unit tests" ON) +option(ENABLE_APPLICATIONS "Build Applications" ON) +option(ENABLE_USDVIEW "Build usdview with python" ON) + # Import the cmake utility functions include(toolbox) @@ -51,6 +70,8 @@ if(MSVC) add_compile_options(/W4 /WX -wd4068) # -wd4068 disables unknown-pragmas warnings # Enable multiple-processor compilation add_compile_options(/MP) + # Add INSTALL project to solution by default. + set(CMAKE_VS_INCLUDE_INSTALL_TO_DEFAULT_BUILD 1) else() # Enables strict standard conformance and warning as errors add_compile_options(-Wall -Wextra -Wpedantic -Werror -Wno-unknown-pragmas -Wno-gnu-zero-variadic-macro-arguments) @@ -78,5 +99,12 @@ else() endif() add_subdirectory(Libraries) -add_subdirectory(Tests) -add_subdirectory(Applications) + +# Unit tests +if (ENABLE_TESTS) + add_subdirectory(Tests) +endif() + +if (ENABLE_APPLICATIONS) + add_subdirectory(Applications) +endif() diff --git a/Doc/Build.md b/Doc/Build.md index 6ea1af8..4990830 100644 --- a/Doc/Build.md +++ b/Doc/Build.md @@ -1,4 +1,38 @@ # Building + +## Prerequisites + +Several prerequisites must be installed before building Aurora. + +### Windows +On windows the following packages should be installed and added to the system PATH environment variable: +* Microsoft Visual Studio 2019 (https://my.visualstudio.com/Downloads?q=visual%20studio%202019) +* CMake 3.26.11 (https://github.com/Kitware/CMake/releases/download/v3.26.1/cmake-3.26.1-windows-x86_64.msi) +* Python 3.9.13 (https://www.python.org/downloads/release/python-3913/) +* PySide6 python package (install with `pip3 install pyside6`) +* PyOpenGL python package (install with `pip3 install PyOpenGL`) +* NASM 2.16 (https://www.nasm.us/pub/nasm/releasebuilds/2.16.01/win64/nasm-2.16.01-installer-x64.exe) +* Vulkan SDK (https://vulkan.lunarg.com/sdk/home#windows). Should be accessible via the VULKAN_SDK environment variable. + +### Linux +The following dependencies are required on Linux. The versions listed are the recommended version for Ubuntu 20.04: +* zlib1g-dev: 1.2.11 +* libjpeg-turbo8-dev: 2.0.3 +* libtiff-dev: 4.1.0 +* libpng-dev: 1.6.37 +* libglm-dev: 0.9.9.7 +* libglew-dev: 2.1.0 +* libglfw3-dev: 3.3.2 +* libgtest-dev: 1.10.0 +* libgmock-dev: 1.10.0 + +On Ubuntu 20.04, these can be installed with the following Advanced Package Tool command: +``` +sudo apt-get -y install zlib1g-dev libjpeg-turbo8-dev libtiff-dev libpng-dev libglm-dev libglew-dev libglfw3-dev libgtest-dev libgmock-dev +``` + +## Building Aurora + Aurora includes a script that retrieves and builds dependencies ("externals") from source. This script is based on the [USD build script](https://github.com/PixarAnimationStudios/USD/tree/release/build_scripts). CMake is used to build Aurora directly, or to create an IDE project which can then be used to build and debug Aurora. 1. **Download or clone** the contents of this repository to a location of your choice. Cloning with Git is not strictly necessary as Aurora does not currently use submodules. We refer to this location as *AURORA_DIR*. diff --git a/Doc/HdAurora.md b/Doc/HdAurora.md index c20152d..329a622 100644 --- a/Doc/HdAurora.md +++ b/Doc/HdAurora.md @@ -16,19 +16,39 @@ Once **HdAurora** has been deployed to your USD folder you can select HdAurora i ## Deploying HdAurora +In this section, we explain how to run usdview using HdAurora with the default Aurora build, and how to deploy HdAurora to a custom USD installation if it is preferred. The commands in this section require "x64 Native Tools Command Prompt for VS 2019". -The provided Python script [deployHdAurora.py](../Scripts/deployHdAurora.py) can be used to deploy **HdAurora** to a USD installation. This can also create a new USD installation if one does not exist, using the `--build` option. +### Running usdview with Default Aurora Build +If Aurora is built with the default build steps, **usdview** and required USD binaries are automatically deployed with HdAurora binaries. Assuming the root of Aurora source tree is *AURORA_ROOT*, the following build steps deploy **usdview** automatically: +``` +cd %AURORA_ROOT% +python Scripts\installExternals.py --build-variant Release ..\AuroraExternals +cmake -S . -B Build +cmake --build Build --config Release +``` +Before running **usdview** with **HdAurora**, you need to configure the run environment with: +``` +set PYTHONPATH=%AURORA_ROOT%\Build\bin\Release\python +set PATH=%PATH%;%AURORA_ROOT%\Build\bin\Release +``` +To render the [Autodesk Telescope USD model](https://drive.google.com/file/d/1RM09qDOGcRinLJTbXCsiRfQrHmKA-1aN/view?usp=share_link) with **HdAurora**, simply extract the ZIP file into *ASSET_DIR*, and run the following command. +``` +usdview %ASSET_DIR%\AutodeskTelescope.usda --renderer=Aurora +``` -Run the following command to create a new USD installation located at *..\USD* relative to the Aurora repository root and deploy **HdAurora** to it. Creating a new USD installation requires a command prompt with compiler tools, such as "x64 Native Tools Command Prompt for VS 2019", and should be run from the root of the Aurora repository. This will take about 30 minutes to complete and will require 10 GB of disk space. +### Deploying HdAurora to Custom USD Installation +For users who want to use their own USD build, Aurora provides [deployHdAurora.py](../Scripts/deployHdAurora.py) to deploy **HdAurora** to the custom USD installation. [deployHdAurora.py](../Scripts/deployHdAurora.py) can also create a new USD installation if one does not exist, using the `--build` option. +Run the following command to build a new USD installation located at *..\USD* relative to the the root of Aurora source tree ( *AURORA_ROOT*). This will take about 30 minutes to complete and will require 10 GB of disk space. ``` +cd %AURORA_ROOT% python Scripts\deployHdAurora.py ..\USD --externals_folder=..\AuroraExternals --config=Release --build ``` -You can then run **usdview** from the *bin* subdirectory within that installed USD directory. The example command below loads the [Autodesk Telescope USD model](https://drive.google.com/file/d/1RM09qDOGcRinLJTbXCsiRfQrHmKA-1aN/view?usp=share_link); simply extract the ZIP file into a directory, referred to as *ASSET_DIR* below. +You can then run **usdview** from the *bin* subdirectory within that installed USD directory. To render the [Autodesk Telescope USD model](https://drive.google.com/file/d/1RM09qDOGcRinLJTbXCsiRfQrHmKA-1aN/view?usp=share_link) with **HdAurora**, simply extract the ZIP file into *ASSET_DIR* and run the following commands. ``` cd ..\USD\bin -python usdview {ASSET_DIR}\AutodeskTelescope.usda --renderer=Aurora +usdview %ASSET_DIR%\AutodeskTelescope.usda --renderer=Aurora ``` diff --git a/Jenkinsfile b/Jenkinsfile index e429793..4aa802e 100644 --- a/Jenkinsfile +++ b/Jenkinsfile @@ -50,6 +50,11 @@ def windowsBuild() { if not exist ${EXTERNALS_DIR}\\nasm-2.15.05 call 7z x -y -aoa ${EXTERNALS_DIR}\\nasm-2.15.05-win64.zip -o${EXTERNALS_DIR} call set PATH=${EXTERNALS_DIR}\\nasm-2.15.05;%PATH% + :: Set up Pyside6/PyOpenGL + call python -m pip install --upgrade pip + call python -m pip install PySide6 + call python -m pip install PyOpenGL + :: Set up Vulkan SDK :: We need to install Vulkan SDK silently. For more details please refer to https://vulkan.lunarg.com/doc/view/latest/windows/getting_started.html if not exist ${EXTERNALS_DIR}\\VulkanSDK-1.3.231.1-Installer.exe call curl -o ${EXTERNALS_DIR}\\VulkanSDK-1.3.231.1-Installer.exe https://sdk.lunarg.com/sdk/download/1.3.231.1/windows/VulkanSDK-1.3.231.1-Installer.exe diff --git a/Libraries/Aurora/CMakeLists.txt b/Libraries/Aurora/CMakeLists.txt index 7b41603..cc358d5 100644 --- a/Libraries/Aurora/CMakeLists.txt +++ b/Libraries/Aurora/CMakeLists.txt @@ -254,11 +254,6 @@ source_group("DirectX/Shaders" FILES ${DIRECTX_PRECOMPILED_SHADERS}) source_group("HGI/Shaders" FILES ${HGI_SHADERS}) if(WIN32) - add_custom_target(MakeRuntimeDir ALL - COMMAND ${CMAKE_COMMAND} -E make_directory ${RUNTIME_OUTPUT_DIR} - ) - set_property(TARGET MakeRuntimeDir PROPERTY FOLDER "Deployment") - if(ENABLE_DIRECTX_BACKEND) # Create custom targets that will copy DLLs of the DirectX shader compiler to the runtime folder. add_custom_target(CopyDXCompilerDLLs ALL @@ -328,8 +323,8 @@ if(WIN32) # Copy additional HGI files of the Pixar USD to our runtime directory add_custom_target(CopyUSDDLLs ALL COMMAND ${CMAKE_COMMAND} -E make_directory ${RUNTIME_OUTPUT_DIR}/usd/hgiVulkan/resources - COMMAND ${CMAKE_COMMAND} -E copy_if_different ${pxr_DIR}/lib/usd/plugInfo.json ${RUNTIME_OUTPUT_DIR}/usd - COMMAND ${CMAKE_COMMAND} -E copy_if_different ${pxr_DIR}/lib/usd/hgiVulkan/resources/plugInfo.json ${RUNTIME_OUTPUT_DIR}/usd/hgiVulkan/resources + COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PXR_LIBRARY_DIR}/usd/plugInfo.json ${RUNTIME_OUTPUT_DIR}/usd + COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PXR_LIBRARY_DIR}/usd/hgiVulkan/resources/plugInfo.json ${RUNTIME_OUTPUT_DIR}/usd/hgiVulkan/resources ) set_property(TARGET CopyUSDDLLs PROPERTY FOLDER "Deployment") add_dependencies(CopyUSDDLLs MakeRuntimeDir) @@ -425,3 +420,10 @@ if(ENABLE_DIRECTX_BACKEND) # TODO: do we have Vulkan computer shaders amd need to do similiar things for them? endif() + +install(TARGETS ${PROJECT_NAME} + RUNTIME DESTINATION "${INSTALL_BIN}" + ARCHIVE DESTINATION "${INSTALL_LIB}" + LIBRARY DESTINATION "${INSTALL_LIB}" + COMPONENT ${PROJECT_NAME} +) diff --git a/Libraries/Aurora/Source/AssetManager.cpp b/Libraries/Aurora/Source/AssetManager.cpp index e6bcb92..0f63462 100644 --- a/Libraries/Aurora/Source/AssetManager.cpp +++ b/Libraries/Aurora/Source/AssetManager.cpp @@ -17,8 +17,13 @@ // Include STB for image loading. // TODO: Image loading will eventually be handled by clients. +#pragma warning(push) +// Disabe type conversion warnings intruduced from stb master. +// refer to the commit in stb https://github.com/nothings/stb/commit/b15b04321dfd8a2307c49ad9c5bf3c0c6bcc04cc +#pragma warning(disable : 4244) #define STB_IMAGE_IMPLEMENTATION #include +#pragma warning(pop) BEGIN_AURORA diff --git a/Libraries/HdAurora/CMakeLists.txt b/Libraries/HdAurora/CMakeLists.txt index e8f7fd8..3b99e5b 100644 --- a/Libraries/HdAurora/CMakeLists.txt +++ b/Libraries/HdAurora/CMakeLists.txt @@ -59,4 +59,42 @@ PRIVATE ) # Add default compile definitions (set in root CMakefile) -target_compile_definitions(${PROJECT_NAME} PRIVATE ${DEFAULT_COMPILE_DEFINITIONS}) \ No newline at end of file +target_compile_definitions(${PROJECT_NAME} PRIVATE ${DEFAULT_COMPILE_DEFINITIONS}) + +if(WIN32) + # Create a custom target that will copy USD executables to the runtime folder. + file(GLOB USD_EXECUTABLES ${pxr_ROOT}/bin/*) + set(USD_PYTHON_LIB_DIR "${pxr_ROOT}/lib/python") + set(USD_LIB_PLUGIN_DIR "${pxr_ROOT}/lib/usd") + set(USD_PLUGIN_DIR "${pxr_ROOT}/plugin/usd") + add_custom_target(CopyUSDExecutables ALL + COMMAND ${CMAKE_COMMAND} -E copy_if_different ${USD_EXECUTABLES} ${RUNTIME_OUTPUT_DIR} + COMMAND ${CMAKE_COMMAND} -E copy_directory ${USD_LIB_PLUGIN_DIR} ${RUNTIME_OUTPUT_DIR}/usd + COMMAND ${CMAKE_COMMAND} -E copy_directory ${USD_PLUGIN_DIR} ${RUNTIME_OUTPUT_DIR}/usd + ) + set_property(TARGET CopyUSDExecutables PROPERTY FOLDER "Deployment") + add_dependencies(CopyUSDExecutables MakeRuntimeDir) + add_dependencies(${PROJECT_NAME} CopyUSDExecutables) + + if(ENABLE_USDVIEW) + add_custom_target(CopyUSDPythonExecutables ALL + COMMAND ${CMAKE_COMMAND} -E copy_directory ${USD_PYTHON_LIB_DIR} ${RUNTIME_OUTPUT_DIR}/python + ) + set_property(TARGET CopyUSDPythonExecutables PROPERTY FOLDER "Deployment") + add_dependencies(CopyUSDPythonExecutables MakeRuntimeDir) + add_dependencies(${PROJECT_NAME} CopyUSDPythonExecutables) + endif() + + set(HDAURORA_PLUGIN_RESOURCES_DIR ${CMAKE_CURRENT_SOURCE_DIR}/resources) + add_custom_command(TARGET ${PROJECT_NAME} POST_BUILD + COMMAND ${CMAKE_COMMAND} -E copy_if_different $ $<$:$> ${RUNTIME_OUTPUT_DIR}/usd + COMMAND ${CMAKE_COMMAND} -E copy_directory ${HDAURORA_PLUGIN_RESOURCES_DIR} ${RUNTIME_OUTPUT_DIR}/usd/hdAurora/resources + ) + +else() # Linux + #TODO anything we need to set to support the Aurora execution? +endif() + +# Install the binaries. +install(TARGETS ${PROJECT_NAME} DESTINATION ${INSTALL_BIN}/usd) +install(FILES "resources/plugInfo.json" DESTINATION ${INSTALL_BIN}/usd/hdAurora/resources) \ No newline at end of file diff --git a/Scripts/cmake/modules/Findpxr.cmake b/Scripts/cmake/modules/Findpxr.cmake index ee004c3..1c5a19e 100644 --- a/Scripts/cmake/modules/Findpxr.cmake +++ b/Scripts/cmake/modules/Findpxr.cmake @@ -3,10 +3,37 @@ if(TARGET pxr::usd) return() endif() +find_library(USD_usdviewq_LIBRARY_RELEASE usd_usdviewq) +find_library(USD_usdviewq_LIBRARY_DEBUG usd_usdviewqd) +if(USD_usdviewq_LIBRARY_RELEASE OR USD_usdviewq_LIBRARY_DEBUG) + # USD requires Python for usdview. If usdview librray exists, Aurora would + # need to link to Boost python component and python librray as well. + set(USD_COMPOMPONENTS_USDVIEW "True") -if(NOT Boost_FOUND) - find_package(Boost REQUIRED) + set(Boost_USE_STATIC_LIBS OFF) + set(Boost_NO_BOOST_CMAKE ON) + + # Iterate through the potential python3 versions to locate the Boost python component. + set(PYTHON_MINOR_VERSIONS 12 11 10 9 8 7) + foreach(_ver ${PYTHON_MINOR_VERSIONS}) + set(PYTHON_COMPONENT python3${_ver}) + find_package(Boost REQUIRED OPTIONAL_COMPONENTS ${PYTHON_COMPONENT}) + + if(Boost_${PYTHON_COMPONENT}_FOUND) + # With the found Boost python component, we can now locate the python library to link to. + add_library(Boost::python ALIAS Boost::${PYTHON_COMPONENT}) + find_package(Python3 3.${_ver} EXACT COMPONENTS Development) + + set(PYTHON_INCLUDE_DIRS "${Python3_INCLUDE_DIRS}") + set(PYTHON_LIBRARIES ${Python3_LIBRARIES}) + + break() + endif() + endforeach() +else() + set(USD_COMPOMPONENTS_USDVIEW "False") endif() + if(NOT TBB_FOUND) find_package(TBB REQUIRED) endif() @@ -34,6 +61,10 @@ set(PXR_LIBRARY_DIRS "${PXR_LIBRARY_DIR}") # Configure all USD targets set(USD_COMPOMPONENTS arch tf gf js trace work plug vt ar kind sdf ndr sdr pcp usd usdGeom usdVol usdMedia usdShade usdLux usdProc usdRender usdHydra usdRi usdSkel usdUI usdUtils usdPhysics garch hf hio cameraUtil pxOsd glf hgi hgiGL hgiVulkan hgiInterop hd hdGp hdsi hdSt hdx usdImaging usdImagingGL usdProcImaging usdRiImaging usdSkelImaging usdVolImaging usdAppUtils) +if(USD_COMPOMPONENTS_USDVIEW) + list(APPEND USD_COMPOMPONENTS "usdviewq") +endif() + foreach(_comp ${USD_COMPOMPONENTS}) add_library(${_comp} SHARED IMPORTED) set_target_properties(${_comp} PROPERTIES @@ -98,7 +129,9 @@ foreach(_comp ${USD_COMPOMPONENTS}) endif() endif() - add_library(pxr::${_comp} ALIAS ${_comp}) + if (USD_${_comp}_LIBRARY_RELEASE OR USD_${_comp}_LIBRARY_DEBUG) + add_library(pxr::${_comp} ALIAS ${_comp}) + endif() endforeach() if (USD_LIBRARIES_RELEASE) @@ -114,9 +147,9 @@ if(WIN32) INTERFACE_LINK_LIBRARIES "Ws2_32.lib;Dbghelp.lib" ) set_target_properties(tf PROPERTIES - INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" - INTERFACE_LINK_LIBRARIES "arch;Shlwapi.lib;TBB::tbb" - INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" + INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};$<$:${PYTHON_INCLUDE_DIRS}>;${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" + INTERFACE_LINK_LIBRARIES "arch;Shlwapi.lib;TBB::tbb;$<$:${PYTHON_LIBRARIES};Boost::python>" + INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "$<$:${PYTHON_INCLUDE_DIRS}>;${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" ) else() set_target_properties(arch PROPERTIES @@ -124,9 +157,9 @@ else() INTERFACE_LINK_LIBRARIES "dl;m" ) set_target_properties(tf PROPERTIES - INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" - INTERFACE_LINK_LIBRARIES "arch;TBB::tbb" - INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" + INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};$<$:${PYTHON_INCLUDE_DIRS}>;${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" + INTERFACE_LINK_LIBRARIES "arch;TBB::tbb;$<$:${PYTHON_LIBRARIES};Boost::python>" + INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "$<$:${PYTHON_INCLUDE_DIRS}>;${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" ) endif() set_target_properties(gf PROPERTIES @@ -140,7 +173,7 @@ set_target_properties(js PROPERTIES ) set_target_properties(trace PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" - INTERFACE_LINK_LIBRARIES "arch;js;tf;TBB::tbb" + INTERFACE_LINK_LIBRARIES "arch;js;tf;TBB::tbb;$<$:Boost::python>" INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" ) set_target_properties(work PROPERTIES @@ -150,17 +183,17 @@ set_target_properties(work PROPERTIES ) set_target_properties(plug PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" - INTERFACE_LINK_LIBRARIES "arch;tf;js;trace;work;TBB::tbb" + INTERFACE_LINK_LIBRARIES "arch;tf;js;trace;work;TBB::tbb;$<$:Boost::python>" INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" ) set_target_properties(vt PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" - INTERFACE_LINK_LIBRARIES "arch;tf;gf;trace;TBB::tbb" + INTERFACE_LINK_LIBRARIES "arch;tf;gf;trace;TBB::tbb;$<$:Boost::python>" INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" ) set_target_properties(ar PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};${Boost_INCLUDE_DIRS}" - INTERFACE_LINK_LIBRARIES "arch;js;tf;plug;vt" + INTERFACE_LINK_LIBRARIES "arch;js;tf;plug;vt;$<$:Boost::python>" INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${Boost_INCLUDE_DIRS}" ) set_target_properties(kind PROPERTIES @@ -169,32 +202,32 @@ set_target_properties(kind PROPERTIES ) set_target_properties(sdf PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};${Boost_INCLUDE_DIRS}" - INTERFACE_LINK_LIBRARIES "arch;tf;gf;trace;vt;work;ar" + INTERFACE_LINK_LIBRARIES "arch;tf;gf;trace;vt;work;ar;$<$:Boost::python>" INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${Boost_INCLUDE_DIRS}" ) set_target_properties(ndr PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};${Boost_INCLUDE_DIRS}" - INTERFACE_LINK_LIBRARIES "tf;plug;vt;work;ar;sdf" + INTERFACE_LINK_LIBRARIES "tf;plug;vt;work;ar;sdf;$<$:Boost::python>" INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${Boost_INCLUDE_DIRS}" ) set_target_properties(sdr PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};${Boost_INCLUDE_DIRS}" - INTERFACE_LINK_LIBRARIES "tf;vt;ar;ndr;sdf" + INTERFACE_LINK_LIBRARIES "tf;vt;ar;ndr;sdf;$<$:Boost::python>" INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${Boost_INCLUDE_DIRS}" ) set_target_properties(pcp PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" - INTERFACE_LINK_LIBRARIES "tf;trace;vt;sdf;work;ar;TBB::tbb" + INTERFACE_LINK_LIBRARIES "tf;trace;vt;sdf;work;ar;$<$:Boost::python>;TBB::tbb" INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" ) set_target_properties(usd PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" - INTERFACE_LINK_LIBRARIES "arch;kind;pcp;sdf;ar;plug;tf;trace;vt;work;TBB::tbb" + INTERFACE_LINK_LIBRARIES "arch;kind;pcp;sdf;ar;plug;tf;trace;vt;work;$<$:Boost::python>;TBB::tbb" INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" ) set_target_properties(usdGeom PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" - INTERFACE_LINK_LIBRARIES "js;tf;plug;vt;sdf;trace;usd;work;TBB::tbb" + INTERFACE_LINK_LIBRARIES "js;tf;plug;vt;sdf;trace;usd;work;$<$:Boost::python>;TBB::tbb" INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" ) set_target_properties(usdVol PROPERTIES @@ -227,12 +260,12 @@ set_target_properties(usdHydra PROPERTIES ) set_target_properties(usdRi PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};${Boost_INCLUDE_DIRS}" - INTERFACE_LINK_LIBRARIES "tf;vt;sdf;usd;usdShade;usdGeom" + INTERFACE_LINK_LIBRARIES "tf;vt;sdf;usd;usdShade;usdGeom;$<$:Boost::python>" INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${Boost_INCLUDE_DIRS}" ) set_target_properties(usdSkel PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" - INTERFACE_LINK_LIBRARIES "arch;gf;tf;trace;vt;work;sdf;usd;usdGeom;TBB::tbb" + INTERFACE_LINK_LIBRARIES "arch;gf;tf;trace;vt;work;sdf;usd;usdGeom;$<$:Boost::python>;TBB::tbb" INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" ) set_target_properties(usdUI PROPERTIES @@ -241,12 +274,12 @@ set_target_properties(usdUI PROPERTIES ) set_target_properties(usdUtils PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};${Boost_INCLUDE_DIRS}" - INTERFACE_LINK_LIBRARIES "arch;tf;gf;sdf;usd;usdGeom" + INTERFACE_LINK_LIBRARIES "arch;tf;gf;sdf;usd;usdGeom;$<$:Boost::python>" INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${Boost_INCLUDE_DIRS}" ) set_target_properties(usdPhysics PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" - INTERFACE_LINK_LIBRARIES "tf;plug;vt;sdf;trace;usd;usdGeom;usdShade;work;TBB::tbb" + INTERFACE_LINK_LIBRARIES "tf;plug;vt;sdf;trace;usd;usdGeom;usdShade;work;$<$:Boost::python>;TBB::tbb" INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${Boost_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" ) set_target_properties(garch PROPERTIES @@ -269,12 +302,12 @@ set_target_properties(cameraUtil PROPERTIES ) set_target_properties(pxOsd PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};${OPENSUBDIV_INCLUDE_DIRS}" - INTERFACE_LINK_LIBRARIES "tf;gf;vt;OpenSubdiv::OpenSubdiv" + INTERFACE_LINK_LIBRARIES "tf;gf;vt;OpenSubdiv::OpenSubdiv;$<$:Boost::python>" INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${OPENSUBDIV_INCLUDE_DIRS}" ) set_target_properties(glf PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};${Boost_INCLUDE_DIRS}" - INTERFACE_LINK_LIBRARIES "ar;arch;garch;gf;hf;hio;plug;tf;trace;sdf;OpenGL::GL" + INTERFACE_LINK_LIBRARIES "ar;arch;garch;gf;hf;hio;plug;tf;trace;sdf;$<$:Boost::python>;OpenGL::GL" INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${Boost_INCLUDE_DIRS}" ) set_target_properties(hgi PROPERTIES @@ -314,9 +347,9 @@ set_target_properties(usdImaging PROPERTIES INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${TBB_INCLUDE_DIRS}" ) set_target_properties(usdImagingGL PROPERTIES - INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};${TBB_INCLUDE_DIRS}" - INTERFACE_LINK_LIBRARIES "gf;tf;plug;trace;vt;work;hio;garch;glf;hd;hdx;pxOsd;sdf;sdr;usd;usdGeom;usdHydra;usdShade;usdImaging;ar;TBB::tbb" - INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${TBB_INCLUDE_DIRS}" + INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};$<$:${PYTHON_INCLUDE_DIRS}>;${TBB_INCLUDE_DIRS}" + INTERFACE_LINK_LIBRARIES "gf;tf;plug;trace;vt;work;hio;garch;glf;hd;hdx;pxOsd;sdf;sdr;usd;usdGeom;usdHydra;usdShade;usdImaging;ar;$<$:Boost::python>;TBB::tbb" + INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "$<$:${PYTHON_INCLUDE_DIRS}>;${TBB_INCLUDE_DIRS}" ) set_target_properties(usdProcImaging PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS}" @@ -337,10 +370,18 @@ set_target_properties(usdVolImaging PROPERTIES ) set_target_properties(usdAppUtils PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};${Boost_INCLUDE_DIRS}" - INTERFACE_LINK_LIBRARIES "garch;gf;hio;sdf;tf;usd;usdGeom;usdImagingGL" + INTERFACE_LINK_LIBRARIES "garch;gf;hio;sdf;tf;usd;usdGeom;usdImagingGL;$<$:Boost::python>" INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${Boost_INCLUDE_DIRS}" ) +if(USD_COMPOMPONENTS_USDVIEW) + set_target_properties(usdviewq PROPERTIES + INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS};${Boost_INCLUDE_DIRS}" + INTERFACE_LINK_LIBRARIES "tf;usd;usdGeom;$<$:Boost::python>" + INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${Boost_INCLUDE_DIRS}" + ) +endif() + if(USD_hgiVulkan_LIBRARY_RELEASE OR USD_hgiVulkan_LIBRARY_DEBUG) set_target_properties(hgiVulkan PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${PXR_INCLUDE_DIRS}" @@ -379,4 +420,4 @@ mark_as_advanced( PXR_LIBRARY_DIR USD_LIBRARIES_RELEASE USD_LIBRARIES_DEBUG -) +) \ No newline at end of file diff --git a/Scripts/installExternals.py b/Scripts/installExternals.py index e65c5f2..0e7ab84 100644 --- a/Scripts/installExternals.py +++ b/Scripts/installExternals.py @@ -52,6 +52,9 @@ # Helpers for printing output verbosity = 1 +# Script version, incrementing this will force re-installation of all packages. +scriptVersion = "00001" + def Print(msg): if verbosity > 0: print(msg) @@ -129,6 +132,100 @@ def IsVisualStudio2019OrGreater(): VISUAL_STUDIO_2019_VERSION = (16, 0) return IsVisualStudioVersionOrGreater(VISUAL_STUDIO_2019_VERSION) +def GetPythonInfo(context): + """Returns a tuple containing the path to the Python executable, shared + library, and include directory corresponding to the version of Python + currently running. Returns None if any path could not be determined. + + This function is used to extract build information from the Python + interpreter used to launch this script. This information is used + in the Boost and USD builds. By taking this approach we can support + having USD builds for different Python versions built on the same + machine. This is very useful, especially when developers have multiple + versions installed on their machine, which is quite common now with + Python2 and Python3 co-existing. + """ + + # If we were given build python info then just use it. + if context.build_python_info: + return (context.build_python_info['PYTHON_EXECUTABLE'], + context.build_python_info['PYTHON_LIBRARY'], + context.build_python_info['PYTHON_INCLUDE_DIR'], + context.build_python_info['PYTHON_VERSION']) + + # First we extract the information that can be uniformly dealt with across + # the platforms: + pythonExecPath = sys.executable + pythonVersion = sysconfig.get_config_var("py_version_short") # "2.7" + + # Lib path is unfortunately special for each platform and there is no + # config_var for it. But we can deduce it for each platform, and this + # logic works for any Python version. + def _GetPythonLibraryFilename(context): + if Windows(): + return "python{version}.lib".format( + version=sysconfig.get_config_var("py_version_nodot")) + elif Linux(): + return sysconfig.get_config_var("LDLIBRARY") + else: + raise RuntimeError("Platform not supported") + + pythonIncludeDir = sysconfig.get_path("include") + if not pythonIncludeDir or not os.path.isdir(pythonIncludeDir): + # as a backup, and for legacy reasons - not preferred because + # it may be baked at build time + pythonIncludeDir = sysconfig.get_config_var("INCLUDEPY") + + # if in a venv, installed_base will be the "original" python, + # which is where the libs are ("base" will be the venv dir) + pythonBaseDir = sysconfig.get_config_var("installed_base") + if not pythonBaseDir or not os.path.isdir(pythonBaseDir): + # for python-2.7 + pythonBaseDir = sysconfig.get_config_var("base") + + if Windows(): + pythonLibPath = os.path.join(pythonBaseDir, "libs", + _GetPythonLibraryFilename(context)) + elif Linux(): + pythonMultiarchSubdir = sysconfig.get_config_var("multiarchsubdir") + # Try multiple ways to get the python lib dir + for pythonLibDir in (sysconfig.get_config_var("LIBDIR"), + os.path.join(pythonBaseDir, "lib")): + if pythonMultiarchSubdir: + pythonLibPath = \ + os.path.join(pythonLibDir + pythonMultiarchSubdir, + _GetPythonLibraryFilename(context)) + if os.path.isfile(pythonLibPath): + break + pythonLibPath = os.path.join(pythonLibDir, + _GetPythonLibraryFilename(context)) + if os.path.isfile(pythonLibPath): + break + else: + raise RuntimeError("Platform not supported") + + return (pythonExecPath, pythonLibPath, pythonIncludeDir, pythonVersion) + +# Check the installed version of a package, using generated .version.txt file. +def CheckVersion(context, packageName, versionString): + fullVersionString = scriptVersion+":"+versionString + versionTextFilename = os.path.join(context.externalsInstDir, packageName+".version.txt") + if(not os.path.exists(versionTextFilename)): + return False + + versionTxt = pathlib.Path(versionTextFilename).read_text() + return versionTxt==fullVersionString + +# Update generated .version.txt file for a package. +def UpdateVersion(context, packageName, versionString): + if(CheckVersion(context, packageName, versionString)): + return + fullVersionString = scriptVersion+":"+versionString + versionTextFilename = os.path.join(context.externalsInstDir, packageName+".version.txt") + versionFile= open(versionTextFilename, "wt") + versionFile.write(fullVersionString) + versionFile.close() + def GetCPUCount(): try: return multiprocessing.cpu_count() @@ -504,6 +601,22 @@ def GitClone(url, tag, cloneDir, context): raise RuntimeError("Failed to clone repo {url} ({tag}): {err}".format( url=url, tag=tag, err=e)) +def GitCloneSHA(url, sha, cloneDir, context): + try: + with CurrentWorkingDirectory(context.externalsSrcDir): + # TODO check if cloneDir is a cloned folder of url + if not os.path.exists(cloneDir): + Run("git clone --recurse-submodules {url} {folder}".format(url=url, folder=cloneDir)) + with CurrentWorkingDirectory(os.path.join(context.externalsSrcDir, cloneDir)): + Run("git checkout {sha}".format(sha=sha)) + elif not IsGitFolder(cloneDir): + raise RuntimeError("Failed to clone repo {url} ({tag}): non-git folder {folder} exists".format( + url=url, tag=tag, folder=cloneDir)) + return os.path.abspath(cloneDir) + except Exception as e: + raise RuntimeError("Failed to clone repo {url} ({tag}): {err}".format( + url=url, tag=tag, err=e)) + def WriteExternalsConfig(context, externals): win32Header = """ # Build configurations: {buildConfiguration} @@ -549,11 +662,12 @@ def WriteExternalsConfig(context, externals): AllDependenciesByName = dict() class Dependency(object): - def __init__(self, name, packageName, installer, *files): + def __init__(self, name, packageName, installer, versionString, *files): self.name = name self.packageName = packageName # cmake package name self.installer = installer self.installFolder = name + self.versionString = versionString self.filesToCheck = files AllDependencies.append(self) @@ -563,10 +677,18 @@ def Exists(self, context): return all([os.path.isfile(os.path.join(context.externalsInstDir, self.installFolder, f)) for f in self.filesToCheck]) + def UpdateVersion(self, context): + UpdateVersion(context, self.packageName, self.versionString) + + def IsUpToDate(self, context): + if(not self.Exists(context)): + return False + return CheckVersion(context, self.packageName, self.versionString) + ############################################################ # zlib -ZLIB_URL = "https://github.com/madler/zlib/archive/v1.2.11.zip" +ZLIB_URL = "https://github.com/madler/zlib/archive/v1.2.13.zip" ZLIB_INSTALL_FOLDER = "zlib" ZLIB_PACKAGE_NAME = "ZLIB" @@ -574,12 +696,12 @@ def InstallZlib(context, force, buildArgs): with CurrentWorkingDirectory(DownloadURL(ZLIB_URL, context, force)): RunCMake(context, force, ZLIB_INSTALL_FOLDER, buildArgs) -ZLIB = Dependency(ZLIB_INSTALL_FOLDER, ZLIB_PACKAGE_NAME, InstallZlib, "include/zlib.h") +ZLIB = Dependency(ZLIB_INSTALL_FOLDER, ZLIB_PACKAGE_NAME, InstallZlib, ZLIB_URL, "include/zlib.h") ############################################################ # boost -BOOST_URL = "https://boostorg.jfrog.io/artifactory/main/release/1.70.0/source/boost_1_70_0.tar.gz" +BOOST_URL = "https://boostorg.jfrog.io/artifactory/main/release/1.78.0/source/boost_1_78_0.tar.gz" if Linux(): BOOST_VERSION_FILE = "include/boost/version.hpp" @@ -588,10 +710,7 @@ def InstallZlib(context, force, buildArgs): # subdirectory, which we have to account for here. In theory, specifying # "layout=system" would make the Windows install match Linux, but that # causes problems for other dependencies that look for boost. - # - # boost 1.70 is required for Visual Studio 2019. For simplicity, we use - # this version for all older Visual Studio versions as well. - BOOST_VERSION_FILE = "include/boost-1_70/boost/version.hpp" + BOOST_VERSION_FILE = "include/boost-1_78/boost/version.hpp" BOOST_INSTALL_FOLDER = "boost" BOOST_PACKAGE_NAME = "Boost" @@ -629,6 +748,27 @@ def InstallBoost_Helper(context, force, buildArgs): '--with-regex' ] + # Required by USD built with python support + b2Settings.append("--with-python") + pythonInfo = GetPythonInfo(context) + # This is the only platform-independent way to configure these + # settings correctly and robustly for the Boost jam build system. + # There are Python config arguments that can be passed to bootstrap + # but those are not available in boostrap.bat (Windows) so we must + # take the following approach: + projectPath = 'python-config.jam' + with open(projectPath, 'w') as projectFile: + # Note that we must escape any special characters, like + # backslashes for jam, hence the mods below for the path + # arguments. Also, if the path contains spaces jam will not + # handle them well. Surround the path parameters in quotes. + projectFile.write('using python : %s\n' % pythonInfo[3]) + projectFile.write(' : "%s"\n' % pythonInfo[0].replace("\\","/")) + projectFile.write(' : "%s"\n' % pythonInfo[2].replace("\\","/")) + projectFile.write(' : "%s"\n' % os.path.dirname(pythonInfo[1]).replace("\\","/")) + projectFile.write(' ;\n') + b2Settings.append("--user-config=python-config.jam") + # Required by OpenImageIO b2Settings.append("--with-date_time") b2Settings.append("--with-chrono") @@ -686,7 +826,7 @@ def InstallBoost(context, force, buildArgs): except: pass raise -BOOST = Dependency(BOOST_INSTALL_FOLDER, BOOST_PACKAGE_NAME, InstallBoost, BOOST_VERSION_FILE) +BOOST = Dependency(BOOST_INSTALL_FOLDER, BOOST_PACKAGE_NAME, InstallBoost, BOOST_URL, BOOST_VERSION_FILE) ############################################################ # Intel TBB @@ -738,7 +878,7 @@ def InstallTBB_Linux(context, force, buildArgs): CopyDirectory(context, "include/serial", "include/serial", TBB_INSTALL_FOLDER) CopyDirectory(context, "include/tbb", "include/tbb", TBB_INSTALL_FOLDER) -TBB = Dependency(TBB_INSTALL_FOLDER, TBB_PACKAGE_NAME, InstallTBB, "include/tbb/tbb.h") +TBB = Dependency(TBB_INSTALL_FOLDER, TBB_PACKAGE_NAME, InstallTBB, TBB_URL, "include/tbb/tbb.h") ############################################################ # JPEG @@ -751,7 +891,7 @@ def InstallJPEG(context, force, buildArgs): with CurrentWorkingDirectory(DownloadURL(JPEG_URL, context, force)): RunCMake(context, force, JPEG_INSTALL_FOLDER, buildArgs) -JPEG = Dependency(JPEG_INSTALL_FOLDER, JPEG_PACKAGE_NAME, InstallJPEG, "include/jpeglib.h") +JPEG = Dependency(JPEG_INSTALL_FOLDER, JPEG_PACKAGE_NAME, InstallJPEG, JPEG_URL, "include/jpeglib.h") ############################################################ # TIFF @@ -785,7 +925,7 @@ def InstallTIFF(context, force, buildArgs): extraArgs += buildArgs RunCMake(context, force, TIFF_INSTALL_FOLDER, extraArgs) -TIFF = Dependency(TIFF_INSTALL_FOLDER, TIFF_PACKAGE_NAME, InstallTIFF, "include/tiff.h") +TIFF = Dependency(TIFF_INSTALL_FOLDER, TIFF_PACKAGE_NAME, InstallTIFF, TIFF_URL, "include/tiff.h") ############################################################ # PNG @@ -798,7 +938,7 @@ def InstallPNG(context, force, buildArgs): with CurrentWorkingDirectory(DownloadURL(PNG_URL, context, force)): RunCMake(context, force, PNG_INSTALL_FOLDER, buildArgs) -PNG = Dependency(PNG_INSTALL_FOLDER, PNG_PACKAGE_NAME, InstallPNG, "include/png.h") +PNG = Dependency(PNG_INSTALL_FOLDER, PNG_PACKAGE_NAME, InstallPNG, PNG_URL, "include/png.h") ############################################################ # GLM @@ -812,20 +952,22 @@ def InstallGLM(context, force, buildArgs): CopyDirectory(context, "glm", "glm", GLM_INSTALL_FOLDER) CopyDirectory(context, "cmake/glm", "cmake/glm", GLM_INSTALL_FOLDER) -GLM = Dependency(GLM_INSTALL_FOLDER, GLM_PACKAGE_NAME, InstallGLM, "glm/glm.hpp") +GLM = Dependency(GLM_INSTALL_FOLDER, GLM_PACKAGE_NAME, InstallGLM, GLM_URL, "glm/glm.hpp") ############################################################ # STB -STB_URL = "https://github.com/nothings/stb/archive/refs/heads/master.zip" +STB_URL = "https://github.com/nothings/stb.git" +STB_SHA = "5736b15f7ea0ffb08dd38af21067c314d6a3aae9" # master on 2023-01-29 STB_INSTALL_FOLDER = "stb" STB_PACKAGE_NAME = "stb" def InstallSTB(context, force, buildArgs): - with CurrentWorkingDirectory(DownloadURL(STB_URL, context, force)): + STB_FOLDER="stb-"+STB_SHA + with CurrentWorkingDirectory(GitCloneSHA(STB_URL, STB_SHA, STB_FOLDER, context)): CopyFiles(context, "*.h", "include", STB_INSTALL_FOLDER) -STB = Dependency(STB_INSTALL_FOLDER, STB_PACKAGE_NAME, InstallSTB, "include/stb_image.h") +STB = Dependency(STB_INSTALL_FOLDER, STB_PACKAGE_NAME, InstallSTB, STB_SHA, "include/stb_image.h") ############################################################ # TinyGLTF @@ -838,7 +980,7 @@ def InstallTinyGLTF(context, force, buildArgs): with CurrentWorkingDirectory(DownloadURL(TinyGLTF_URL, context, force)): RunCMake(context, force, TinyGLTF_INSTALL_FOLDER, buildArgs) -TINYGLTF = Dependency(TinyGLTF_INSTALL_FOLDER, TinyGLTF_PACKAGE_NAME, InstallTinyGLTF, "include/tiny_gltf.h") +TINYGLTF = Dependency(TinyGLTF_INSTALL_FOLDER, TinyGLTF_PACKAGE_NAME, InstallTinyGLTF, TinyGLTF_URL, "include/tiny_gltf.h") ############################################################ # TinyObjLoader @@ -851,7 +993,7 @@ def InstallTinyObjLoader(context, force, buildArgs): with CurrentWorkingDirectory(DownloadURL(TinyObjLoader_URL, context, force)): RunCMake(context, force, TinyObjLoader_INSTALL_FOLDER, buildArgs) -TINYOBJLOADER = Dependency(TinyObjLoader_INSTALL_FOLDER, TinyObjLoader_PACKAGE_NAME, InstallTinyObjLoader, "include/tiny_obj_loader.h") +TINYOBJLOADER = Dependency(TinyObjLoader_INSTALL_FOLDER, TinyObjLoader_PACKAGE_NAME, InstallTinyObjLoader, TinyObjLoader_URL, "include/tiny_obj_loader.h") ############################################################ # IlmBase/OpenEXR @@ -880,7 +1022,7 @@ def InstallOpenEXR(context, force, buildArgs): RunCMake(context, force, OPENEXR_INSTALL_FOLDER, extraArgs) -OPENEXR = Dependency(OPENEXR_INSTALL_FOLDER, OPENEXR_PACKAGE_NAME, InstallOpenEXR, "include/OpenEXR/ImfVersion.h") +OPENEXR = Dependency(OPENEXR_INSTALL_FOLDER, OPENEXR_PACKAGE_NAME, InstallOpenEXR, OPENEXR_URL, "include/OpenEXR/ImfVersion.h") ############################################################ # OpenImageIO @@ -925,7 +1067,7 @@ def InstallOpenImageIO(context, force, buildArgs): RunCMake(context, force, OIIO_INSTALL_FOLDER, extraArgs, configExtraArgs=tbbConfigs) -OPENIMAGEIO = Dependency(OIIO_INSTALL_FOLDER, OIIO_PACKAGE_NAME, InstallOpenImageIO, "include/OpenImageIO/oiioversion.h") +OPENIMAGEIO = Dependency(OIIO_INSTALL_FOLDER, OIIO_PACKAGE_NAME, InstallOpenImageIO, OIIO_URL, "include/OpenImageIO/oiioversion.h") ############################################################ # OpenSubdiv @@ -977,7 +1119,7 @@ def InstallOpenSubdiv(context, force, buildArgs): context.cmakeGenerator = oldGenerator context.numJobs = oldNumJobs -OPENSUBDIV = Dependency(OPENSUBDIV_INSTALL_FOLDER, OPENSUBDIV_PACKAGE_NAME, InstallOpenSubdiv, +OPENSUBDIV = Dependency(OPENSUBDIV_INSTALL_FOLDER, OPENSUBDIV_PACKAGE_NAME, InstallOpenSubdiv, OPENSUBDIV_URL, "include/opensubdiv/version.h") ############################################################ @@ -994,7 +1136,7 @@ def InstallMaterialX(context, force, buildArgs): RunCMake(context, force, MATERIALX_INSTALL_FOLDER, cmakeOptions) -MATERIALX = Dependency(MATERIALX_INSTALL_FOLDER, MATERIALX_PACKAGE_NAME, InstallMaterialX, "include/MaterialXCore/Library.h") +MATERIALX = Dependency(MATERIALX_INSTALL_FOLDER, MATERIALX_PACKAGE_NAME, InstallMaterialX, MATERIALX_URL, "include/MaterialXCore/Library.h") ############################################################ # USD @@ -1032,7 +1174,7 @@ def InstallUSD(context, force, buildArgs): extraArgs.append('-DPXR_ENABLE_VULKAN_SUPPORT=ON') - extraArgs.append('-DPXR_BUILD_USD_TOOLS=OFF') + extraArgs.append('-DPXR_BUILD_USD_TOOLS=ON') extraArgs.append('-DPXR_BUILD_IMAGING=ON') extraArgs.append('-DPXR_ENABLE_PTEX_SUPPORT=OFF') @@ -1044,18 +1186,30 @@ def InstallUSD(context, force, buildArgs): extraArgs.append('-DPXR_BUILD_USD_IMAGING=ON') - extraArgs.append('-DPXR_BUILD_USDVIEW=OFF') + extraArgs.append('-DPXR_BUILD_USDVIEW=ON') extraArgs.append('-DPXR_BUILD_ALEMBIC_PLUGIN=OFF') extraArgs.append('-DPXR_BUILD_DRACO_PLUGIN=OFF') extraArgs.append('-DPXR_ENABLE_MATERIALX_SUPPORT=OFF') - extraArgs.append('-DPXR_ENABLE_PYTHON_SUPPORT=OFF') - # Turn off the text system in USD (Autodesk extension) extraArgs.append('-DPXR_ENABLE_TEXT_SUPPORT=OFF') + extraArgs.append('-DPXR_ENABLE_PYTHON_SUPPORT=ON') + extraArgs.append('-DPXR_USE_PYTHON_3=ON') + pythonInfo = GetPythonInfo(context) + if pythonInfo: + # According to FindPythonLibs.cmake these are the variables + # to set to specify which Python installation to use. + extraArgs.append('-DPYTHON_EXECUTABLE="{pyExecPath}"' + .format(pyExecPath=pythonInfo[0])) + extraArgs.append('-DPYTHON_LIBRARY="{pyLibPath}"' + .format(pyLibPath=pythonInfo[1])) + extraArgs.append('-DPYTHON_INCLUDE_DIR="{pyIncPath}"' + .format(pyIncPath=pythonInfo[2])) + extraArgs.append('-DPXR_USE_DEBUG_PYTHON=OFF') + if Windows(): # Increase the precompiled header buffer limit. extraArgs.append('-DCMAKE_CXX_FLAGS="/Zm150"') @@ -1076,7 +1230,7 @@ def InstallUSD(context, force, buildArgs): } RunCMake(context, force, USD_INSTALL_FOLDER, extraArgs, configExtraArgs=tbbConfigs) -USD = Dependency(USD_INSTALL_FOLDER, USD_PACKAGE_NAME, InstallUSD, "include/pxr/pxr.h") +USD = Dependency(USD_INSTALL_FOLDER, USD_PACKAGE_NAME, InstallUSD, USD_URL, "include/pxr/pxr.h") ############################################################ # Slang @@ -1092,7 +1246,7 @@ def InstallSlang(context, force, buildArgs): Slang_SRC_FOLDER = DownloadURL(Slang_URL, context, force, destDir="Slang") CopyDirectory(context, Slang_SRC_FOLDER, Slang_INSTALL_FOLDER) -SLANG = Dependency(Slang_INSTALL_FOLDER, Slang_PACKAGE_NAME, InstallSlang, "slang.h") +SLANG = Dependency(Slang_INSTALL_FOLDER, Slang_PACKAGE_NAME, InstallSlang, Slang_URL, "slang.h") ############################################################ # NRD @@ -1123,7 +1277,7 @@ def InstallNRD(context, force, buildArgs): CopyFiles(context, "Shaders/Include/NRD.hlsli", "Shaders/Include", NRD_INSTALL_FOLDER) CopyFiles(context, "External/MathLib/*.hlsli", "Shaders/Source", NRD_INSTALL_FOLDER) -NRD = Dependency(NRD_INSTALL_FOLDER, NRD_PACKAGE_NAME, InstallNRD, "include/NRD.h") +NRD = Dependency(NRD_INSTALL_FOLDER, NRD_PACKAGE_NAME, InstallNRD, NRD_URL, "include/NRD.h") ############################################################ # NRI @@ -1149,7 +1303,7 @@ def InstallNRI(context, force, buildArgs): CopyFiles(context, "_Build/Debug/*.dll", "bin", NRI_INSTALL_FOLDER) CopyFiles(context, "_Build/Debug/*", "lib", NRI_INSTALL_FOLDER) -NRI = Dependency(NRI_INSTALL_FOLDER, NRI_PACKAGE_NAME, InstallNRI, "include/NRI.h") +NRI = Dependency(NRI_INSTALL_FOLDER, NRI_PACKAGE_NAME, InstallNRI, NRI_URL, "include/NRI.h") ############################################################ # GLEW @@ -1164,7 +1318,7 @@ def InstallGLEW(context, force, buildArgs): CopyFiles(context, "bin/Release/x64/*.dll", "bin", GLEW_INSTALL_FOLDER) CopyFiles(context, "lib/Release/x64/*.lib", "lib", GLEW_INSTALL_FOLDER) -GLEW = Dependency(GLEW_INSTALL_FOLDER, GLEW_PACKAGE_NAME, InstallGLEW, "include/GL/glew.h") +GLEW = Dependency(GLEW_INSTALL_FOLDER, GLEW_PACKAGE_NAME, InstallGLEW, GLEW_URL, "include/GL/glew.h") ############################################################ # GLFW @@ -1180,7 +1334,7 @@ def InstallGLFW(context, force, buildArgs): RunCMake(context, force, GLFW_INSTALL_FOLDER, cmakeOptions) -GLFW = Dependency(GLFW_INSTALL_FOLDER, GLFW_PACKAGE_NAME, InstallGLFW, "include/GLFW/glfw3.h") +GLFW = Dependency(GLFW_INSTALL_FOLDER, GLFW_PACKAGE_NAME, InstallGLFW, GLFW_URL, "include/GLFW/glfw3.h") ############################################################ # CXXOPTS @@ -1193,7 +1347,7 @@ def InstallCXXOPTS(context, force, buildArgs): with CurrentWorkingDirectory(DownloadURL(CXXOPTS_URL, context, force)): RunCMake(context, force, CXXOPTS_INSTALL_FOLDER, buildArgs) -CXXOPTS = Dependency(CXXOPTS_INSTALL_FOLDER, CXXOPTS_PACKAGE_NAME, InstallCXXOPTS, "include/cxxopts.hpp") +CXXOPTS = Dependency(CXXOPTS_INSTALL_FOLDER, CXXOPTS_PACKAGE_NAME, InstallCXXOPTS, CXXOPTS_URL, "include/cxxopts.hpp") ############################################################ # GTEST @@ -1207,7 +1361,7 @@ def InstallGTEST(context, force, buildArgs): extraArgs = [*buildArgs, '-Dgtest_force_shared_crt=ON'] RunCMake(context, force, GTEST_INSTALL_FOLDER, extraArgs) -GTEST = Dependency(GTEST_INSTALL_FOLDER, GTEST_PACKAGE_NAME, InstallGTEST, "include/gtest/gtest.h") +GTEST = Dependency(GTEST_INSTALL_FOLDER, GTEST_PACKAGE_NAME, InstallGTEST, GTEST_URL, "include/gtest/gtest.h") ############################################################ # Installation script @@ -1293,6 +1447,9 @@ def InstallGTEST(context, force, buildArgs): group.add_argument("--build-args", type=str, nargs="*", default=[], help=("Custom arguments to pass to build system when " "building libraries (see docs above)")) +group.add_argument("--build-python-info", type=str, nargs=4, default=[], + metavar=('PYTHON_EXECUTABLE', 'PYTHON_INCLUDE_DIR', 'PYTHON_LIBRARY', 'PYTHON_VERSION'), + help=("Specify a custom python to use during build")) group.add_argument("--force", type=str, action="append", dest="force_build", default=[], help=("Force download and build of specified library " @@ -1356,6 +1513,14 @@ def __init__(self, args): self.buildConfigs = [] + # Build python info + self.build_python_info = dict() + if args.build_python_info: + self.build_python_info['PYTHON_EXECUTABLE'] = args.build_python_info[0] + self.build_python_info['PYTHON_INCLUDE_DIR'] = args.build_python_info[1] + self.build_python_info['PYTHON_LIBRARY'] = args.build_python_info[2] + self.build_python_info['PYTHON_VERSION'] = args.build_python_info[3] + # Build type self.buildDebug = (args.build_variant == BUILD_DEBUG) or (args.build_variant == BUILD_DEBUG_AND_RELEASE) self.buildRelease = (args.build_variant == BUILD_RELEASE) or (args.build_variant == BUILD_DEBUG_AND_RELEASE) @@ -1440,7 +1605,7 @@ def ForceBuildDependency(self, dep): dependenciesToBuild = [] for dep in requiredDependencies: - if context.ForceBuildDependency(dep) or not dep.Exists(context): + if context.ForceBuildDependency(dep) or not dep.IsUpToDate(context): if dep not in dependenciesToBuild: dependenciesToBuild.append(dep) @@ -1556,6 +1721,7 @@ def FormatBuildArguments(buildArgs): dep.installer(context, buildArgs=context.GetBuildArguments(dep), force=context.ForceBuildDependency(dep)) + dep.UpdateVersion(context) except Exception as e: PrintError(str(e)) sys.exit(1) diff --git a/Scripts/setUSDEnv.bat b/Scripts/setUSDEnv.bat new file mode 100644 index 0000000..638126a --- /dev/null +++ b/Scripts/setUSDEnv.bat @@ -0,0 +1,8 @@ +@echo off + +set AURORA_BUILD_TYPE=%1 +set AURORA_BUILD_DIR=%2 + +set PYTHONPATH=%cd%\%AURORA_BUILD_DIR%\bin\%AURORA_BUILD_TYPE%\python +set PATH=%PATH%;%cd%\%AURORA_BUILD_DIR%\bin\%AURORA_BUILD_TYPE% + diff --git a/Tests/Helpers/AuroraTestHelpers.cpp b/Tests/Helpers/AuroraTestHelpers.cpp index 4dbf15a..9d885e0 100644 --- a/Tests/Helpers/AuroraTestHelpers.cpp +++ b/Tests/Helpers/AuroraTestHelpers.cpp @@ -20,8 +20,14 @@ #include "TeapotModel.h" #include "TestHelpers.h" + +#pragma warning(push) +// Disabe type conversion warnings intruduced from stb master. +// refer to the commit in stb https://github.com/nothings/stb/commit/b15b04321dfd8a2307c49ad9c5bf3c0c6bcc04cc +#pragma warning(disable : 4244) #define STB_IMAGE_IMPLEMENTATION #include +#pragma warning(pop) using namespace Aurora::Foundation; using namespace Aurora;