Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Fix compilation issues when building USD using the C++20 standard #2605

Conversation

mattyjams
Copy link
Contributor

Hello! :)

This may be a ways out on the road map still since the VFX reference platform hasn't made it there yet, but I took an initial crack at addressing some C++20 compilation issues and wanted to offer this small handful of changes that worked for me and still maintains compatibility with the currently chosen standard (C++14).

With the changes here, I was able to build USD with the C++ standard set forward to C++20 using the following patch:

diff --git a/cmake/defaults/CXXDefaults.cmake b/cmake/defaults/CXXDefaults.cmake
index e970dfbe2..48c132ff7 100644
--- a/cmake/defaults/CXXDefaults.cmake
+++ b/cmake/defaults/CXXDefaults.cmake
@@ -25,8 +25,8 @@ include(CXXHelpers)
 include(Version)
 include(Options)

-# Require C++14
-set(CMAKE_CXX_STANDARD 14)
+# Require C++20
+set(CMAKE_CXX_STANDARD 20)
 set(CMAKE_CXX_STANDARD_REQUIRED ON)
 set(CMAKE_CXX_EXTENSIONS OFF)

I tested on the following configurations:

  • Windows 10, Visual Studio 2022 v17.6.5/msvc 19.36.32537, Python 3.9.13
  • macOS 13.5, Apple clang 14.0.3, Python 3.9.6

I unfortunately don't have a Linux environment I can test this with at the moment. Interestingly, I did not need to incorporate #2242 (which aims to address #2183), so that particular issue may be specific to using gcc.

Here are some details about the changes:


The first three commits replace the use of std::shared_ptr's unique() with use_count() == 1 instead. unique() was deprecated in C++17 and removed in C++20, and should be equivalent to use_count() == 1 which is available back to C++11. I broke them out into individual commits updating hd, hdSt, and pcp, but I'm also happy to create separate pull requests for each if that's preferable.

I did want to flag the note on cppreference.com though:

This function was deprecated in C++17 and removed in C++20 because use_count is only an approximation in multithreaded environment (see Notes in use_count).

I'm not sure whether that's of concern for the instances of unique() currently in place, but switching to use_count() at least shouldn't be any less safe.


The change in hio/stb addresses an issue where unnamed classes used in typedefs can no longer have a default member initializer. This is specific to the gamma field in stb__png which the hio version adds. Since the affected file is sourced externally, I also updated the .patch file used to produce hio's version from the original.


The last change in sdf is somewhat perplexing, but the Boost Python wrapping for SdfPredicateExpression's GetParseError() seems to fail to deduce properly when C++20 is enabled. I was able to get around this by adding an explicit static function to call the appropriate version, but I'd be curious to see whether anyone with stronger Boost and/or type deduction debugging-fu has a better idea. I tried bumping Boost to the latest version (1.82.0) as well, but that didn't seem to make a difference.

I'll post the wall-of-text compilation error that was generated prompting the change separately so as not to (further) muddy the description here.


Hope these are helpful! Thanks!

  • I have verified that all unit tests pass with the proposed changes
  • I have submitted a signed Contributor License Agreement

@jesschimein
Copy link
Collaborator

Filed as internal issue #USD-8601

@mattyjams mattyjams force-pushed the pr/fix_compilation_errors_with_cxx20 branch from 3ca32be to 7f7cae3 Compare March 8, 2024 17:26
@mattyjams
Copy link
Contributor Author

Freshening this with a rebase on the current dev branch, but I also added a commit for the Ts library where it looks like instances of std::shared_ptr::unique() crept in when that library was added.

@jesschimein
Copy link
Collaborator

/AzurePipelines run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@mattyjams mattyjams force-pushed the pr/fix_compilation_errors_with_cxx20 branch from 7f7cae3 to 92f80ca Compare May 20, 2024 20:59
@mattyjams
Copy link
Contributor Author

Freshening with a rebase on the current dev branch (35dbce1).

@mattyjams mattyjams force-pushed the pr/fix_compilation_errors_with_cxx20 branch from 92f80ca to dac6168 Compare June 17, 2024 13:44
@mattyjams
Copy link
Contributor Author

Rebased on the current dev branch (5332d62) and resolved conflicts in stb_image.patch from the merge of #2880.

@jesschimein
Copy link
Collaborator

/AzurePipelines run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@mattyjams mattyjams force-pushed the pr/fix_compilation_errors_with_cxx20 branch from dac6168 to f10237c Compare August 12, 2024 14:18
@mattyjams
Copy link
Contributor Author

Rebased on the current dev branch (30392d9) and resolved conflicts in pxr/base/ts/spline.cpp by reverting, since usages of std::shared_ptr::unique() were removed in c898413.

@jesschimein
Copy link
Collaborator

/AzurePipelines run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Copy link
Contributor

@sunyab sunyab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @mattyjams, I'm looking to merge these changes in shortly. I had just one real question about the change in wrapPredicateExpression.cpp, would love to get your thoughts. Thanks!

pxr/usd/sdf/wrapPredicateExpression.cpp Outdated Show resolved Hide resolved
pxr/usd/sdf/wrapPredicateExpression.cpp Outdated Show resolved Hide resolved
…ceRegistry

unique() was deprecated in C++17 and removed in C++20. Testing use_count() == 1
should be equivalent, though both may create thread safety concerns when used
in a multithreaded environment.

See the notes here for more detail:

https://en.cppreference.com/w/cpp/memory/shared_ptr/use_count
…sourceRegistry and HdSt_SamplerObjectRegistry

unique() was deprecated in C++17 and removed in C++20. Testing use_count() == 1
should be equivalent, though both may create thread safety concerns when used
in a multithreaded environment.

See the notes here for more detail:

https://en.cppreference.com/w/cpp/memory/shared_ptr/use_count
…Index_Graph

unique() was deprecated in C++17 and removed in C++20. Testing use_count() == 1
should be equivalent, though both may create thread safety concerns when used
in a multithreaded environment.

See the notes here for more detail:

https://en.cppreference.com/w/cpp/memory/shared_ptr/use_count
…te patch

The local patch to stb_image.h adds a "gamma" member to the previously unnamed
struct used in the typedef of stbi__png. It also includes a default initializer
to set the member to 0.

When using C++17 or later, some compilers will emit an error that an unnamed
class cannot be used within a typedef if it has a non-static data member with a
default member initializer. In particular, this is the case with msvc:

https://learn.microsoft.com/en-us/cpp/error-messages/compiler-warnings/c5208

In this case, the error can be avoided simply by giving the problematic struct
a name. stb_image.h in hio was updated to make this change, and the patch file
used to create it from the original source version was regenerated to ensure
that the same change is applied if/when stb is upgraded in the future.
@mattyjams mattyjams force-pushed the pr/fix_compilation_errors_with_cxx20 branch from f10237c to 7f1de71 Compare December 9, 2024 15:15
@mattyjams
Copy link
Contributor Author

Hey @mattyjams, I'm looking to merge these changes in shortly. I had just one real question about the change in wrapPredicateExpression.cpp, would love to get your thoughts. Thanks!

Hey @sunyab! Thanks for taking a look at this one. Totally agree that with boost::python soon to be removed, we can drop the changes in wrapPredicateExpression.cpp.

I just pushed a rebase of these changes on the current tip of the dev branch (20acbba). I also re-generated the stb_image.patch just to put an updated timestamp in there to keep an accurate record of when it was last touched.

One note though: When I tried to re-build to run the tests one more time, I'm actually hitting an unrelated compilation error in one of the pxr_boost::python unit tests:

/Users/matt.johnson/Developer/OpenUSD/pxr/external/boost/python/test/result.cpp:88:9: error: no matching function for call to 'result'
   88 |         result(std::plus<int>(),0)
      |         ^~~~~~
In file included from /Users/matt.johnson/Developer/OpenUSD/pxr/external/boost/python/test/result.cpp:10:
/Users/matt.johnson/Developer/OpenUSD/pxr/external/boost/python/detail/result.hpp:58:1: note: candidate template ignored: substitution failure [with X = std::plus<int>]: no type named 'result_type' in 'std::plus<int>'
   57 | type<typename X::result_type>*
      |                  ~~~~~~~~~~~
   58 | result(X const&, short = 0) { return 0; }
      | ^
/Users/matt.johnson/Developer/OpenUSD/pxr/external/boost/python/detail/result.hpp:37:10: note: candidate template ignored: could not match 'R (*)(A...)' against 'std::plus<int>'
   37 | type<R>* result(R (*)(A...), int = 0)
      |          ^
/Users/matt.johnson/Developer/OpenUSD/pxr/external/boost/python/detail/result.hpp:49:35: note: candidate template ignored: could not match 'R (T::*)(A...)' against 'std::plus<int>'
   49 | PXR_BOOST_PYTHON_APPLY_QUALIFIERS(PXR_BOOST_PYTHON_RESULT_MEMBER_FN)
      |                                   ^
/Users/matt.johnson/Developer/OpenUSD/pxr/external/boost/python/detail/result.hpp:49:35: note: candidate template ignored: could not match 'R (T::*)(A...) const' against 'std::plus<int>'
/Users/matt.johnson/Developer/OpenUSD/pxr/external/boost/python/detail/result.hpp:49:35: note: candidate template ignored: could not match 'R (T::*)(A...) volatile' against 'std::plus<int>'
/Users/matt.johnson/Developer/OpenUSD/pxr/external/boost/python/detail/result.hpp:49:35: note: candidate template ignored: could not match 'R (T::*)(A...) const volatile' against 'std::plus<int>'
/Users/matt.johnson/Developer/OpenUSD/pxr/external/boost/python/detail/result.hpp:54:10: note: candidate template ignored: could not match 'R (T::*)' against 'std::plus<int>'
   54 | type<R>* result(R (T::*), int = 0) { return 0; }
      |          ^

And then again on line 112 of result.cpp, so both places std::plus is used. It would seem that that probably has to do with the removal of result_type as of C++20 in std::plus?:
https://en.cppreference.com/w/cpp/utility/functional/plus

We seem to expect that to be available here:

type<typename X::result_type>*

Did you run into that in your testing? Maybe you've got this addressed internally already as well?

@sunyab
Copy link
Contributor

sunyab commented Dec 9, 2024

@mattyjams Oh interesting, no I haven't run into that in our test builds. What compiler version were you using for that one?

@mattyjams
Copy link
Contributor Author

@mattyjams Oh interesting, no I haven't run into that in our test builds. What compiler version were you using for that one?

Ah, hmm. This was on an M1 Mac, with MacOS 15.1.1, the latest Xcode (16.1) with clang 16.0.0, and Python 3.11.9.

Here's the preamble of the USD build log (built through build_usd.py):

2024-12-09 12:42
cmake -DCMAKE_INSTALL_PREFIX="/Users/matt.johnson/Developer/OpenUSD_test_inst" -DCMAKE_PREFIX_PATH="/Users/matt.johnson/Developer/OpenUSD_test_inst" -DCMAKE_BUILD_TYPE=Release -DCMAKE_MACOSX_RPATH=ON -G "Xcode"  -DPXR_PREFER_SAFETY_OVER_SPEED=ON -DPXR_ENABLE_PYTHON_SUPPORT=ON -DPXR_USE_DEBUG_PYTHON=OFF -DPXR_USE_BOOST_PYTHON=OFF -DPython3_EXECUTABLE="/Library/Frameworks/Python.framework/Versions/3.11/bin/python3" -DPython3_LIBRARY="/Library/Frameworks/Python.framework/Versions/3.11/lib/libpython3.11.dylib" -DPython3_INCLUDE_DIR="/Library/Frameworks/Python.framework/Versions/3.11/include/python3.11" -DBUILD_SHARED_LIBS=ON -DTBB_USE_DEBUG_BUILD=OFF -DPXR_BUILD_DOCUMENTATION=OFF -DPXR_BUILD_HTML_DOCUMENTATION=OFF -DPXR_BUILD_PYTHON_DOCUMENTATION=OFF -DPXR_BUILD_TESTS=ON -DPXR_BUILD_EXAMPLES=ON -DPXR_BUILD_TUTORIALS=ON -DPXR_BUILD_USD_TOOLS=ON -DPXR_BUILD_IMAGING=ON -DPXR_ENABLE_PTEX_SUPPORT=ON -DPXR_ENABLE_OPENVDB_SUPPORT=OFF -DPXR_BUILD_EMBREE_PLUGIN=ON -DPXR_BUILD_PRMAN_PLUGIN=OFF -DPXR_BUILD_OPENIMAGEIO_PLUGIN=ON -DPXR_BUILD_OPENCOLORIO_PLUGIN=OFF -DPXR_BUILD_USD_IMAGING=ON -DPXR_BUILD_USDVIEW=ON -DPXR_BUILD_ALEMBIC_PLUGIN=ON -DPXR_ENABLE_HDF5_SUPPORT=OFF -DPXR_BUILD_DRACO_PLUGIN=OFF -DPXR_ENABLE_MATERIALX_SUPPORT=ON -DPXR_BUILD_MAYAPY_TESTS=OFF -DPXR_BUILD_ANIMX_TESTS=OFF -DBoost_NO_BOOST_CMAKE=On -DBoost_NO_SYSTEM_PATHS=True -DCMAKE_XCODE_ATTRIBUTE_ONLY_ACTIVE_ARCH=YES -DCMAKE_OSX_ARCHITECTURES=arm64 "/Users/matt.johnson/Developer/OpenUSD"
-- The C compiler identification is AppleClang 16.0.0.16000026
-- The CXX compiler identification is AppleClang 16.0.0.16000026
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Found Python3: /Library/Frameworks/Python.framework/Versions/3.11/bin/python3 (found version "3.11.9") found components: Interpreter Development Development.Module Development.Embed
-- Found Jinja2
-- Found TBB: /Users/matt.johnson/Developer/OpenUSD_test_inst/include (found version "2020.3") found components: tbb
-- Using default system allocator because PXR_MALLOC_LIBRARY is unspecified
-- Found OpenImageIO: /Users/matt.johnson/Developer/OpenUSD_test_inst/lib/libOpenImageIO.dylib;/Users/matt.johnson/Developer/OpenUSD_test_inst/lib/libOpenImageIO_Util.dylib (found version "2.3.21")
CMake Deprecation Warning at cmake/defaults/Packages.cmake:208 (cmake_policy):
  The OLD behavior for policy CMP0072 will be removed from a future version
  of CMake.

  The cmake-policies(7) manual explains that the OLD behaviors of all
  policies are deprecated and that a policy should be set to OLD only under
  specific short-term circumstances.  Projects should be ported to the NEW
  behavior and not rely on setting a policy to OLD.
Call Stack (most recent call first):
  CMakeLists.txt:23 (include)


-- Found OpenGL: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.1.sdk/System/Library/Frameworks/OpenGL.framework
-- Found OpenSubdiv: /Users/matt.johnson/Developer/OpenUSD_test_inst/include (found suitable version "3.6.0", minimum required is "3")
-- Found PTex: /Users/matt.johnson/Developer/OpenUSD_test_inst/include (found version "4.1.4")
-- Found Embree: /Users/matt.johnson/Developer/OpenUSD_test_inst/include
-- Found PySide6: with /Library/Frameworks/Python.framework/Versions/3.11/bin/python3, will use /Library/Frameworks/Python.framework/Versions/3.11/bin/pyside6-uic for pyside-uic binary
-- Found PySide: True
-- Found PyOpenGL
-- Found PyOpenGL: True
-- Found Alembic: /Users/matt.johnson/Developer/OpenUSD_test_inst/lib/libAlembic.dylib
-- C++ namespace configured to (external) pxr, (internal) pxrInternal_v0_25_2
-- Skipping validation of gf generated code because PXR_VALIDATE_GENERATED_CODE=OFF
-- Skipping Draco-based usddiff tests because PXR_BUILD_DRACO_PLUGIN=OFF
-- Skipping hgi tests because they are currently unsupported on macOS
-- Skipping hgiGL tests because they are currently unsupported on macOS
-- Skipping hgiVulkan because PXR_BUILD_GPU_SUPPORT or PXR_ENABLE_VULKAN_SUPPORT is OFF
-- Skipping rest of hdSt tests because they are currently unsupported on macOS
-- Skipping hdx tests because they are currently unsupported on macOS
-- Skipping usdImagingGL tests because they are currently unsupported on macOS
-- Skipping usdrecord_script tests because they are currently unsupported on macOS
-- Skipping usdviewq tests because they are currently unsupported on macOS
-- Configuring done (7.5s)
-- Generating done (3.1s)

@jesschimein
Copy link
Collaborator

/AzurePipelines run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@sunyab
Copy link
Contributor

sunyab commented Dec 9, 2024

Ah got it, I'm still testing using Xcode 13. It does seem likely to be due to the removal of result_type like you mentioned -- IIRC we've run into similar issues before.

Oddly enough, it seems like that test is exercising functionality that isn't actually used in boost::python. The detail/result.hpp header it's covering isn't included anywhere else in the library. I might just delete that in a follow-up change.

@mattyjams
Copy link
Contributor Author

Oddly enough, it seems like that test is exercising functionality that isn't actually used in boost::python. The detail/result.hpp header it's covering isn't included anywhere else in the library. I might just delete that in a follow-up change.

That also sounds great to me. :)

Thanks again, @sunyab!

@pixar-oss pixar-oss merged commit a40dca2 into PixarAnimationStudios:dev Dec 12, 2024
5 checks passed
@mattyjams mattyjams deleted the pr/fix_compilation_errors_with_cxx20 branch December 12, 2024 01:56
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants