Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When I set SPM_PROTOBUF_PROVIDER to "package" in CMakeLists.txt, the compilation fails. #1029

Open
hhxdestiny opened this issue Jun 25, 2024 · 0 comments

Comments

@hhxdestiny
Copy link

hhxdestiny commented Jun 25, 2024

I need to use sentencepiece for tokenization, and I also need OpenVINO for NLP task inference. I am using vcpkg to manage both sentencepiece and OpenVINO. The protobuf for OpenVINO comes from vcpkg, while the protobuf for sentencepiece defaults to internal rather than package. When I link both OpenVINO and sentencepiece libraries, I encounter symbol redefinition issues. Therefore, I have to change the SPM_PROTOBUF_PROVIDER of sentencepiece to "package" to use protobuf from vcpkg. However, after making this change, the compilation fails. How can I resolve this issue?

cmake change

@@ -33,7 +33,7 @@ option(SPM_NO_THREADLOCAL "Disable thread_local operator" OFF)
 option(SPM_ENABLE_MSVC_MT_BUILD, "Use /MT flag in MSVC build" OFF)
 option(SPM_CROSS_SYSTEM_PROCESSOR, "Override system processor" "")

-set(SPM_PROTOBUF_PROVIDER "internal" CACHE STRING "Provider of protobuf library")
+set(SPM_PROTOBUF_PROVIDER "package" CACHE STRING "Provider of protobuf library")
 set_property(CACHE SPM_PROTOBUF_PROVIDER PROPERTY STRINGS "internal" "package")
 set(SPM_ABSL_PROVIDER "internal" CACHE STRING "Provider of absl library")
 set_property(CACHE SPM_ABSL_PROVIDER PROPERTY STRINGS "internal" "module" "package")

log

···

[build] C:\PROGRA1\MICROS1\2022\COMMUN1\VC\Tools\MSVC\14401.338\bin\Hostx86\x64\cl.exe /nologo /TP -D_USE_EXTERNAL_PROTOBUF -D_USE_INTERNAL_STRING_VIEW -IH:\cpp_faster\sentencepiece -IH:\cpp_faster\sentencepiece\out\build\win-x64-release -IH:\cpp_faster\sentencepiece\third_party\absl -IH:\vcpkg\vcpkg_win\installed\x64-windows-static\include -IH:\cpp_faster\sentencepiece\out\build\win-x64-release\src -IH:\cpp_faster\sentencepiece\src..\third_party /DWIN32 /D_WINDOWS /W3 /GR /EHsc /O2 /Ob2 /DNDEBUG -std
++17 -MD /wd4267 /wd4244 /wd4305 /Zc
/utf-8 /showIncludes /Fosrc\CMakeFiles\sentencepiece-static.dir\error.cc.obj /Fdsrc\CMakeFiles\sentencepiece-static.dir\sentencepiece-static.pdb /FS -c H:\cpp_faster\sentencepiece\src\error.cc
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/casts.h(93): error C2872: “internal”: ambiguous symbol
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: could be “absl::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/internal/identity.h(23): note: or “absl::lts_20240116::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/casts.h(93): error C2039: "type_identity_t": is not a member of "absl::internal"
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: see declaration of "absl::internal"
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/casts.h(93): error C2988: unrecognizable template declaration/definition
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/casts.h(93): error C2143: syntax error: missing “,” before “<”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/casts.h(93): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/utility/utility.h(94): error C2872: “internal”: ambiguous symbol
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: could be “absl::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/internal/identity.h(23): note: or “absl::lts_20240116::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/utility/utility.h(94): error C2039: "type_identity_t": is not a member of "absl::internal"
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: see declaration of "absl::internal"
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/utility/utility.h(94): error C2143: syntax error: missing “;” before “<”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/utility/utility.h(94): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/utility/utility.h(94): error C2737: “absl::lts_20240116::type_identity_t”: constexpr object must be initialized
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/utility/utility.h(94): error C2143: syntax error: missing “;” before “{”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/utility/utility.h(94): error C2447: “{”: missing function header (old-style formal list?)
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(744): error C2872: “internal”: ambiguous symbol
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: could be “absl::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/internal/identity.h(23): note: or “absl::lts_20240116::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(744): error C2039: "type_identity": is not a member of "absl::internal"
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: see declaration of "absl::internal"
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(744): error C2988: unrecognizable template declaration/definition
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(744): error C2143: syntax error: missing “,” before “<”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(744): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(756): error C2872: “internal”: ambiguous symbol
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: could be “absl::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/internal/identity.h(23): note: or “absl::lts_20240116::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(756): error C2039: "type_identity": is not a member of "absl::internal"
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: see declaration of "absl::internal"
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(756): error C2988: unrecognizable template declaration/definition
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(756): error C2143: syntax error: missing “)” before “<”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(756): error C2143: syntax error: missing “,” before “<”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(756): error C2059: syntax error: “)”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(755): error C2091: function returns function
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/synchronization/mutex.h(761): error C2872: “internal”: ambiguous symbol
[build] H:\cpp_faster\sentencepiece\third_party/absl/flags/flag.h(24): note: could be “absl::internal”
[build] H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/base/internal/identity.h(23): note: or “absl::lts_20240116::internal”

···

C:\PROGRA~1\MICROS~1\2022\COMMUN~1\VC\Tools\MSVC\1440~1.338\bin\Hostx64\x64\cl.exe   /TP -D_USE_EXTERNAL_PROTOBUF -D_USE_INTERNAL_STRING_VIEW -IH:\vcpkg\vcpkg_win\buildtrees\sentencepiece\src\v0.1.99-67e41e1247.clean -IH:\vcpkg\vcpkg_win\buildtrees\sentencepiece\x64-windows-static-dbg -IH:\vcpkg\vcpkg_win\installed\x64-windows-static\include -IH:\vcpkg\vcpkg_win\buildtrees\sentencepiece\x64-windows-static-dbg\src -IH:\vcpkg\vcpkg_win\buildtrees\sentencepiece\src\v0.1.99-67e41e1247.clean\src\..\third_party /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /GR /EHsc /MP  /D_DEBUG /MTd /Z7 /Ob0 /Od /RTC1  -std:c++17 -MTd   /wd4267 /wd4244 /wd4305 /Zc:strictStrings /utf-8 /showIncludes /Fosrc\CMakeFiles\sentencepiece_train-static.dir\unigram_model_trainer.cc.obj /Fdsrc\CMakeFiles\sentencepiece_train-static.dir\sentencepiece_train-static.pdb /FS -c H:\vcpkg\vcpkg_win\buildtrees\sentencepiece\src\v0.1.99-67e41e1247.clean\src\unigram_model_trainer.cc
H:\vcpkg\vcpkg_win\buildtrees\sentencepiece\src\v0.1.99-67e41e1247.clean\src\sentencepiece_processor.h(58): error C2872: 'string_view': ambiguous symbol
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\include\xstring(1788): note: could be 'std::basic_string_view<char,std::char_traits<char>> std::string_view'
H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/strings/string_view.h(162): note: or       'absl::lts_20240116::string_view'
H:\vcpkg\vcpkg_win\buildtrees\sentencepiece\src\v0.1.99-67e41e1247.clean\src\sentencepiece_processor.h(244): error C2872: 'string_view': ambiguous symbol
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\include\xstring(1788): note: could be 'std::basic_string_view<char,std::char_traits<char>> std::string_view'
H:\vcpkg\vcpkg_win\installed\x64-windows-static\include\absl/strings/string_view.h(162): note: or       'absl::lts_20240116::string_view'
H:\vcpkg\vcpkg_win\buildtrees\sentencepiece\src\v0.1.99-67e41e1247.clean\src\sentencepiece_processor.h(248): error C2872: 'string_view': ambiguous symbol

···
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant