Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tips for Termux installation #950

Open
Manamama opened this issue Dec 14, 2023 · 7 comments
Open

Tips for Termux installation #950

Manamama opened this issue Dec 14, 2023 · 7 comments

Comments

@Manamama
Copy link

Manamama commented Dec 14, 2023

This had to be added, when compiling it in Termux:

.../sentencepiece/CMakeLists.txt

set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -llog"): This line is setting a CMake variable called CMAKE_SHARED_LINKER_FLAGS, which specifies the flags to be used by the linker when creating shared libraries⁵. The -llog flag tells the linker to link against liblog.so, which provides the __android_log_write function that was causing the undefined reference error in your initial build attempt⁶.

Compile: cmake .. -DCMAKE_INSTALL_PREFIX=$PREFIX: Here, you're invoking CMake and passing it a command-line argument to set the CMAKE_INSTALL_PREFIX variable. This variable determines the directory where the project will be installed[^10^]¹¹. By setting it to $PREFIX, you're telling CMake to install the project in the directory specified by the PREFIX environment variable, which in Termux is typically /data/data/com.termux/files/usr.

After applying these fixes and successfully building the project:

cd ..
cd python/
python setup.py bdist_wheel
pip install dist/sentencepiece*.whl
pip show sentencepiece
@ghost
Copy link

ghost commented Dec 18, 2023

😍😭❤️ you're a genius I was about to give up , please please please can you explain this in more detail I am using tarfile to install this and I cannot find the python/ directory 😢 and I am a little bit confused about where I've to make that edit is that line number specific in that file ?

@Manamama
Copy link
Author

Manamama commented Dec 18, 2023

@FindingMeaning :
I am not a genius, by far. I run most such snags by Ms Bing (mostly in its Precise mode) or Claude AI, with my neofetch results and just ask it politely, yet insistently, and interactively.

Re your question - it requires

git clone https://github.com/google/sentencepiece
cd sentencepiece

first, obviously, then changing these tidbits ( in .../sentencepiece/CMakeLists.txt, as I wrote) that you will see, and maybe paying attention to the missing libraries that
python setup ...
or
pip install ...
logs may throw at you, and installing these missing bits (modules etc.), if any.

In short - just paste these errors to a live session with any LLM/AI and explain to it politely what you are after, assuming it is also a blind man seeing (your) elephant, that is the system errors.

@Manamama
Copy link
Author

Manamama commented Dec 18, 2023

Another tip for Termux lovers (like me)

~ $ pkg list-all | grep gperf

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

gperf/stable,now 3.1-7 aarch64 [installed,automatic]
~ $ 

or

cmake-curses-gui/stable 3.28.1 aarch64
cmake/stable,now 3.28.1 aarch64 [installed]
extra-cmake-modules/x11 5.112.0 aarch64
...
libgoogleperf-tools-dev
~ $ 

(e.g. for MALLOC error) is what is needed to compile, the termux equivalents thereof

Anyhow, use that syntax to find the Termux precompiled equivalents of most anything: numpy (needed for numba), scikit, torch, etc.

Another trick that (may, a big may) work out of the box

~ $ time  pip install --no-dependencies   --no-binary SentencePiece SentencePiece
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: SentencePiece in /data/data/com.termux/files/usr/lib/python3.11/site-packages (0.2.0)

real	0m2.616s
user	0m1.569s
sys	0m0.577s
~ $ 

Manamama added a commit to Manamama/AI_scripts_Public_1 that referenced this issue Dec 19, 2023
@Manamama
Copy link
Author

Manamama commented Dec 19, 2023

FYI, I know it should not be here, as it is not a bug, but I wrote a quick script that runs it in Termux etc, with cute statistics.

Ms Bing explained it to me via PlantUML as below
SentencePiece

@ghost
Copy link

ghost commented Dec 19, 2023

first, obviously, then changing these tidbits ( in .../sentencepiece/CMakeLists.txt, as I wrote) that you will see, and maybe paying attention to the missing libraries that
python setup ...
or
pip install ...
logs may throw at you, and installing these missing bits (modules etc.), if any.

Thankyou very very much 😌 I've successfully installed sentence-transformers because of you, for me you'll always be a genius and a person who helped me when I was loosing hope 😭

RDT_20231219_084911264719658203460677.jpg

Thank you very very very much once again ☺️❤️

@xhy2008
Copy link

xhy2008 commented Jan 21, 2024

Which version of sentencepiece are you using?
I looked through the CMakeLists.txt,but I didn't find the line you mentioned.
I am using the latest version,and it still make errors.

@ghost
Copy link

ghost commented Jan 22, 2024

Just add this line at top of CMakeLists.txt set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -llog")
if you still have problems see this Reddit Post.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants