OCR Plugin on another computer

[tcsenpai]

As the OCR plugin is very heavy on the CPU, I managed to follow the README.md for the OCR plugin and start it on another PC in my LAN.

When I configure, either manually or with the plugin command, the OCR endpoint to be remote, I get always "500 Error" with "NoneType" received by the OCR endpoint.

Configuration:

[arkohut]

Are you using the file https://github.com/arkohut/pensieve/blob/master/memos/plugins/ocr/server.py ? It works for me...

Can you show more logs and error messages about this.

[tcsenpai]

Yes I am using it, on another computer indeed.

The error I get is:

Server

OCR Plugin server.py (running on a LAN address)

Additional notes

• The embeddings endpoint (ollama) works
• The multimodal model (ollama) works
• They both are on the same PC running the OCR plugin

[arkohut]

Could you please show the full commands to start up the ocr predict server.

[tcsenpai]

I just run uvicorn server:app --host 0.0.0.0 --port 5555 on the PC I want to use as a local predict server. I had to use uvicorn server:app and not uvicorn main:app as the app object is into server.py . Also I had to pip install memos anyway on that computer else it would complain about dependencies missing.

[arkohut]

That’s so weird. I’m using the same method, and it works for me. Based on your log, it seems like there’s no indication of which inference engine is being used. For instance, in my logs, it shows that I’m using onnxruntime CPU.

[tcsenpai]

What data do the client sends to the server? I am on Linux using the PR I opened, could it be that?

[arkohut]

You can open link http://localhost:5555/predict to see the doc about the api.

@tcsenpai I am sorry. It is http://localhost:5555/docs

[tcsenpai]

It spawns 405 Method Not Allowed on the OCR Plugin Server console, both with localhost on the OCR Plugin Server PC and through lan.

[tcsenpai]

@arkohut ok the /docs endpoint works.

I have to diagnose what is sent from the localhost client to the OCR Server , will try to do it

[rty813]

@arkohut I used ocr plugin in master branch, and same issue occured.

[rty813]

I know the reason. Because you initialize process_pool in the if __name__ == '__main__' entry. But if you start it through uvicorn, this entry will not be executed. Therefore, process_pool will not be initialized, but None

[rty813]

share my dockerfile:

FROM python:3.12-slim

RUN sed -i 's@http://deb.debian.org@http://mirrors.tuna.tsinghua.edu.cn@g' /etc/apt/sources.list.d/debian.sources \
    && apt-get update \
    && apt-get install -y --no-install-recommends libglib2.0-0 libgl1 \
    && rm -rf /var/lib/apt/lists/*

ADD requirements.txt /tmp/
RUN pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple -r /tmp/requirements.txt

WORKDIR /root/ocr
ADD . .
ADD models/* /usr/local/lib/python3.12/site-packages/rapidocr_onnxruntime/models/

EXPOSE 8000
CMD [ "python3", "server.py", "--port", "8000" ]

requirements.txt:

rapidocr_onnxruntime
httpx
fastapi
uvicorn
memos

logs:

root@104 ~/z/ocr# docker logs -f ocr
INFO:     Started server process [1]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
INFO:__main__:Initialized OCR with RapidOCR ONNX Runtime (CPU)
INFO:     192.168.3.123:27449 - "POST /predict HTTP/1.1" 200 OK
INFO:     192.168.3.123:3123 - "POST /predict HTTP/1.1" 200 OK
INFO:     192.168.3.123:3136 - "POST /predict HTTP/1.1" 200 OK
INFO:     192.168.3.123:6954 - "POST /predict HTTP/1.1" 200 OK
INFO:     192.168.3.123:7014 - "POST /predict HTTP/1.1" 200 OK

[arkohut]

I know the reason. Because you initialize process_pool in the if __name__ == '__main__' entry. But if you start it through uvicorn, this entry will not be executed. Therefore, process_pool will not be initialized, but None

Yes, you are right. I will fix such a bug.

[tcsenpai]

I know the reason. Because you initialize process_pool in the if __name__ == '__main__' entry. But if you start it through uvicorn, this entry will not be executed. Therefore, process_pool will not be initialized, but None

Great debugging work. I will wait for the fix and test on my instance as well.

[arkohut]

I know the reason. Because you initialize process_pool in the if __name__ == '__main__' entry. But if you start it through uvicorn, this entry will not be executed. Therefore, process_pool will not be initialized, but None

Great debugging work. I will wait for the fix and test on my instance as well.

The work is already done. Check it out!

[tcsenpai]

aaaaand it works :) Kudos!