How to run and test deepdoctection in Docker?

[rsoika]

Hi,
I did not yet understand how to test deepdoctection with the provided docker image.

I created the following docker-compose.yaml and .env file:

version: '3'
services:
  dd-gpu:
    image: deepdoctection/deepdoctection:v.0.29.0
    stdin_open: true
    tty: true
    volumes:
      #- ${CACHE_HOST}:${DEEPDOCTECTION_CACHE}
      - ${WORK_DIR}:/home/files

.env:

CACHE_HOST="/home/test/deepdoctection/cache"
WORK_DIR="/home/test/deepdoctection/workdir"

I can start the container successfully with docker compose up but I did not understand how to test the framework. Just copy .pdf files into the WORK_DIR did not trigger anything.

What is needed to analyze a pdf test file?

Thanks for any hints.

===
Ralph

[Doronal-LQC]

I am also facing the same problem, Thanks @rsoika

[rsoika]

In the meantime I figured out that the container simply provides you with a python environment. So what you need to do is, open a bash on the running container. Than you can execute the phyton scripts mentioned in all the examples.

In my case it was necessary to install the additional tools:

$ pip3 install IPython
$ apt install qpdf

This can all be done from within the container.

In your $workdir you can now create your php scripts. For example here is a script that I used for testing:

import deepdoctection as dd
from IPython.core.display import HTML
from matplotlib import pyplot as plt

analyzer = dd.get_dd_analyzer()  # instantiate the built-in analyzer similar to the Hugging Face space demo

df = analyzer.analyze(path = "/home/files/Rechnung_Muster.pdf")  # setting up pipeline
df.reset_state()                 # Trigger some initialization
doc = iter(df)
page = next(doc) 
image = page.viz()
plt.figure(figsize = (25,17))
plt.axis('off')
plt.imshow(image)
print(page.text)
# HTML(page.tables[0].html)
print(" Pages=")
print (len(page.tables))

Hope this will help you to get started

[Doronal-LQC]

Thanks for the quick response
I had an Idea based on what you did and described in documentation, using the base image for the Dockerfile and build a consumer that deals with the pipeline
This is the Dockerfile

FROM deepdoctection/deepdoctection:v.0.29.0

COPY .. .

RUN python3 -m pip install -r "requirements.txt"

CMD ["python3", "consumer.py"]

and inside the consumer.py there is -

from deepdoctection import get_dd_analyzer
import pika
from pathlib import Path, PurePath
import os
import pickle


def parse_pdf(file_path):
    analyzer = get_dd_analyzer(config_overwrite=["USE_PDF_MINER=True", "USE_OCR=True"])
    df = analyzer.analyze(path=file_path)
    df.reset_state()
    doc = iter(df)
    page = next(doc)
    # with open('output.pkl', 'wb') as file:
    #     pickle.dump(chunks, file)
    chunks = [{"layout": chunk[5], "text": chunk[6]} for chunk in page.chunks]
    os.remove(file_path)
    return chunks