creating requirements.txt for applications using PDFPlumber

[petermr]

I am having some difficulty loading and running tests using PDFPlumber and wonder if there are any particular versions of pdfminer or pdfminer.six that should be used.

My tests and main run OK from the commandline but fail in Pycharm.

Here is a typical runtime error:

/usr/local/bin/python3.9 /Users/pm286/workspace/amilib/amilib/amix.py 
Traceback (most recent call last):
  File "/Users/pm286/workspace/amilib/amilib/amix.py", line 20, in <module>
    from amilib.pdf_args import PDFArgs
  File "/Users/pm286/workspace/amilib/amilib/pdf_args.py", line 15, in <module>
    from amilib.ami_pdf_libs import AmiPage, PDFParser, DEBUG_OPTIONS
  File "/Users/pm286/workspace/amilib/amilib/ami_pdf_libs.py", line 18, in <module>
    import pdfplumber
  File "/usr/local/lib/python3.9/site-packages/pdfplumber/__init__.py", line 15, in <module>
    from .pdf import PDF
  File "/usr/local/lib/python3.9/site-packages/pdfplumber/pdf.py", line 17, in <module>
    from .page import Page
  File "/usr/local/lib/python3.9/site-packages/pdfplumber/page.py", line 25, in <module>
    from pdfminer.pdfinterp import PDFPageInterpreter, PDFStackT
ImportError: cannot import name 'PDFStackT' from 'pdfminer.pdfinterp' (/usr/local/lib/python3.9/site-packages/pdfminer/pdfinterp.py)

Here is my requirements.txt:

chardet
geopy
lxml
matplotlib
networkx
nltk
numpy
packaging
pandas
pdfminer
pdfminer.six
pdfplumber==0.11.0
Pillow
pyvis
requests
scikit_learn
selenium
setuptools
SPARQLWrapper
tinycss
tkinterweb
webdriver_manager

I appreciate it's difficult to solve other people's library problems but this sort of error is sometimes reported on StackOverflow.
https://stackoverflow.com/questions/57079815/how-to-fix-cannot-import-name-process-pdf-from-pdfminer-pdfinterp-error

[jsvine]

From a quick glance, my best hunch is that the problem stems from including pdfminer (the outdated, pre-fork package that pdfminer.six has replaced). Does it work if you remove that line?

[petermr]

Thanks - makes sense. It's not a problem on the commandline so it seems the pdfminer library is cached in Pycharm somewhere.
Will let you know if it continues to be a problem

[petermr]

The problem is "going away" as I repeatedly flush libraries froom PyCharm!
So to confirm,
We don't need:

• pdfminer
• pdfminer3
  and since all my PDF stuff is done through pdfplumber I have tried to eradicate them from my system.

We do need pdfminer.six (which seems to have similar/identical calls to the above libraries).

• Q: do we need pdfminer.six in the requirements.txt?

and presumably we need to update pdfminer.six if they add significant functionality (such as control points on Beziers)

[jsvine]

pdfplumber includes pdfminer.six among its explicit requirements, pinned to the most recent version found to be compatible with pdfplumber. Currently it is pdfminer.six's most recent release:

pdfplumber/requirements.txt

Line 1 in 2e9819c

pdfminer.six==20231228

If, in the future, you wanted a more recent pdfminer.six version than pinned here, you could manually upgrade, but there would be a potential risk of that version no longer being compatible with pdfplumber.