Parsing PDF with 2 columns in layout?

[Rapid1898-code]

Hello - i try to parse the text form this pdf where the data i seperated in 2 columns as you can see in the pdf.

When i try to extract the lines using this code:

import pdfplumber
import os
import sys
if __name__ == '__main__':
  path = os.path.abspath(os.path.dirname(sys.argv[0])) 
  fn = os.path.join(path, "partly.pdf")
  pdf = pdfplumber.open(fn)    

  workLines = []
  for idxPage, page in enumerate(pdf.pages, start=1):                         
    workLines.extend(page.extract_text().splitlines())

  for l in workLines:
    print(l)

The output is mixed with some information from the the left and the right column of the text:

Is there some way to parse the data in the correct way using the both individual columns from the layout?

partly.pdf

[jsvine]

Hi @Rapid1898-code, and thanks for your interest in this library. I'd suggest using page.crop(...) and then extracting the text from the two halves separately. See, e.g., here:

• extracting text from a two columns page #244 (comment)

• Wrong parsing on two columns pdf file #620 (comment)

Does that work for you?

[Rapid1898-code]

Thanks a lot for the reply - this works fine with the cropping - i have now the following code

import pdfplumber
import os
import sys
if __name__ == '__main__':
  path = os.path.abspath(os.path.dirname(sys.argv[0])) 
  fn = os.path.join(path, "partly.pdf")
  pdf = pdfplumber.open(fn)    

  workLines = []
  for idxPage, page in enumerate(pdf.pages, start=1):                         
    left = page.crop((0, 0, 0.5 * float(page.width), float(page.height)))       
    print(left.extract_text(x_tolerance=2))           
    exit()

My only problem now is the that the order of the text-parts is not full correct
(i suppose that this happend cause its partly engish and partly arabic which is read from right to left instead)

So the first part of the output is:

This is mostly correct but the following sentece

We must act intelligently before
the scandal becomes known among the
people

is interruppted in the middle by
ﺱﺎﻨﻟﺍ ﲔﺑ

Is there any way to get the output as seen in the pdf?

[jsvine]

This is a tricky situation, due to where pdfplumber (via pdfminer.six) believes the character positions to be. See, e.g., this:

page = pdf.pages[0]
im = page.to_image(resolution=150)
im.draw_rects(page.chars)

Here's a section of that image:

As you can see, the bounding boxes for both the English and Arabic letters are vertically misaligned. Although you can sometimes handle issues like that by setting .extract_text(...)'s y_tolerance parameter, the lines overlap too much here for that to work.

However, if you repair the PDF, the boxes improve for the English characters:

With that version, we can get closer to what it seems you want, although the rendering of the Arabic characters might not be quite right:

left = page.crop((0, 0, (page.width / 2 + 10), page.height))
print(left.extract_text(x_tolerance=2, y_tolerance=4))

Output:

1657 ﻑﺮﱡ ﺼﹶ ﺗﹶ n. pl. -aat behavior, conduct;
ﺕﺎﻓﺮﱡ ﺼﹶ ﺗﹶ actions
ﺔﺤﻴﻀﻔﻟﺍ ﴩﺘﻨﺗ ﻥﺃ ﻞﺒﻗ ﺔﻴﻧﻼﻘﻌﺑ ﻑﴫﺘﻟﺍ ﻦﻣ ﺪﺑ ﻻ
ﺱﺎﻨﻟﺍ ﲔﺑ — We must act intelligently before
the scandal becomes known among the
people
91 | 1795 |
1658 ﺪﻋﹺ ﺎﺴﻣﹸ n. assistant, aide, supporter; adj.
helping, assisting, supporting
ﺎﻬﻌﺿﻭ ﻥﻷ ؛ﺝﺍﻭﺯ ﲑﻐﺑ ﲔﺛﻼﺜﻟﺍ ﺕﺯﻭﺎﺟ ﺀﲈﻴﺷ
ﺎﻬﺘﺻﺮﻓ ﻦﻣ ﺍﲑﺜﻛ ﻞﻠﻗ ﺐﻄﻟﺍ ﺔﻴﻠﻛ ﰱ ﺪﻋﺎﺴﻣ ﺱﺭﺪﻤﻛ
— Shima had passed thirty without marriage;
because her situation as a teaching assistant
in the College of Medicine decreased her
opportunities

[...]

... or, with ..., layout=True, y_density=12)

                                   
          1657 ﻑﺮﱡ ﺼﹶ ﺗﹶ n. pl. -aat behavior, conduct;
             ﺕﺎﻓﺮﱡ ﺼﹶ ﺗﹶ actions   
             ﺔﺤﻴﻀﻔﻟﺍ ﴩﺘﻨﺗ ﻥﺃ ﻞﺒﻗ ﺔﻴﻧﻼﻘﻌﺑ ﻑﴫﺘﻟﺍ ﻦﻣ ﺪﺑ ﻻ
             ﺱﺎﻨﻟﺍ ﲔﺑ — We must act intelligently before
             the scandal becomes known among the
             people                
             91 | 1795 |           
                                   
          1658 ﺪﻋﹺ ﺎﺴﻣﹸ n. assistant, aide, supporter; adj.
             helping, assisting, supporting
             ﺎﻬﻌﺿﻭ ﻥﻷ ؛ﺝﺍﻭﺯ ﲑﻐﺑ ﲔﺛﻼﺜﻟﺍ ﺕﺯﻭﺎﺟ ﺀﲈﻴﺷ
             ﺎﻬﺘﺻﺮﻓ ﻦﻣ ﺍﲑﺜﻛ ﻞﻠﻗ ﺐﻄﻟﺍ ﺔﻴﻠﻛ ﰱ ﺪﻋﺎﺴﻣ ﺱﺭﺪﻤﻛ
             — Shima had passed thirty without marriage;
             because her situation as a teaching assistant
             in the College of Medicine decreased her
             opportunities         
             93 | 1760 |           
     
[...]                              

[Rapid1898-code]

Thanks a lot for your support - i will check that