Lots of whitespaces in between words

[fintech132]

Describe the bug

A clear and concise description of what the bug is.
When I try to extract text from the PDF file, there are lots of empty spaces in between words.

Code to reproduce the problem

`
#utility to test pdfplumber
import sys
import pdfplumber

def pdf_to_text(input_path, output_path):
page1_text = ''
with pdfplumber.open(input_path) as pdf:
#deal with multiple pages
i = 1
for p in pdf.pages:
page1_text = '\n\npage ' + str(i) + '\n'
tmp = p.dedupe_chars().extract_text(x_tolerance=3, y_tolerance=3, x_density=7.25, y_density=7.25, layout=True).split('\n')
page1_text += str(tmp)
i = i + 1

with open(output_path, "w", encoding="utf-8") as output_file:
    output_file.write(page1_text)

if len(sys.argv) != 3:
print("Usage: python script.py ")
sys.exit(1)

pdf_to_text(sys.argv[1], sys.argv[2])
`

PDF file

Please attach any PDFs necessary to reproduce the problem.

See attached pdf file

SampleOutput.pdf

Expected behavior

What did you expect the result should have been?
The sentence should be extracted as the following:
This is a sample employment application form. Please carefully read

This issue is all of the extracted text. I just point out one sentence.

Actual behavior

What actually happened, instead?
The output text is (note there are lots of whitespaces):
This is a sample employment application form. Please carefully read

Screenshots

If applicable, add screenshots to help explain your problem.

Environment

• pdfplumber version: [e.g., 0.5.22]: 0.10.3
• Python version: [e.g., 3.8.1]: 3.9.18
• OS: [e.g., Mac, Linux, etc.]: Mac

[jsvine]

This PDF uses character sizes that are unusually large:

import pdfplumber
from collections import Counter

pdf = pdfplumber.open("./SampleOutput.pdf")
page = pdf.pages[0]

print(Counter(sorted(c["size"] for c in page.chars)))

Result:

Counter({92.0: 13,
         93.0: 62,
         94.0: 184,
         94.99999999999989: 17,
         95.0: 32,
         96.0: 8,
         114.0: 25,
         115.0: 67,
         116.0: 22})

For that reason, you'll need to use x_density and y_density values much larger than the defaults. With that, it seems to work. E.g.,:

page.extract_text(
  layout=True,
  x_density=75,
  y_density=90,
  y_tolerance=10
)

... produces this:

                                                                      
                                                                      
                         Employment Application                       
                                                                      
                                                                      
  This is a sample employment application form. Please carefully read 
  and answer all questions.                                           
                                                                      
                                                                      
   Personal Information                                               
                                                                      
   Full Name:        Jane Doe                                         
                                                                      
   Phone Number:     555-0100                                         
                                                                      
   Home Address:     123 Any Street, Any Town, USA                    
                                                                      
   Mailing Address:  Same as home address                             
                                                                      
                                                                      
   Work History                                                       
                                                                      
   Current Company:  Any Company (2018-Current)                       
                                                                      
                     Any Role                                         
                                                                      
   Company#1:        Previous Company # 1 (2014-2018)                 
                                                                      
                     Previous Role # 1                                
                                                                      
   Company#2:        Previous Company #2 (2010-2014)                  
                                                                      
                     Previous Role # 2                                
                                                                      
                                                                                                                                        

[fintech132]

Thank you very much. But is it possible to set the x_density and y_density dynamically according to the font size?

[jsvine]

You're very welcome. Philosophically, text-extraction settings in pdfplumber involve no magic guesswork, largely because PDFs differ in so many ways, and users have such different ideas of what the ideal settings would be. My suggestion would be to write your own version of a dynamic density-choosing function to suit your particular needs.

[fintech132]

Awesome, great work.