Can tables with an unclear format be parsed successfully? #897
Closed
codePigPig
started this conversation in
Ask for help with specific PDFs
Replies: 2 comments
-
|
In this particular PDF, there are no line based separators. For such, you need to use the text separators and define the vertical lines explicitly. For best results, crop the page. Here is the result for the top table. You can do the same for the bottom table. import pdfplumber
pdf = pdfplumber.open("pdftest02.pdf")
page0 = pdf.pages[0]
# Crop the page by keeping only the 11-56% of the page.
page0 = page0.crop((0.0 * float(page0.width), 0.11 * float(page0.height), 1.0 * float(page0.width), 0.56 * float(page0.height)))
ts = {
"vertical_strategy": "explicit",
"horizontal_strategy": "text",
"explicit_vertical_lines": [per * float(page0.width) for per in [0.11, 0.226, 0.29, 0.35, 0.41, 0.47, 0.53, 0.59, 0.65, 0.71, 0.77, 0.83, 0.89]], # Define explicit vertical lines. What this is doing is adding a line at the 11%, 22.6%, ... of the PDF's width.
"intersection_x_tolerance": 10,
}
# This code saves the debug visual output.
im = page0.to_image(resolution=200)
im.reset().debug_tablefinder(ts)
im.save("image.png", format="PNG")
# Extract the tables.
tables = page0.extract_tables(table_settings=ts)
for table in tables:
print()
for row in table:
print(row)The output will be
Please note that in this, certain table headers are spanning into multiple rows and that you would need to handle in the post processing. For example, the header ts = {"vertical_strategy": "explicit", "horizontal_strategy": "text"}and it will give you the result as but it misses out on the header rows. |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
Perhaps some of this is also useful. I tried to use the left-most lines as page dividers: page_left = min(page.lines, key=itemgetter('x0'))['x0']
lines = [ line for line in page.lines if line['x0'] == page_left ]
Then we:
import pdfplumber
from itertools import pairwise
from operator import itemgetter
pdf = pdfplumber.open("Downloads/pdftest02.pdf")
page = pdf.pages[0]
page_left = min(page.lines, key=itemgetter('x0'))['x0']
page_right = page.bbox[-2]
page_bottom = page.bbox[-1]
lines = [ line for line in page.lines if line['x0'] == page_left ]
crops = []
for top, bottom in pairwise(lines):
crop = page.crop((top['x0'], top['top'], page_right, bottom['top']))
crops.append(crop)
crop = page.crop((bottom['x0'], bottom['bottom'], page_right, page_bottom))
crops.append(crop)
tables = []
for crop in crops:
crop_tables = (
crop.find_tables(dict(
vertical_strategy="text",
horizontal_strategy="text"
))
)
crop_bottom = crop.bbox[-1]
crop_right = max(crop.lines, key=itemgetter('x1'))['x1']
vertical_lines = [ line['x0'] for line in crop.lines ] + [crop_right]
horizontal_lines = [ cell[1] for table in crop_tables for cell in table.cells ] + [crop_bottom]
tables.extend(
crop.find_tables(dict(
explicit_vertical_lines=vertical_lines,
explicit_horizontal_lines=horizontal_lines
))
)Table 1: Table 2: |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Question:
Sorry to bother you again. I have a very complex PDF table and I have tried to extract the data from it, but without success. The format of this table is indeed very complicated, which makes me very frustrated. I am looking forward to your reply. Thank you very much.
This is my test pdf:
pdftest02.pdf
Beta Was this translation helpful? Give feedback.
All reactions