Difficulty extracting 'cells' from PDF without edges

[alexreg]

This is the PDF I'm working with. It's proving rather troublesome to even get any table cells extracted from it. I've tried different values for horizontal_strategy and vertical_strategy to no avail.

[samkit-jain]

Thanks for sharing the PDF @alexreg I don't think there is any straightforward table-settings-only way of extracting the data as tables from the PDF. That is because the data is not necessarily represented as tables. It is actually represented as a 2 column page. I would recommend you try the table settings as

{
    "vertical_strategy": "explicit",
    "horizontal_strategy": "text",
    "snap_tolerance": 5,
    "explicit_vertical_lines": [Decimal(page.width) * Decimal('0.1'), Decimal(page.width) * Decimal('0.5'), Decimal(page.width) * Decimal('0.9')],
    "intersection_x_tolerance": 10,
    "keep_blank_chars": True,
}

Since there are no lines in separating the columns, I am using the explicit_vertical_lines strategy to explicitly specify the location of the vertical lines (10% from the left, in the middle, 10% from the right). The table identified is like

Then after extracting the table, you may apply some postprocessing logic to correctly combine the rows.

One other alternative could be to crop the page into 2 halves (left and right) and then extract the text from each half and apply some postprocessing logic so that it is in the way you want it to be.

[erkin98]

Thanks for sharing the PDF @alexreg I don't think there is any straightforward table-settings-only way of extracting the data as tables from the PDF. That is because the data is not necessarily represented as tables. It is actually represented as a 2 column page. I would recommend you try the table settings as

{
    "vertical_strategy": "explicit",
    "horizontal_strategy": "text",
    "snap_tolerance": 5,
    "explicit_vertical_lines": [Decimal(page.width) * Decimal('0.1'), Decimal(page.width) * Decimal('0.5'), Decimal(page.width) * Decimal('0.9')],
    "intersection_x_tolerance": 10,
    "keep_blank_chars": True,
}

Since there are no lines in separating the columns, I am using the explicit_vertical_lines strategy to explicitly specify the location of the vertical lines (10% from the left, in the middle, 10% from the right). The table identified is like

Then after extracting the table, you may apply some postprocessing logic to correctly combine the rows.

One other alternative could be to crop the page into 2 halves (left and right) and then extract the text from each half and apply some postprocessing logic so that it is in the way you want it to be.

what is decimal in there?

[samkit-jain]

Decimal here is a class in the decimal standard library. You can import it using from decimal import Decimal. You may find more information here. https://docs.python.org/3/library/decimal.html#decimal.Decimal

[jsvine]

Hi @alexreg. I think @samkit-jain's answer is a reasonable one. That said, I'd also like to suggest another approach. As @samkit-jain noted, what you have isn't really a table in a strict sense, and so .extract_table(...) isn't going to be a great fit. Instead, I'd recommend doing something like this:

• Divide the page in half, as above
• For each half, identify the unique top positions for each half's bold chars (something you can do by examining the fontname property) of the body-text size (i.e., excluding the header at the top of the page)
• Iterate through those top positions, extracting the text between topi and topi+1. You can do this through page.crop(...).extract_text() or simply through pdfplumber.utils.extract_text(chars) on the relevant subset of characters.

[alexreg]

Thanks very much, @samkit-jain and @jsvine. They both seem like good solutions. My best previous attempt was quite similar to @samkit-jain's, but missed a couple of settings that prevented it from working. What I'll probably do is have a play with both these solutions and see which is more reliable. Cheers!

[alexreg]

Small side question: the accents are getting extracted as separate chars; how can I fix this?

@jsvine