Not extracting tabular data

[88arvin]

How to extract only tabular data from a pdf in which columns are separated by tabs and in some pages there are multiple tabular data.

I have tried extract.table and extract.tables, but I get an empty list or none in return. I have tried extract.text also; however, it is showing complete (text + tabular) data. 

[jsvine]

Hi @88arvin, and thanks for your interest in this library. Unfortunately, without seeing a specific PDF, it's difficult to provide guidance. Can you attach the PDF you're working with?

[88arvin]

sample.pdf

[samkit-jain]

Hi @88arvin Appreciate your interest in the library. Thanks for sharing the PDF. To extract the table, you should use the explicit_vertical_lines table strategy. For the PDF you shared, a setting might be

{
    "vertical_strategy": "explicit",
    "horizontal_strategy": "text",
    "explicit_vertical_lines": [60, 120, 300, 400, 500, 560, 660, 750]
}

Feel free to tweak as you see fit. You can find more options here. Using the above strategy, it will give you only one table with all the data. You will have to apply some post processing to remove the unwanted rows. For example, a regex based approach could be to keep only those rows that have the first column as a date. It will give you all the rows that are having a transaction.

Furthermore, a word of caution, the PDF you shared hasn't been properly redacted. The data that you have masked by using the black highlighter, is still very much visible. One can just select the text, copy and paste. Since it is a bank statement, my suggestion would be to properly redact and hide any sensitive information. This advice may be ignored provided the necessary consent has been taken from the user.

[samkit-jain]

Scanned statements won't work. You'll have to run an OCR on it and create a copyable PDF and then run on it.

[chanpreet90]

It is a copyable PDF. Only first (date) column is not being extracted.

[samkit-jain]

Hi @chanpreet90 Please share the PDF so that it becomes easier to look into the issue. Also, might be a better option to create a new discussion around the issue.

[chanpreet90]

20010222002119 (1).pdf

[samkit-jain]

The whole first page is an image and is rightly a proper scanned statement. Auto detection of line separators for table extraction won't work on this PDF.

[88arvin]

Thank you soo much. :)
I tried your suggested approach, and it is working well, but it is extracting only the rows that have dates in the rows.
The issue now is that this method excludes certain details from the particulars column because they are spread across numerous s lines that don't have dates.

[samkit-jain]

In that case, you can write your own custom logic. For example, if the rows are like

date	particulars	chq	withdrawal	deposit	balance
date	particular 1	chq	withdrawal	deposit	balance
	particular 2
date	particular 3	chq	withdrawal	deposit	balance
date	particular 4	chq	withdrawal	deposit	balance

You can have the logic as - Merge the rows i and i+n when row i+n+1 and row i are valid transaction rows and all the rows i+1 to i+n have only the particular columns filled.