Retrieve text from a PDF and change each table into CSV or Markdown table

[ahmed-bhs]

Is there any method you're aware of for converting a PDF document that contains both tables and text into a text format? Specifically, the tables should be transformed into comma-separated data.

For instance, the outcome should be:

Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Donec quam felis, ultricies nec, pellentesque eu, pretium quis, sem. Nulla consequat massa quis enim. Donec pede justo, fringilla vel, aliquet nec, vulputate eget, arcu. In enim justo, rhoncus ut, imperdiet a, venenatis vitae, justo. Nullam dictum felis eu pede mollis pretium. Integer tincidunt. Cras dapibus. Vivamus elementum semper nisi. Aenean vulputate eleifend tellus. Aenean leo ligula, porttitor eu, consequat vitae, eleifend ac, enim. Aliquam lorem ante, dapibus in, viverra quis, feugiat a, tellus. Phasellus viverra nulla ut metus varius laoreet. Quisque rutrum. Aenean imperdiet. Etiam ultricies nisi vel augue. Curabitur ullamcorper ultricies nisi. 

Sexe,Prénom,Année de naissance
M,Alphonse,1932
F,Béatrice,1964
F,Charlotte,1988

Nam eget dui. Etiam rhoncus. Maecenas tempus, tellus eget condimentum rhoncus, sem quam semper libero, sit amet adipiscing sem neque sed ipsum. Nam quam nunc, blandit vel, luctus pulvinar, hendrerit id, lorem. Maecenas nec odio et ante tincidunt tempus. Donec vitae sapien ut libero venenatis faucibus. Nullam quis ante. Etiam sit amet orci eget eros faucibus tincidunt. Duis leo. Sed fringilla mauris sit amet nibh. Donec sodales sagittis magna. Sed consequat, leo eget bibendum sodales, augue velit cursus nunc, 

[cmdlineluser]

Hi there ahmed-bhs

It's kind of similar to the recent discussion: #1005

There is an approach discussed there, but I didn't think about it too much, so I'm not sure if it's a general solution - it may need some tweaking.

(#1009 is another recent question which essentially asks for the same thing)

Perhaps @jsvine / @samkit-jain have some thoughts on the issue?

Is this particular type of extraction strategy something that can be done reliably?

Is it something that could be supported by pdfplumber?

[jsvine]

Good question, @cmdlineluser. It comes up often enough that it's worth considering. I worry slightly about there being so many different potential page layouts (particularly with regards to intermingling tables and other text) — and so many different user expectations for how those are handled — that maybe this is better handled through a third-party extension / separate repo. But maybe I'm overthinking it, and that a 80% solution would still be valuable.

[cmdlineluser]

Ah, very true - weighing up the pros/cons of such things is quite difficult.

Perhaps if it is asked enough times an "FAQ / Cookbook" doc section for 80% solutions could be helpful.

Using the Discussions search feature I've been able to find most things so far, so perhaps it's a non-issue.

[jsvine]

I do like the idea of expanding the "examples" notebooks (and perhaps reframing it as a "cookbook," which I like). Then there's one semi-canonical link for answering this type of question, while keeping the core library more focused.

[lawrencenika]

Following up on this question, I face the similar problem:

table contains alot of "None" values in the result content. Is it possible to remove them and combine the result table in the format like:
Name | Age | Place
Alex | 21 | USA
Beverly | 22 | UK

Also, if the table span across pages, how to combine them together?

[cmdlineluser]

It kind of depends on the specific PDF you're working with.

In some cases, I've had success by merging everything into a new single page pdf, then chopping out specific sections. #1000 (comment)

But it may not be applicable depending on the exact format of the tables you're working with.

If you have an actual PDF you can share you could open a new discussion.

[lawrencenika]

yes, this file https://drive.google.com/file/d/167Y6KKW5cv0-7r8FV830iofWvYZWP4b6/view?usp=sharing should I start anew discussion?

[lawrencenika]

it contains table of 2 columns

[cmdlineluser]

I see Table 1 "Inputs and Outputs in the tanning industry" that spans 3 pages and has 10 columns, is this the one?

Starting a new discussion is probably a good idea as it could help future readers find it as an example.