How to extract rotated text

[RitchieP]

I'm currently trying to extract text from a PDF file that contains rotated text. My current code works fine in terms of extracting the correct rotated text. However, the text extracted is not within a line.

An example of the PDF file:

Text extracted

D
o
1. 5+0.1
0.0
D
1
1.
5
0
MI
N.

My expected output will be, for example, Do in one line, D1 in one line, MIN. in one line.

I think there's a rotation parameter exposed by pdfminer.six here at line 38, but I'm not sure if that's used to extract rotated text, and whether can I use from pdfplumber

[jsvine]

Hello, and interesting example! Right now, pdfplumber does not have much direct support for text that is not rotated by 0, 90, 180, or 270 degrees. I'd have to think a bit about how best to support that. In the meantime, you might be able to achieve what you want, for your particular use-case, by taking advantage of the char["matrix"] property. See here for a bit more detail: #642 (reply in thread)

[RitchieP]

So, within the extract_text function, I'll have to specify the extra_attrs=["matrix"]? How do I specify the extra attributes to extract the rotated text?

For example, do I have to write code like this:

page.extract_text(extra_attrs=["matrix"=(SOME, VALUE, 1, 2, 3, 4)])

Or is there some other way to specify the rotation?

[petermr]

I have done this sort of analysis in other environments. I don't know the precise details of pdfminer's / pdfplumber's CTM so don't hesitate to correct me! but normally these follow the description in https://en.wikipedia.org/wiki/Transformation_matrix#Rotation. Your image looks like a pure rotation - the normal problem is to find the point about which it is rotated. (Note that signs of rotation may differ in different systems - you may have - and + reversed
If so in

a b e 
c d f

a b c d describes the rotation (which appears to be about 30 deg clockwise (which is normally a negative value as positive is anticlockwise)

cost sint
-sint cost

(depending on the direction of t)
and e f the translation after rotation. It doesn't seem necessary to have shear.

Your problem is to assemble the correct characters on a line and calculate the (new) x coordinates so you can determine the sequence of characters, spaces, etc. You need to determine the points about which the string is rotated. In this case you could rotate the whole diagram by +30 deg (e.g. anticlockwise around the origin) and try to find new strings as if they were aligned along X.

It's quite likely that the reading order of the characters is arbitrary so you will need to use geometry to determine which string is which. You can't rely on them being written 1.5... and then 1.50 MIN

Hope this is enlightening rather than confusing. May be useful to post a snippet of the raw matrices.

[petermr]

Note that you only need to rotate the coordinates of the characters, not the glyphs. (That requires translate-to-origin, rotate, translate-back). It's possible that the y coordinate will be a bit variable so you may have to smooth these. If you post the matrices (and all char data) for the character strings I would be interested in doing this as I expect I will need it some time.

[RitchieP]

Thanks for the insights! I'm pretty sure I'm able to understand your point. The challenge now is how do I implement it within the code.

If you would like to try it out, here is my code currently

with pdfplumber.open("example.PDF", laparams={"detect_vertical": True}) as pdf:
  page = pdf.pages[0]
  page.crop(region).extract_text(vertical_ttb=False)

The region is a value of four tuples provided by the user. The PDF file can be found here

I'm thinking I'll need to add an extra_attrs argument in the extract_text function to include the rotated matrix of the rotated strings during extraction. (Maybe I'm wrong)

[RitchieP]

Would you also mind explaining how can I rotate the PDF page? That idea could be good enough to solve the problem.

[petermr]

If all the strings are rotated by the same amount, then assume the characters are

x0a y0a x1a, y1a, chara
x0b y0b x1b, y1b, charb

which I think you already have. Estimate the rotation angle.
then transform each (by multiplying by a. 2d rotation matrix. The absolute position shouldn't matter. You should find that
y0a' y0b' ... should be identical within error. If not it's a 1-D optimisation and numpy will have methods

If their are many lines at different angles you will need heuristics. If the characters come in an ordered stream, then see if the vectors
(x0a, y0a) -> (x0b, y0b)
and
(x0b, y0b) -> (x0c, y0c)
are parallel

If the characters come in random order, you will have to calculate inter-character distances (Pythagoras) and see which join into strings and then find the rotation angle for each string.

[RitchieP]

I have tried to multiply the 2d rotation matrix. However, I realized I'm not able to overwrite the original coordinates of the character. I would also like to know that, if I follow this method, am I rotating the text, or am I rotating the axes?

[petermr]

You are rotating the x0,y0 (and possibly x1, y1) of each character. These are points annotated/linked to the codepoints. But it's only the coordinates that are transformed. I can't give a general answer as you have only shown a small image (I assume there is more). The key thing is whether all the text is rotated by the same amount or whether each string is at a different angle. That will be harder.

…

On Tue, May 9, 2023 at 10:18 AM Ritchie Poh ***@***.***> wrote: I have tried to multiply the 2d rotation matrix. However, I realized I'm not able to overwrite the original coordinates of the character. I would also like to know that, if I follow this method, am I rotating the text, or am I rotating the axes? — Reply to this email directly, view it on GitHub <#875 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFTCS5Y5XFSAP62JX5BV73XFIDUVANCNFSM6AAAAAAXOV5T3U> . You are receiving this because you commented.Message ID: ***@***.***>

-- Peter Murray-Rust Founder ContentMine.org and Reader Emeritus in Molecular Informatics Dept. Of Chemistry, University of Cambridge, CB2 1EW, UK

[RitchieP]

Hi peter! Thanks for the help! I'll need to take some time off this issue first to take care of other client requirements. However, I appreciate your help and will mark your response as an answer when I start working back on it!