Not sure what is different, or if it's Adobe, but something weird is happening...

[samanthavenialogan]

Hello!

I have been using PDF++ for a month or so now and I've been particularly concerned with exporting annotations from Obsidian over to Airtable where I use a data warehouse infrastructure. A big part of this is using the global copy highlights function.

@RyotaUshio has been incredibly helpful in getting this to work by making it possible to pull both the highlighted text as a quote but also the annotations added to it.

Sadly, I've run into several odd PDFs, and I cannot tell what's different about them that breaks the system and prevents me from pulling the information I need. For some reason the copy/paste does not return any results at all. the underlines there are supposed to be the highlighted text.

When highlighting the text in the initial reading phase, I can see the OCR worked and there IS TEXT to be highlighted. When I use a simple raw copy/paste function on text it works just fine. But when you use the annotation copier to get the backlink, quote, and comment, the quote is not present at all.

I've also asked Obsidian to open the file in standard Adobe Acrobat and I discovered that the highlight pulled into the annotation system with dummy text only of, "highlighted text." From here if you leave an annotation note it is kind of random whether it actually pulls or not - I can't tell the difference between the reports that work and the ones that do not.

First my setup:

My callout template: I have rebuilt this from simplest back up to its current state

Note: to be clear I will be posting my intended template code and the result is the SAME regardless of the {{selection}} or {{text}} variable being used.

I have 2 files included. Both have been uploaded to this system.

1. Report-CMX-Community Industry Trends Report-Bevy-2020.pdf <-- This one works.
2. report-cmx-community-industry-report-2023.pdf <-- This one does not.

Expected performance based on the 2020 document (works):

1. Begin highlighting directly through Obsidian PDF++. Annotations should appear in Adobe Acrobat reader OR Nitro.
2. click on each highlight and assign the color and the comment.
3. once completed with the reading copy/paste all highlights using the text extractor function
4. Paste and convert markdown format to HTML document format, and then place in Airtable Quotes database.

What the system returns for the 2020 document (as expected).

> [!PDF ] [Report-CMX-Community Industry Trends Report-Bevy-2020, p.2](Report-CMX-Community%20Industry%20Trends%20Report-Bevy-2020.pdf#page=2&annotation=614R)
>>**Quote**: _The value and differences between in-person and online communities._
>
>>**Comment**:  Specific focus on what to study with metrics still present in the report. 

What I have tried on the 2023 report that does not work

Confirm that the PDF has an OCR index and is searchable with recognizable text:

It's clear that the text is highlighted and functional for extraction.

Raw copy works:

Communities are (finally!) back in person! Last year, 29% of those who host events said they offered an in-person event, and this increased to 59% this year. However, virtual events remain a staple for 90% of communities.

Copy comment returns: >

[!PDF ] [report-cmx-community-industry-report-2023, p.35](report-cmx-community-industry-report-2023.pdf#page=35&annotation=1444R)
>>**Quote**: __
>
>>**Comment**:  This is a comment.

In this result, you can see that the comment was pulled but the highlighted text was not. The underscore in its place is actually the markdown for italicizing it and when the quote is not pulled, it shows empty. This only happens on some PDFs and I can't figure out what the difference is between the various files.

To better understand, I've pulled the PDF into Adobe Acrobat, Nitro Pro, Scribus, and Libra Draw - removing Obsidian from the equation. All of them, when observing the highlight actually look like this instead of pulling the actual text as intended:

In this Screenshot, you'll see that the text is available in the OCR but the highlight is coming across very weird - as bounding boxes rather than anything else. The text is not highlighted individually but as a group. In the comments listing on the right, instead of the text represented it simply says "highlighted text"

This suggests to me that the PDF itself is not rendering the highlights correctly so it must not be the obsidian engine, but the other reports from other years do just fine. The issue is that both documents clearly show in their priorities the same permissions:

2020 report (works)

2023 report (does not work)

Other steps I've tried

1. Re-ran both documents through Nitro Pro OCR
2. Re-rendered both PDFs using Scribus
3. Completely opened both PDFs restrictions as a result.
4. Completely re-wrote my prompt within obsidian using very basic calls {{text}} only.
5. Completely re-highlighted a full document in native Adobe to see if it wasn't that - still get the issue.
6. Totally rebuilt the copy/paste prompt in Obsidian hoping that was the issue - it was not.

TBH I have spent WAY TOO MUCH time trying to figure this out but I really need the data. I resorted to copy/pasting things manually so that I'm not slowing down my project but if someone can figure out why this is not pulling correctly, what is different and how to prepare PDFs for use in Obsidian PDF++ I would GREATLY appreciate it :)

That said, thanks for listening!

[RyotaUshio]

In short, this difference comes from the fact that these two PDFs have their text written in different ways (maybe because of the OCR software used for the 2023 file?).
And importantly, you will find this problem persist when you disable PDF++ and use Obsidian's native feature for copying annotations:

Screen.Recording.2024-06-12.at.20.56.38.mov

So this is basically Obsidian's problem, and PDF++ just inherits it. You might want to send them a feature request (or bug report, though I'm not sure if they would think of this as a bug) via the official forum.

That being said, it might be possible to fix this behavior on my end. I've created a new issue for that: #224
There's no ETA, but I'll work on it when I have time.

[samanthavenialogan]

TYou are a phenomenal creator~! I am still exploring Obsidian as a workflow option (I'm only one month in and use has been sporadic as I definitely have enough covering my process at the moment - but once I'm set and established would you prefer a Github contribution or do you have a patreon-like?

[samanthavenialogan]

Also totally seperate sidenote regarding all of this - my template is labeled in Markdown as that's the preferred method for obsidian. I totally agree with this but could you add a stripped down text format that strips the markdown character implementations from the text and returns only the highlighted bit and the comment without markown involved - or do you know of a separate tool that I can use for this?

Additional update though, Out of curiosity I wanted to know what Discords in-browser system is going to do to handle the comments as I know that it reads FPF files and it resulted in the same issue when I pull those via my browser extension. Might be additional data in the process of producing the issue, but I'm hesitant to move the behavior into the issue as I haven't looked in to it to any advanced degree. n

I've been using the OCR text extractor which only returns all data as a string and does not limit to the highliths - totally understandable giving the simplicity of the result due to the complexity of the pull. I'm also hoping that eventually obsidian OCR will improve such that it can pull character by character. I think the current plugins result in this format of pulling data instead of the other way around.

[RyotaUshio]

would you prefer a Github contribution or do you have a patreon-like?

Thank you for considering supporting me! Currently I have three options of sponsorship platforms:

• GitHub Sponsor
• Buy Me a Coffee
• Ko-fi: the only platform that supports PayPal

could you add a stripped down text format that strips the markdown character implementations from the text and returns only the highlighted bit and the comment without markown involved

You can customize the format in which the annotation link is copied. Please refer to the docs: https://ryotaushio.github.io/obsidian-pdf-plus/link-copy-formats.html

In short, you can go to PDF++ > Link copy templates > Custom link copy formats and then add/edit/delete the formats.

You will want to set something like this (this is just an example):

"{{text}}"

{{comment}}

Additional update though, Out of curiosity I wanted to know what Discords in-browser system is going to do to handle the comments as I know that it reads FPF files and it resulted in the same issue when I pull those via my browser extension. Might be additional data in the process of producing the issue, but I'm hesitant to move the behavior into the issue as I haven't looked in to it to any advanced degree.

Sorry, I don't understand this part. Probably out of my knowledge.