AdvancedProfanityFilter does not detect words of different fonts I think?

[MaryAtlas]

Hello. I am using Chrome browser with the extension installed, and with all modes tried, Normal, Advanced, Deep, it would not filter the bolded words. Here is the exact link where it occurred.
https://shopee.com.my/product/74423802/25055503788
If you scroll down to the Product Description paragraph, the bolded words are
"𝐁𝐞𝐬𝐭 𝐓𝐫𝐚𝐯𝐞𝐥 𝐏𝐚𝐫𝐭𝐧𝐞𝐫 𝐈 𝐒𝐚𝐟𝐞 & 𝐄𝐚𝐬𝐲 𝐔𝐬𝐞
𝐋𝐞𝐭'𝐬 𝐏𝐥𝐚𝐧 𝐘𝐨𝐮𝐫 𝐇𝐨𝐥𝐢𝐝𝐚𝐲 𝐰𝐢𝐭𝐡 𝐓𝐫𝐚𝐯𝐞𝐥𝐥𝐨𝐫 𝐍𝐨𝐰".
Not a single word in these sentences could be filtered by AdvancedProfanityFilter.
What is going on?

P.S: Try opening the link in incognito mode if it redirects you to the sign up page.

[richardfrost]

@MaryAtlas, The issue is that those characters are actually not "normal" letters. They look like it, but they are actually special Unicode characters. I'll have to see if I can find a good way to try and match characters that are close/look like other characters.

I found a link to avoid the sign-up page here.

[MaryAtlas]

Hmm? Do you mean that AdvancedProfanityFilter could not filter special Unicode characters? It does not support them?
I actually copied the exact special Unicode words from the text and added it into filter. And it did not work.

I don't mind if actual words like B did not filter special "𝐁". But shouldn't it filter "𝐁" if I add exactly "𝐁" into filter?

Awesome, can't wait to continue using the awesome tool that you created.

[richardfrost]

@MaryAtlas, If you copy-paste the characters, it should be able to filter them correctly. For example, I added the word 𝐁𝐞𝐬𝐭 to the filter and it filtered the text. Is it not working that way for you? If it isn't filtering on the Shopee page, does it filter here on this Github page (since it was added in the discussion)? Also, if it isn't working, could you share what the "Word" options are for it, like this:

[MaryAtlas]

Yes, it is not working like yours. Here are the screenshots.


I tried adding 𝐁𝐞𝐬𝐭, then I also tried adding 𝐓𝐫𝐚𝐯𝐞𝐥. I tried all 3 modes Normal, Advanced, Deep, nothing works. It still shows.

While it filtered the non-special words as usual, like in the screenshot where I filtered "be" in "could be filtered".

[MaryAtlas]

While waiting for you, I tried "RESTORE DEFAULTS", then solely adding 𝐓𝐫𝐚𝐯𝐞𝐥. It worked and filtered it. However every time I import my backup.json, it goes back to not working. The difference I noticed between them is on a fresh default, when I add 𝐓𝐫𝐚𝐯𝐞𝐥, it is displayed in *** in Words & Phrases dropdown. On my backup.json, it displays as plain 𝐓𝐫𝐚𝐯𝐞𝐥 in Words & Phrases dropdown. Please help for me.

[MaryAtlas]

So this is obviously a bug in your program. Do you want to open a bug report? Are you able to fix it?

[richardfrost]

@MaryAtlas, We can definitely create an issue for this bug, but I will probably need to see a copy of your .json config file to figure out what settings are causing the issue. Can you share a copy (either here, or you can email it to me).

[MaryAtlas]

I have emailed the .json file to richard@richardfrost.info
Please take a look.

[richardfrost]

Thanks for sending that over @MaryAtlas. I think it helped me to figure out where the issue is, but it would be great if you could confirm it. Can you see if it works if you uncheck these options for the "𝐁𝐞𝐬𝐭" word? I think there is a conflict there with these special Unicode characters and those options that I'll have to look into more.

[MaryAtlas]

Oh my, yes it does fix the issue! It is finally filtered. You are a genius. Thank you for the hard work!

[richardfrost]

@MaryAtlas I'm so glad that it is working again for you. I have created an issue with these details. I don't have a lot of time to work on it right now, but at least its on the list now, and I'll work on it when I can. Thanks for you help and patience!