Improve HTML filtering rules $$ -- allow CSS-like selectors

[Alex-302]

Add ability to use syntax, similar to CSS.
For example:
example.com$$td[id="main_content"] > span[id^="post_"][tag-content="advertising"]
and
example.com$$td[id="main_content"] > span[id^="post_"][wildcard="<a href=\"*==""]
and
example.com$$[id="main_content"] > span[id^="post_"][wildcard="<a href=\"*==""]
if tag of [id="main_content"] is random(see 4pda.ru)

[ngorskikh]

Propose to change the current spec in the following way.

Replace

rule = [domains] "$$" tagName [attributes]

with

elementSelector = (tagName [attributes]) / attributes
childCombinator = ">"
rule = [domains] "$$" elementSelector *(childCombinator elementSelector)

.

Replace:

`tagName` — name of the element in lower case, for example, `div` or `script`.

with:

`tagName` — name of the element in lower case, for example, `div` or `script`. An empty `tagName` matches any element.

.

Add to the description of rule parts:

`childCombinator` — an operator that works similarly to the [CSS child combinator](insert link to MDN): that is, the `elementSelector` on the right of the `childCombinator` will only match an element whose direct parent matches the `elementSelector` on the left of the `childCombinator`.

.

Add to the description of the tag-content special attribute:

Due to the way HTML filtering works, the `tag-content` attribute is **not** supported in an `elementSelector` that stands to the left of a `childCombinator`: such rules will be rejected.

.

@ameshkov @Alex-302 @sfionov Discuss.

[ameshkov]

I'd like us to do some improvements alongside this.

1. First of all, add :contains() pseudo support.
2. Second, remove native tag-content and wildcard support. Instead of that make them convert to :contains implicitly and mark them as deprecated in the knowledge base.
3. Open an issue in the tsurlfilter when this is done.

[Alex-302]

Do they have max-length limit for custom value?

[ameshkov]

@Alex-302 this is a very good point!

Btw, do you think it makes sense to extend :contains or :has-text and introduce the max-length there?

[Alex-302]

I'm not sure. Does the length of the removing script or length limit affect applying performance?
If not, it's probably not necessary.

Will there be support for regular expressions in :contains / :has-text?

[ameshkov]

If not, it's probably not necessary.

@ngorskikh Which div will be removed when there's a structure like the one below and a rule like this: $$div:contains(trying to remove)?

If we can guarantee that the inner one (and not the outer one) will be removed, then I guess it's okay to simply drop max-length support. But we should document the logic.

Also, we should check existing rules and decide what we do with them. Do we simply remove max-length from these rules or let CoreLibs handle that attribute and ignore it?

<div>
    ...
    lots of content
    ...
    <div>
        the div that you're actually trying to remove.
    </div>
    ...
    lots of content
    ...
</div>

Will there be support for regular expressions in :contains / :has-text?

Yes, sure.

[ngorskikh]

@ameshkov In CoreLibs, HTML rules are matched against the unmodified HTML stream, which means that in your example, both div elements match and are, as a consequence, deleted. That, probably, is the rationale behind the max-length attribute: to try and disambiguate such cases.

By the way, the same sort of ambiguity exists in CSS rules: for example, the :contains pseudo-class uses the textContent attribute of a node for matching, which is a concatenation of text contents of each child. Meaning that in your example, the outer div also matches and can be hidden by a CSS rule.

[ameshkov]

@ngorskikh makes sense, thank you.

@Alex-302 can there be any trouble because of that? Is max-length used to disambiguate in some cases?

[Alex-302]

@ameshkov

can there be any trouble because of that? Is max-length used to disambiguate in some cases?

Usually we try to make the matching specific. But sometimes we apply a max-length limit slightly larger than the size of the script.

[ameshkov]

@Alex-302 generally, it does not matter much. Also, if there will be regex support, you can do something like that: $$script:contains(/.*stringtosearch.{10,1000}/)

[Alex-302]

Ok, that's what I meant by the regexp question.

[ngorskikh]

tsurlfilter task: AdguardTeam/tsurlfilter#96
Knowledge base PR: AdguardTeam/KnowledgeBase#264

[Yuki2718]

Is :last-of-type supported? If I use on Firefox ext, the site never loads. I wanted to use to fix https://github.com/AdguardTeam/AdguardFilters/issues/183252 but gave up. #1008 is not a duplicate unless you actually support position-based argument like the above or +.

[ameshkov]

No, :last-of-type is not supported. We don't bake a full-scale CSS engine inside CoreLibs.

We may extend it gradually, but please open new issues for every case so that we could choose what to implement.
Considering that CoreLibs use a SAX parser and does not build a DOM tree, implementing complete CSS is not an option. If we chose that way we would have been forced to buffer the whole page HTML inside CoreLibs and this will cause a noticeable delay in page rendering.