[Feature Request] non-domain indicators parser for /maltrail-malware-domains.txt

[MikhailKasimov]

Inspired by: stamparm/aux#8

[i] Target: https://raw.githubusercontent.com/stamparm/aux/master/maltrail-malware-domains.txt
[✓] Status: Retrieval successful
[✓] Parsed 290973 exact domains and 0 ABP-style domains (ignored 13 non-domain entries)

To have a filter\parser for non-domain indicators when collecting domains from maltrail static trails to /maltrail-malware-domains.txt:

maltrail static trails --> [non-domain indicators] --> /maltrail-malware-domains.txt

In case of non-domain indicator != 0, to send emails to MS and MK (e.g. ...| mail -s "non-domain entries found" email@email.email).

This should speed up correction, if junk/orphan records are going to get incorporate into /maltrail-malware-domains.txt.

[MikhailKasimov]

Domain validator (preliminary): ^((?!-)[A-Za-z0-9-]{1,63}(?<!-)\.)+[A-Za-z]{2,7}$

[MikhailKasimov]

mail.tsinghua.institute <-- ^((?!-)[A-Za-z0-9-]{1,63}(?<!-)\.)+[A-Za-z]{2,9}$

[MikhailKasimov]

https://data.iana.org/TLD/tlds-alpha-by-domain.txt <-- ^((?!-)[A-Za-z0-9-]{1,63}(?<!-)\.)+[A-Za-z-0-9]{2,24}(?<![0-9-])$

[MikhailKasimov]

Relative reports on invalid domains:

stamparm/aux#8
stamparm/aux#10

[MikhailKasimov]

@stamparm Any implementations for /maltrail-malware-domains.txt ?

[dspruell]

@MikhailKasimov @stamparm Hey. 👋 I noticed this issue may be applicable today when attempting to load data from maltrail-malware-domains.txt into a DNS server for content filtering. A number of invalid domains are in the data:

$ fgrep .. maltrail-malware-domains.txt
saldirmorukss222..net
saldirmoruksas282..net
saldirmorukss122..net
saldirmoruks6s22..net
saldirmoruk7ss22..net
nj1337..ddns.net
saldirmoruk4ss22..net
saldirmoruks8s22..net
saldirmorukss2322..net

The double dots would need to be cleaned out or collapsed to a single dot, but figured the validation/parser might be applicable. Let me know if you'd prefer this as a separate issue.

[MikhailKasimov]

@MikhailKasimov @stamparm Hey. 👋 I noticed this issue may be applicable today when attempting to load data from maltrail-malware-domains.txt into a DNS server for content filtering. A number of invalid domains are in the data:

$ fgrep .. maltrail-malware-domains.txt
saldirmorukss222..net
saldirmoruksas282..net
saldirmorukss122..net
saldirmoruks6s22..net
saldirmoruk7ss22..net
nj1337..ddns.net
saldirmoruk4ss22..net
saldirmoruks8s22..net
saldirmorukss2322..net

The double dots would need to be cleaned out or collapsed to a single dot, but figured the validation/parser might be applicable. Let me know if you'd prefer this as a separate issue.

Fixed:

feeb969
d48bf83

Pretty thank you for pointing this problem out!

Let me know if you'd prefer this as a separate issue. <-- let it be in current one.