Writing a module

Let's implement a module for downloading domains associated with the Zeus trojan.

First we create a ZeusTracker item in the config.json:

            "ZeusTracker": {
                "URL": "https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist",
                "DESCRIPTION": "ZeusTracker",
                "ENABLED": true
            }

The important part is the URL key, it contains the URL from where we will download the blocklist.

Next we create the ZeusTracker.py (filename needs to be the same as the key in config.json) in the module directory and create the basic structure of the module:

from dnsintel.lib.abstractbase import AbstractBase
from dnsintel.lib.sqlpeewee import MalwareDomains

from logzero import logger

class Module(AbstractBase):

    def __init__(self):
        super().__init__()

    def transform(self, path: str):
        pass

    def run(self, config):
        pass

Now that we have a basic layout, we can start writing some code, let's begin with the run method. The run method is the method that actually gets called by dnsIntel, it does three things:

• Load the URL(S) from the configuration file
• Transform the downloaded data, meaning, do all the filtering needed to extract the actual domains
• Extract the domains found and insert them to the database

1.    def run(self, config):
2.       file = self.load(config)
3.        if file:
4.            data = self.transform(file.location)
5.            self.extract(data)

At line 1 we actually download the blocklist from the URL key in the config.json. Line 4 takes the path to the downloaded file as a parameter and does its filtering. Finally, at line 5, we take the filtered domains and insert them to the database. Easy as 1 2 3 :)

Now, for the transformation part, we do whatever we need to filter out the domains.

    def transform(self, path: str):
        try:
            with open(path, "r") as file:
                for line in file:
                    if line.startswith("#"):
                        continue
                    if len(line.strip()) == 0 :
                        continue

                    line = line.replace("\r", "").replace("\n", "")
                    domain = MalwareDomains(domain=line, type="Zeus", reference="ZeusTracker")
                    yield domain

        except IOError as e:
            logger.error(e)

There you go, now you have created your own module! Final script below:

from dnsintel.lib.abstractbase import AbstractBase
from dnsintel.lib.sqlpeewee import MalwareDomains

from logzero import logger

class Module(AbstractBase):

    def __init__(self):
        super().__init__()

    def transform(self, path: str):
        try:
            with open(path, "r") as file:
                for line in file:
                    if line.startswith("#"):
                        continue
                    if len(line.strip()) == 0 :
                        continue

                    line = line.replace("\r", "").replace("\n", "")
                    domain = MalwareDomains(domain=line, type="Zeus", reference="ZeusTracker")
                    yield domain

        except IOError as e:
            logger.error(e)


    def run(self, config):
        file = self.load(config)
        if file:
            data = self.transform(file.location)
            self.extract(data)