README.md

WebRunner

This scraping/crawling and website cloner tool will help you to extract all url, emails on a website, allows you create your own searches using RegEx on a website and create a website copy for your phishing labs (educational purposes only)

Modules

url -Extract url from a website

email -Extract emails from a website

regx -Allows you to create your own searches using RegEx on a website

clone -Allows you make a website copy to local folder (usefull for phishing labs) # educational purposes only

help -Help about any command

install requirements

pip3 install -r requirements.txt

show module help menu

 python3 WebRunner.py [module] -h 
 example: python3 WebRunner.py url -h

optional arguments:

  -u, --url                 set target url
  -s, --string              set RegEx query
  -a, --user-agent          set user-agent 'DirRunner v1.0' by default
  -t, --threads             set threads
  -h, --help                show this message
  -c, --cookie              set cookies to use for the requests
  -k, --no-tls-validation   skip TLS certificate verification
  -n, --name                set project name 
  -f, --folder              set destination folder 
      --timeout             HTTP Timeout (default 15s)

CLONE

 python3 WebRunner.py clone -u https://www.domain.com -n mysite

URL extractor mode

  python3 WebRunner.py url -u https://www.domain.com

Emails extractor mode

  python3 WebRunner.py email -u https://www.domain.com

RegEx mode, example for extract THM{T3ST_M3SSAG3} on a website, use https://regexr.com for help

  python3 WebRunner.py regx -u https://www.domain.com -s "THM[A-Z0-9_{}]{6,}"