Skip to content

Commit

Permalink
Update README
Browse files Browse the repository at this point in the history
  • Loading branch information
Your Name committed May 26, 2024
1 parent e897fc8 commit 7bda4ca
Showing 1 changed file with 3 additions and 0 deletions.
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ Crawls web pages and prints any link it can find.
- user-defined headers, same as curl: `-header "ONE: 1" -header "TWO: 2" -header @headers-file`
- tag filter - allow to specify tags to crawl for (single: `-tag a -tag form`, multiple: `-tag a,form`, or mixed)
- url ignore - allow to ignore urls with matched substrings from crawling (i.e.: `-ignore logout`)
- subdomains support - allow depth crawling for subdomains as well (e.g. `crawley http://some-test.site` will be able to crawl `http://www.some-test.site`)

# examples

Expand Down Expand Up @@ -94,6 +95,8 @@ possible flags with default values:
suppress info and error messages in stderr
-skip-ssl
skip ssl verification
-subdomains
support subdomains (e.g. if www.domain.com found, recurse over it)
-tag value
tags filter, single or comma-separated tag names
-timeout duration
Expand Down

0 comments on commit 7bda4ca

Please sign in to comment.