Skip to content

Commit

Permalink
Run parked domains check daily instead of weekly
Browse files Browse the repository at this point in the history
  • Loading branch information
jarelllama authored Dec 20, 2024
1 parent 68b9e5a commit 1f5ead3
Show file tree
Hide file tree
Showing 3 changed files with 12 additions and 13 deletions.
9 changes: 4 additions & 5 deletions .github/workflows/auto_retrieve.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,10 @@ jobs:
if: ${{ ! cancelled() && needs.test.result == 'success' }}
uses: ./.github/workflows/check_dead.yml

# Moved to weekly workflow
#prune-parked:
# needs: [test, prune-dead]
# if: ${{ ! cancelled() && needs.test.result == 'success' }}
# uses: ./.github/workflows/check_parked.yml
check-parked:
needs: [test, check-dead]
if: ${{ ! cancelled() && needs.test.result == 'success' }}
uses: ./.github/workflows/check_parked.yml

validate:
needs: [test, check-dead]
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/check_parked.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@ name: Check for parked domains
run-name: Check for parked domains
on:
workflow_dispatch:
schedule:
#schedule:
# Run weekly to give external organisations time to flag NRDs.
- cron: '0 4 * * 0'
#workflow_call:
# - cron: '0 4 * * 0'
workflow_call:
permissions:
contents: write

Expand Down
10 changes: 5 additions & 5 deletions scripts/update_readme.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

# Updates the README.md content and statistics.

# TODO: mawk '{sum += $1} END {print sum}' can be used to print 0 when there is no value.
# Note: mawk '{sum += $1} END {print sum}' can be used to print 0 when there is no value.

update_readme() {
cat << EOF > README.md
Expand Down Expand Up @@ -123,16 +123,16 @@ Resurrected domains added today: $(mawk "/${TODAY},resurrected_count/" "$DOMAIN_
### Parked domains
Parked domains are removed weekly. A list of common parked domain messages is used to automatically detect these domains. This list can be viewed here: [parked_terms.txt](https://github.com/jarelllama/Scam-Blocklist/blob/main/config/parked_terms.txt).
Parked domains are removed daily. A list of common parked domain messages is used to automatically detect these domains. This list can be viewed here: [parked_terms.txt](https://github.com/jarelllama/Scam-Blocklist/blob/main/config/parked_terms.txt).
Parked sites no longer containing any of the parked messages are assumed to be unparked and are included back into the blocklist.
> [!TIP]
For list maintainers interested in integrating the parked domains as a source, a list of weekly-updated parked domains can be found here: [parked_domains.txt](https://github.com/jarelllama/Scam-Blocklist/blob/main/data/parked_domains.txt) (capped to newest 50000 entries).
For list maintainers interested in integrating the parked domains as a source, the list of parked domains can be found here: [parked_domains.txt](https://github.com/jarelllama/Scam-Blocklist/blob/main/data/parked_domains.txt) (capped to newest 50000 entries).
\`\`\` text
Parked domains removed this month: $(mawk "/${THIS_MONTH},parked_count/" "$DOMAIN_LOG" | csvcut -c 3 | mawk '{sum += $1} END {print sum}')
Unparked domains added this month: $(mawk "/${THIS_MONTH},unparked_count/" "$DOMAIN_LOG" | csvcut -c 3 | mawk '{sum += $1} END {print sum}')
Parked domains removed today: $(mawk "/${TODAY},parked_count/" "$DOMAIN_LOG" | csvcut -c 3 | mawk '{sum += $1} END {print sum}')
Unparked domains added today: $(mawk "/${TODAY},unparked_count/" "$DOMAIN_LOG" | csvcut -c 3 | mawk '{sum += $1} END {print sum}')
\`\`\`
## Resources / See also
Expand Down

0 comments on commit 1f5ead3

Please sign in to comment.