Releases: fredwu/crawler
Releases · fredwu/crawler
v1.5.0
v1.4.0
- [Added] Allow multiple instances of Crawler sharing the same queue
- [Improved] Logger will now log entries as
debug
orwarn
Full Changelog: v1.3.0...v1.4.0
v1.3.0
- [Added]
:store
option, defaults tonil
to save memory usage - [Added]
:max_pages
option - [Added]
Crawler.running?/1
to check whether Crawler is running - [Improved] The queue is being supervised now
Full Changelog: v1.2.0...v1.3.0
v1.2.0
v1.2.0 [2023-09-29]
- [Added]
Crawler.Store.all_urls/0
to find all scraped URLs - [Improved] Memory usage optimisations
Full Changelog: v1.1.2...v1.2.0