This repository has been archived by the owner on Aug 12, 2020. It is now read-only.
Releases: industrydive/smt-scrapy
Releases · industrydive/smt-scrapy
Export into chunks
This version allows (forces) you to export the SMT content in chunks, based on a given limit and offset when calling the craw_smt.py script
Export from database URLs
Combines the SMT Drupal database with scrapy framework to feed URLs from the database directly to scrapy's start_urls