Skip to content

nickolyamba/puppeteer-crawler

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

33 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Puppeteer Based Website Crawler

Building Project

  1. Install NodeJS
  2. Run $ npm install to install dependencies
  3. Add a base url and DOM selector that both match your use case to ./config/default.js. Base url is a url for a webpage that contains a list of urls you would like to crawl
  4. Run $ npm run start to develop the project on a local machine
  5. Run $ npm run build to build the project and $ npm run serve to serve it