readme.md

scrape-it

A web scraping API for humans

Quick Start

Welcome to the quick-start! Below are some examples for common ways of using the API via cURL.

Also be sure to check out the full reference of API endpoints.

Intro

This API is a hosted version of the open source scrape-it library, and the API usage is exactly the same. The examples here show how to use the same API via an endpoint.

Scraping Hacker News

curl -X POST \
  'https://api.saasify.sh/1/call/dev/scrape-it@b2e60ee3/index' \
  -H 'content-type: application/json' \
  -d '{
  "url": "https://news.ycombinator.com",
  "opts": {
    "posts": {
      "listItem": ".athing",
      "data": {
        "title": ".storylink",
        "href": {
          "selector": ".storylink",
          "attr": "href"
        }
      }
    }
  }
}'

Returns a JSON payload

{
  "posts": [
    {
      "title": "FBI designates FaceApp as counterintelligence threat [pdf]",
      "href": "https://www.democrats.senate.gov/..."
    },
    {
      "title": "0.30000000000000004",
      "href": "https://0.30000000000000004.com/"
    }
  ]
}

Hitting our public rate limit? Consider upgrading to remove all rate limits.

License

MIT © Saasify