Skip to content

Latest commit

 

History

History
74 lines (49 loc) · 2.54 KB

README.md

File metadata and controls

74 lines (49 loc) · 2.54 KB
Raw

Web Scraper Hits

Web Scraper is a simple Flask-based API that allows users to retrieve meta data from any website by passing its URL as a request parameter.

How to Use the API

To use the API, simply make a GET request to the following endpoint:

https://web-scrapper-coral.vercel.app/api/read_url_meta?url={website_url}

Here, {website_url} refers to the URL of the website whose metadata you want to retrieve. The API returns a JSON object containing the website's title, description, and image etc.

For example, to retrieve the metadata for https://www.theverge.com, you would make the following request:

https://web-scrapper-coral.vercel.app/api/read_url_meta?url=https://www.theverge.com

Project Setup To set up the project, follow these steps:

  • Clone the repository to your local machine.
git clone https://github.com/TheAlphamerc/web-scrapper
  • Change into the project directory.
 cd web-scrapper
  • Create a virtual environment and activate it.
python3 -m venv venv

`source venv/bin/activate

  • Install the project dependencies.
pip install -r requirements.txt
  • Setup the development environment by running these commands in your terminal.
export FLASK_APP=index.py
export FLASK_ENV=development
  • Run the Flask development server.
python app.py

The API will be accessible at http://127.0.0.1:5000/api/read_url_meta?url={website_url}.

Contributing Contributions to the project are welcome. If you'd like to contribute, please create a pull request with your changes.

Created & Maintained By

Sonu Sharma (Twitter) (Youtube) (Insta) (Dev.to) Twitter Follow

If you found this project helpful or you learned something from the source code and want to thank me, consider buying me a cup of ☕

You can also nominate me for Github Star developer program https://stars.github.com/nominate

Visitors Count

Loading