GitHub - yenngvp/ecommerce-product-scrawler: A python (scrappy) robot to crawl online products from e-commerce sites

yenngvp / ecommerce-product-scrawler Public

Notifications You must be signed in to change notification settings
Fork 0
Star 0

A python (scrappy) robot to crawl online products from e-commerce sites

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 31 Commits
.settings		.settings
frontier		frontier
productinfo		productinfo
.project		.project
.pydevproject		.pydevproject
LICENSE		LICENSE
README.txt		README.txt
db_export_import.sql		db_export_import.sql
scrapy.cfg		scrapy.cfg
seeds_es_smp.txt		seeds_es_smp.txt
server_start_stop.sh		server_start_stop.sh

Repository files navigation

# Author
Yen Nguyen - yen.ngvp@gmail.com

# Project: Ecommerce produts crawling project

# Features
* Fast
* Resource efficient
* Building spiders for crawling products from top e-commerce web sites in Vietnam.
Then the massive product data will be used to build a social network app for consumers, shoppers to find any selling product information, do price comparison and ask other past buyers for consulting buying decision and criteria.

# Technology
* Python
* Scrapy
* MySQL
* Redis
* Amazon
* Google Computing Platform

# LICENSE
MIT