Welcome To E-Commerce Crawler
You need to have these following packages in your environment :
*beautifulsoup4==4.9.0
*PyInquirer==1.0.3
*pyfiglet==0.7
*termcolor 1.1.0
To install packages in requirements.txt run ;
pip install -r requirements.txt
Crawler can operate with scraper, you need to add it to src/ folder as scraper.py . If you are using my scraper you also need to put csv_lib.py under src/ . Operetable vendors are registered under src/scrape_elements.py, if you need to add new vendor just provide every object members like default vendors. Default vendors : hepsiburada, vatan, gittigidiyor, n11, teknosa, mediamarkt, trendyol, amazon.tr
To run the program use;
python .\main.py
This crawler is open source version of my commercial crawler, if you are interested in more professional crawler for e-commerce websites contact me : berkay.kocak@hotmail.com If you have any issue or found a killer bug please open a github issue on the repository page.