豆瓣电影 Top 250 爬虫
This script is used to crawl the movie information of Douban Movie Top-250, including movie name, movie link, director, starring, release time, production country/region, type, rating, number of reviews and introduction. 豆瓣电影 Top-250 爬虫,爬取豆瓣电影 Top-250 的电影信息,包括电影名称、电影链接、导演、主演、上映时间、制片国家/地区、类型、评分、评价人数、简介等信息。
- Support command line interface and GUI interface;
- Support fast mode, get director and actor information from the top-250 list page directly;
- Support resume from the specified page;
- Support saving data to csv file;
- Support setting sleep time between requests.
- 支持命令行界面和 GUI 界面;
- 支持快速模式,直接从 Top-250 列表页获取导演和演员信息;
- 支持从指定页码开始爬取;
- 支持保存数据到 csv 文件;
- 支持设置请求间隔时间。
- Python 3.6+(3.9)
- requests~=2.31.0
- beautifulsoup4~=4.12.3
- lxml~=5.2.1
- tqdm~=4.66.2
douban_movie.py [-h] [-p PATH] [-s SLEEP] [-r {1,2,3,4,5,6,7,8,9,10}] [-f] [-g] [-v]
example:
python douban_movie.py -p data.csv -s 5 -r 4 -f
: 将数据保存到 data.csv 文件中,每次请求间隔 5 秒,从第 4 页开始爬取,使用快速模式python douban_movie.py -g
: 使用 GUI 界面python douban_movie.py -v
: 显示版本信息
optional arguments:
-h, --help show this help message and exit
-p PATH, --path PATH Relative path to save the csv file
-s SLEEP, --sleep SLEEP
Sleep time between requests(unit: s)
-r {1,2,3,4,5,6,7,8,9,10}, --resume {1,2,3,4,5,6,7,8,9,10}
Resume from the specified page, value=1: no resume; in [2, 10]: normal resume
-f, --fast Fast mode, get director and actor information from the top-250 list page directly
-g, --gui Use GUI interface
-v, --version show program's version number and exit
“断点续爬”功能 Resume from the specified page:
修改请求间隔时间&快速模式 Sleep time between requests&Fast mode:
帮助&版本信息 Help&Version: