Iterate Through Pages?

[brandon-scott-pritchard]

Hi again. Figured I'd start a discussion instead of raise an issue here.

It appears the if you set too many pages or set too many posts per page, and then try to scrape a page that has fewer posts, the process will fail.

I could set page_limit=None, but doing so would pull a LOT of posts from pages that are much older/more active.

Is there a way built in to set a page limit or set a certain number of posts per page and keep the process running if there are less posts than what I originally asked for?

[brandon-scott-pritchard]

Never mind, I just realized it was pulling everything available.