How to use cursor endpoint?

[shoaibiqbalk]

Hi! I have been using this framework and it's very good for scraping tweets. However, I want to use it daily to scrape and keep a record of the tweets of a few users. For example, If I scrape 10 latest tweets of the current date (today) and then 10 latest tweets of the next day (tomorrow) and so on. How should I use the cursor endpoint in such a case? Is the cursor endpoint describes the cursor for starting tweet or ending tweet? If it is for the starting tweet, then on the next run, the cursor must stop before that tweet. A little guidance will be appreciated.

[iSarabjitDhiman]

Hey @shoaibiqbalk
Check this #7 for the cursor_endpoint use cases.

Here is how it works:
Say you are extracting tweets from a user's profile and there are total of 5000 tweets. You managed to fetch like 2000 results and script stopped for some reason (api limits exceeded or you chose to stop the script for a while). Now the next time you want to continue you can use the cursor_point to resume from where you left off, in this case from 2000 tweets. Note any latest posts/status which were tweeted in the meantime, may not be included in this. Its like you are scrolling a page and you are half way, and next time you start scrolling again, there might be latest tweets at the top of the page. Hope it helps.

Also, if you want to extract results between a given period of time i.e. from some specific date til some given date. 10 July - 20 July. You can use the async-await branch for that purpose, example code. Check out this #6 for an overview.

[shoaibiqbalk]

That's very helpful of you. I have checked #6 and It gives me my desired results.