Scratching my on this one...

[micjgam]

So, when I run the following code, I get no results. The app goes to sleep (built in with time.sleep). I KNOW that I should be getting results wit the keyword (there is post listed fourth in line on the page that has the keyword in it!). I tested it with a base script and got an immediate return.

There are no syntax errors with the script... I can only wonder if it is in the comparison with c_date and post['time']? Or maybe there is a timeout and my script is too long? A few days ago, I was getting specific results as coded and then... nothing. Things seemed to have changed when I introduced datetime into the script; I have a while loop setup to run this during a specific time in the day, but that has been debugged.

      c_date = dt.datetime.now()
      posts = list(get_posts(group=os.getenv('FB_GROUP'), cookies='fb_cook.json', pages=3))
      posts = sorted(posts, key=lambda x: x['time'], reverse=True)
      keyword = os.getenv('KEYWORD')
      
      for post in posts:
          if keyword in post['text']:
              if c_date < post['time'] <= 2:
                  post_date = str(post['time'])
                  poster = post['username']
                  posting = reft(post['text'][:75])                     #reft is the var used for ascii conversion of post['text']
                  print(f'{post_date}\n{poster}\n{posting}')

Here is the base script and it works just fine and it's only one page...

      from facebook_scraper import get_posts
      from pydotenv import env_var
      import os
      
      env_var()
      for post in get_posts(group=os.getenv('FB_GROUP'), cookies='fb_cook.json', pages=1):
          print(post)

[neon-ninja]

c_date < post['time'] <= 2: doesn't make any sense. You're checking that the post is both from the future (current time less than post time) and less than 2 (you can't compare a datetime with an int so this should throw an Exception)? I think you mean - not <. You also need parenthesis for order of operations. See #320 (reply in thread) for example

[micjgam]

Yeah... I wondered about that line. I misread the datetime documentation for the comparison operator.

Thank you for your help again. Code is fixed and up and running.

[micjgam]

Alright... so I cannot figure out why I keep getting temp banned... I'm only scraping 3 pages and I am waiting more than two hours to run instances...

      posts = list(get_posts(group=os.getenv('FB_GROUP1'), cookies='fb_cook.json', pages=3))
      posts = sorted(posts, key=lambda x: x['time'], reverse=True)

      for post in posts:
          with open('words.txt') as keyword:
              keyword = keyword.read().splitlines()
              for kw in keyword:
                  if kw in post['post_text']:
                      post_date = str(post['time'])
                      poster = post['username']
                      posting = reft(post['post_text'][:75])
                      c_date = dt.datetime.now()
                      delta = c_date - post['time']
                      delta = delta.total_seconds()
                      if delta <= float(172800):
                          print(f'{post_date}\n{poster}\n{posting}')

Everything checks out in the script...

This was working a few days ago and nothing has been changed. I have a in-development version that I continue to make changes on and run as well, but I am not running side by side instances.

[neon-ninja]

Maybe try set options={"allow_extra_requests": False}, as it looks like you don't need HQ images

[micjgam]

True... I'll add it in. Other than that, any idea why this would continue? I just tried again, and within 10 seconds, I was banned. Does this method somehow keep a running tally of IP bans?

Which IP is being banned? Can I run this program from another account on a another comp within my network?

[neon-ninja]

You must be making too many requests, somehow. Facebook keep records of how many requests you made with which set of cookies & from which IP, and if you go over a certain amount of requests, you get temp banned. The scraper doesn't keep any records of IP bans, it only raises the exception when it sees the message served by Facebook servers. It's not possible for the scraper to tell whether it's an account ban or an IP ban - try switching IPs or exporting some cookies from a fresh login, and see if that makes a difference.

[micjgam]

yeah... I just switched to my wife's account... seems to be running fine now, though I am still on the same comp...

going to let my account rest for a while...

I was actually wondering if FB was keeping a running tally on bans. Just a probing question and you may not really know.

Thank you!

[neon-ninja]

Only Facebook engineers can answer that question, but from my experience Facebook is generally pretty forgiving when it comes to temp bans. After a certain amount of time has passed, everything seems to work fine again.

[micjgam]

Just some data for you...

Been running this now for almost six hours... first 4 hours was good and then starting get the temp bans. I added additional time in the script to allow for the bans, but apparently not enough.

The increase in runtime at the bottom is due to the temp ban buffer (15 additional minutes). I will increase the temp ban buffer to 1 hour and see the results.

 program runtime: 123 seconds
 sleeping for an hour...

 program runtime: 73 seconds
 sleeping for an hour...

 program runtime: 112 seconds
 sleeping for an hour...

 program runtime: 76 seconds
 sleeping for an hour...

 TEMPORARY IP BAN
 program runtime: 949 seconds
 sleeping for an hour...

 TEMPORARY IP BAN
 program runtime: 906 seconds
 sleeping for an hour...

[micjgam]

I find it disturbing that I can refresh my cookies after more than 2 hours, run the program and still have a ban, but I can switch cookies to my wife's account and run the same script (on the same computer) five minutes later with no ban.

What IP are they banning exactly? I like collecting data to help myself and others understand the environment we're in and make it work/run better.

[neon-ninja]

Then it's an account ban, not an IP ban